cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

SOLVED
Cloud11
Conversationalist

MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

Greetings - We noticed a multiple orders of magnitude drop in "Passersby" traffic counts between Jan.20 (122,420) and Jan.21 (1765). So I asked Meraki support for an explanation which went like this:

 

Meraki: we implemented a solution to this on January 21st to bypass this randomization and give an accurate representation of your passerby traffic (emphasis mine)

 

Me: Interesting – how did you bypass MAC randomization for iOS devices that have never authenticated to the WiFi network?

 

Meraki: For passerby traffic to be generated on the dashboard we don't need devices to connect only for them to send an association to any SSID/AP or the like and our APs see that. As far as what our development team did to workaround that MAC randomization feature, unfortunately, I have no visibility into that code base or actual inter-working of how that change was implemented and even if I did, I would be unable to provide you with it as it would be considered proprietary information.

 

Me: (mmkay) I still need to explain to my clients why Meraki has been providing misleading data for the last seven years and why this massive change in their analytics is now more accurate considering iOS has developed technology specifically to thwart this kind of data capture.

 

Still waiting on a response to this, but if anyone has any insight into this I'd be grateful.

 

Until then I will be telling clients that Meraki's solution was: "WE DIVIDED THAT BIG NUMBER BY 70!!!"

 

Thank you.

 

 

 

 

 

1 ACCEPTED SOLUTION

Accepted Solutions
PhilipDAth
Kind of a big deal

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

https://documentation.meraki.com/General_Administration/Cross-Platform_Content/Meraki_and_iOS_14_MAC... 

 

"You might see a decrease in “Connected” clients as devices transition to randomized MAC addresses because the Meraki dashboard will filter out all randomized MAC addresses by default.

 

You also might notice a decrease in “Passerby” and “Visitor” because the Meraki dashboard will also filter out all randomized MAC addresses for these categories. Please note Apple devices have been using a randomized MAC address for Probe Requests prior to the iOS 14 release.

 

Loyalty and Engagement graphs will be impacted in a similar way but will stabilize over time."

View solution in original post

12 REPLIES 12
BrandonS
Head in the Cloud

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

This may be helpful?  At least to read their extensive disclaimers around the accuracy of analytics due to MAC randomization. https://meraki.cisco.com/product-collateral/location-analytics-whitepaper/?file

 

This part kind of admits it is a difficult problem and so difficult to be accurate:

 

Location data is largely captured per device using that device's media access control (MAC) address as a unique identifier. As part of a privacy technique, some mobile operating systems have added functionality that attempts to randomize the WLAN MAC address a device uses, making it more difficult to track by solutions such as Meraki Location Analytics. As the number of mobile devices that implement randomization increases, solutions to detect and locate devices have changed. Meraki provides additional capabilities such as bluetooth information via the Meraki Scanning API, enabling Meraki customers to anonymously include wearable devices as part of their location analytics dataset.

Based on empirical evidence from Meraki's own experiments and those of our analytics partners. This behavior tends to vary greatly based on the operating system and which apps are installed on the phone for example, if a certain app is very active, it could cause a device that is asleep to probe several times a minute.


 

PhilipDAth
Kind of a big deal

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

https://documentation.meraki.com/General_Administration/Cross-Platform_Content/Meraki_and_iOS_14_MAC... 

 

"You might see a decrease in “Connected” clients as devices transition to randomized MAC addresses because the Meraki dashboard will filter out all randomized MAC addresses by default.

 

You also might notice a decrease in “Passerby” and “Visitor” because the Meraki dashboard will also filter out all randomized MAC addresses for these categories. Please note Apple devices have been using a randomized MAC address for Probe Requests prior to the iOS 14 release.

 

Loyalty and Engagement graphs will be impacted in a similar way but will stabilize over time."

View solution in original post

PhilipDAth
Kind of a big deal

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

ps. If you are interested in analytics, start looking at transitioning those clients across to MV Analytics instead.

BrandonS
Head in the Cloud

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

Good point, @PhilipDAth 

 

I am not deep into the analytics industry but it seems this may be where the industry is headed for accurate head counts that are important for retail and COVID occupancy and other things I am sure I am not thinking of.  

 

Counting bodies is probably better not only because of MAC randomization, but also the fact that many people have multiple devices like smart watches, wearables, etc. that are only going to increase. I think it will just become increasingly difficult to accurately count bodies attached to devices as device counts per person increase and (hopefully) privacy increases as well.

 

 

PhilipDAth
Kind of a big deal

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

The other is cost.

 

If you want to generate heat maps over floor plans, you need each part of the floor plan to be visible by 3 to 4 APs to be able to do triangulation.

 

You need 1 camera.

Cloud11
Conversationalist

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

@PhilipDAthFamiliar with the MV capabilities, but never tried IRL. What's your experience been in terms of accuracy, de-duping, range, etc?
 
 
We tend to focus on outdoor shopping centers so heat maps are better for show and tell vs. analysis or deriving meaningful information.
 
And considering our locations are mostly outdoors, positioning APs is more opportunistic vs. indoor grid deployments so accurate location data is basically unachievable at this point.
 
 
Appreciate your input!
 
Cloud11
Conversationalist

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

@BrandonSAgree with you 100%. We advise clients that these analytics should only be used directionally and it is NOT an accounting system. I wouldn't use WiFi as a reliable platform for any kind of Covid tracing/tracking unless you could map MACs to emails or MDNs for communication about proximity. Even then, I'd have a 200 page disclaimer about false positives and accuracy.

BrandonS
Head in the Cloud

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

Interesting thought about COVID proximity tracking.  I was imagining  just complying to 25% capacity type regulations and how that could be difficult when one person = more than one device.  

 

Anyhow, here is a one minute MV analytics video that you might like to see: https://www.youtube.com/watch?v=MgqbvIvJHcY

 

 

PhilipDAth
Kind of a big deal

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

What is IRL?

BrandonS
Head in the Cloud

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

IRL = "in real life", old man..  😉

PhilipDAth
Kind of a big deal

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

Accuracy = good

De-duping = a single camera can track unique humans

Range = Not so great.  I've been using them indoors up to maybe 20m.

 

WB
Here to help

Re: MAC IOS Randomization "Fix" results in 70x drop in Passersby traffic

Not a huge shock unfortunately! One of the primary reason for MAC address randomization coming in is to stop things like tracking analytics having a DB of a users movements. At the moment it's just 1x random MAC per-SSID but I'd expect that to become either per-connection or 24 hour rollover by default in the future.

 

Tough one for retail that previously leveraged those footfall/passerby stats and I'm not sure of how you can get around it!

Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.