iOS IP Conflicts

[TBisel]

I have multiple sites sending alerts for IP conflicts recently. It is only iPhones, and only on our wireless. I have SSIDs and a VLAN setup dedicated just to mobile phones, and the MX running as DHCP server for that VLAN. This was not an issue until two or three weeks ago but we are getting two to three alerts for different sites almost every day. Anyone have any idea where to start looking on how to fix this? Is this another iOS bug?

[kYutobi]

Check network settings. iOS made an update that randomizes MAC addresses. 

https://community.meraki.com/t5/Wireless-LAN/MAC-Randomization-using-IOS14-and-Android-10-and-above/...

[BrandonS]

iOS 14 makes randomizes MAC addresses default.  You have some options.  You can turn that feature off in network settings on the devices, you can disable the annoying alerts from Meraki Dashboard.  You can try a shorter DHCP lease time.

There are probably other options too.  It is not an Apple bug or specific to Apple though. AFAIK, Android and Windows, etc. do this also, but may not be enabled at default.  Since Apple prides itself on their privacy and security posture it makes sense they turned it on at default.  

I know I am drifting off topic, but ultimately this is a "good thing" because MAC addresses are way too easy to connect to a person and that means good and bad actors can track you too easily and without your permission so it needs to stop somehow..

[Asavoy]

On Sept 1, Apple released the 13.7 update that pertains to Covid tracing. My guess, since it's only started happening in the last couple weeks and only to iPhones, is the update is intent on trying to keep the same IP address for continuity of tracking. We all know how that goes when it comes to DHCP requests.

ETA- iOS 14 is only a week old, so while the randomized MAC address might be the issue, it wouldn't have started a couple weeks ago.

[Gryffindor · Accepted solution]

This is from Apple and what is causing this issue.

Use private Wi-Fi addresses in iOS 14, iPadOS 14, and watchOS 7

To further protect your privacy, your iPhone, iPad, iPod touch, or Apple Watch can use a different MAC address with each Wi-Fi network.

To communicate with a Wi-Fi network, a device must identify itself to the network using a unique network address called a media access control (MAC) address. If the device always uses the same Wi-Fi MAC address across all networks, network operators and other network observers can more easily relate that address to the device's network activity and location over time. This allows a kind of user tracking or profiling, and it applies to all devices on all Wi-Fi networks.

To reduce this privacy risk, iOS 14, iPadOS 14, and watchOS 7 use a different MAC address for each Wi-Fi network. This unique, static MAC address is your device's private Wi-Fi address for that network only.

[wtesolutions]

Thanks - This is a big help.  We were wondering why we kept getting conflict alerts all of a sudden. 

[AlexP]

Hey everyone, MX Support Specialist chiming in here.

We opened up a case with Apple this morning, because it looks like iOS devices with this feature enabled are doing something a little weird with ARP requests:

If you look at frame 548 in that packet capture screenshot, you can see that it's an ARP response sourced from one MAC address, but saying that the IP in question belongs to a completely different MAC address.

Then, later on, in 964, you see it respond again, but this time it says the IP, correctly, belongs to the same MAC as what's seen in the Ethernet source address.

It's this disparity that's causing these duplicate IP alerts, because MX's rely on ARP mappings to verify that two devices aren't trying to both use the same IP address.

Once we have a better idea on how they plan to address this, I'll be sure to share what updates I can provide here.

[BrandonS]

Thanks @AlexP Good to know.  Subscribing to and bookmarking this post now..

Not totally off topic, but I read about a change to the way iOS 14 handles DNS today.  You might be interested: https://mailman.nanog.org/pipermail/nanog/2020-September/209823.html

Best.

[JohnD]

@AlexP I believe this is the way Apple handles Bonjour sleep proxying, but recently it seems to have extended to more devices than just Apple TVs and HomePods that respond to ARPs on behalf of a sleeping client. It seems like if a client wants to go into deep sleep, it picks another awake client to answer on behalf of its IP. 

[Eruch]

I notice Apple released an update in the last day or two. I just installed it and the notes referenced a bug fix that might help the phone get on a wifi network. Anyone know if this fixed our problem?

[AlexP]

Unfortunately, we tested iOS 14.0.1 this morning, and still noticed the same bug; we submitted our findings to Apple today, so hopefully that means we'll have some kind of update to share on this soon.

[AlexP]

Apple's currently investigating our report, and isn't asking for any new data. As before, once I have more to share, I will do so.

[J-Edge]

Thanks for following up @AlexP! Replying to this thread so that I can follow it and add my voice to the choir - this has been an issue for us as well. Hopeful for a resolution!

[kmr]

This is inline with my version of the issue.  The MAC address that iOS has assigned to another Wi-Fi network is the one Meraki shows as in conflict on my Z1's wi-fi network.  Checking settings on the iPhone, one of the private MACs is assigned to the Z1 wi-fi connection. The other MAC is assigned to another wi-fi connection. That other network doesn't assign the same IP address range, or anything like that. Looking forward to an iOS update!  

[AlexP]

Hey everyone,

We've confirmed that a fix for this is in iOS 14.2 Beta 3, so when the 14.2 release is finalized, that means this problem should hopefully start to drop away.

Here's hoping you're not dealing with too much more noise in the mean time!

[Stingburst]

I've tried to troubleshoot some of these and didn't make the connection but in hindsight, I do remember iPhones being the devices linked to these. That's good stuff to know! 

Thanks!

[stefbauer]

For those that are using Microsoft Intune to manage devices - and  have a configuration profile that is pushing the SSID/Login information from Intune via MDM - there is now an option in the configuration that allows for "Disable MAC address Randomization" - or as was previously suggested, you can ask your users to set that on their devices as suggested by apple: https://support.apple.com/en-us/HT211227 

[Green_Ghost]

Please be aware that the ability to disable MAC randomization via MDM profile may also be subject to an Apple side bug.

If the option to disable MAC randomization is selected, the user still has the ability to re-enable it within the UI. Meraki has also sent feedback for this issue as well.

This behavior is observable via profiles created directly within Apple Configurator which suggests that this may only be resolved through a future iOS update.

[TBisel]

Took DHCP leases down to an hour, still getting these stupid alerts.  Dont want to disable the alerts, but we currently do not have MDM. So looks like I am going to be losing alerting for this... Fun.

[Ryan_C]

This is creating issues. Some of the suggestions do not really work. If i have a guest network using meraki dhcp i can not disable random mac via MDM. Any way to adjust alerts so we can get ip conflict alerts based on dhcp scopes 

[TBisel]

I dont think there is a way to limit the alerts on a per-scope or VLAN. I think that might end up being the Meraki work around though. Seems like Android is going to do a similar thing. 

[AlexP]

Hey everyone,

So we're clear here, you should not need to be implementing any workarounds for this based on what is expected behavior on Apple's (and hopefully Android's) part - we're trying to test a new release from Apple that should hopefully have this resolved.

[shauno]

I’m seeing this behaviour even when MAC randomisation (Private Address setting on the iOS device) is Disabled.

Hopefully Apple has a fix soon.

[Kevin31]

Same here

[Kleingeld_B]

Very interested in a working solution... hope that Apple will supply an update to solve this issue...

[Green_Ghost]

Apple mentioned that iOS 14.2 beta 3 should have resolved this issue. We've been testing this build in the lab where we have no longer observed this behavior.

[TBisel]

Which part is considered "The Issue"? Is it the problem in general with Apple swapping MAC addresses, or is it that even with randomization turned off, its still happening? We don't have MDM, and walking people though settings is not realistic.

[Green_Ghost]

For reference, this is the issue.

iOS devices on some iOS 14 builds will randomly reply with an ARP message sourced from the expected MAC address, however the ARP payload will contain a different source MAC address. The MX will detect this potential conflict (in this example, MAC ending in ad:e0) and send an email alert.

This issue seems to be reported regardless of whether MAC randomization AKA "private address" is enabled on the iOS device.

[edit]: Once we have updated the devices to iOS14 beta 3, we have not observed this behavior at all. 

[edit 2]: Photo edited to clarify

[etb]

Please interpret this as a newb-ish reply, especially because I have not been able to physically get my hands on any of our Apple devices due to COVID, so I have had to rely on our users to disable the feature on their own.  But my anecdotal experience is that some of them had to disable the feature twice - they described it that the setting somehow did not save the first time.  Other users did not seem to have that issue.

In addition to that, I think in many cases I saw one final IP conflict alert per device after a user had successfully disabled the feature.  I assume that this was caused by the Apple device returning to using its factory MAC after it had been more recently using its randomized MAC.  YMMV.

[starbuck]

I presume this feature in iOS will also break policy settings on iOS devices right? So if I have certain users iPhones set to be whitelisted or have addtional bandwidth on our network, this will break that functionality, correct?  Or is this going to be resolved by the fix coming in iOS 14.2?

[Bruce]

If you had policy applied directly to an iOS device and they update to iOS 14 then you’ll see the ‘old’ device (I.e. the device’s real MAC) will never connect to the SSID again, and it will appear as a ‘new’ device (I.e. using its randomised MAC). So yes you will need to re-apply policies to the devices. The only way you can avoid this is if you can get the users to turn of MAC randomisation, or turn it off using an MDM (if you’re using one).

[AlexP]

Fair warning too, that devices with policies do not age out after 30 days continual of inactivity like those without them do either, so you may end up having to deal with cleaning up old policy entries as a result of this.

[AlexP]

The 14.2 update will only fix the duplicate IP alert issue - it will not change anything else as far as we're aware

[akfrnd]

Is there an ETA for the 14.2 release?

[Blake-M]

It looks like iOS 14.2 is expected before 11/13/2020. 

https://www.gottabemobile.com/ios-14-release-date/

[Bruce]

Its being pushed now - received the update notification for 14.2 on my iOS device yesterday.

[TBisel]

I really am starting to hate this company. To add to the trouble, Apple has released the update. And its getting flagged by Meraki and AMP as malware.... So the alerts have just begun... Yippy!

File downloads as
86ea5441393e1f0b96656bb5ad56364b308b7ca6.zip

From:
http://updates-http.cdn-apple.com/2020FallFCS/mobileassets/001-62165/F4052D84-DC41-414E-90A4-7E6E971...

And is identified in AMP for Endpoints as:
W32.6B9A473044-95.SBX.TG

With SHA of:
6b9a47304497324fe40e3ccfba43bbb056c152500550849d1597fc47b4ebab35