[Help] Windows client receiving incorrect MAC address of Gateway in the ARP table

[chuyendang]

Update with detailed information.

• Many Macbooks in the network keeps broadcast them as Gateway MAC that causes other Windows clients on the network could not access network.

• So far, only Windows clients received bad ARP.

• Macbooks, Mobile (iOS/Android) have not faced this problem.

• On the network, we do setup dhcp snooping (Meraki at both layer 2 and layer 3 and wireless)

• I have checked the Macbooks that broadcast ARP, but could not find anything special on them

• This is a wireless network with client isolation setup.

any input appreciated!

[BrechtSchamp]

Sounds like your DHCP server handing out the gateway's IP address to clients? Can you check the IP pool used by the DHCP server for that VLAN and ensure that it doesn't include the gateway address.

[chuyendang]

Yes, we have Gateway address on the DHCP pool.

Edit: The GW Address is excluded from the DHCP pool. It's weird as the problem only happens with Windows.

Mac, iPad, Android ... are fine.

[BrechtSchamp]

Well strictly speaking I believe clients should do an ARP request for the IP address a DHCP server proposes before they actually start using it to avoid duplicate IP addresses. But maybe this sometimes fails and maybe the bevavior differs between OSes. Either way, your DHCP address pool shouldn't include addresses that are statically assigned so you should rectify that.

[chuyendang]

Edit: My bad, I misunderstood.

I excluded the GW IP from the DHCP Pool already, so I don't think DHCP is not the cause...

[BrechtSchamp]

Some devices must be using the gateway's address for some reason. Do you know the devices behind those MAC addresses to check their configuration if that is indeed the case?

You could also use these instructions to try and locate them:

https://documentation.meraki.com/MX/DHCP/Troubleshooting_DHCP_Conflicts#Client_IP_Conflicts

There could also be a rogue DHCP server on the network, the switch should detect it and show it in Switch > DHCP servers & ARP.

[PhilipDAth]

I have a couple of thoughts.

• Any chance there is more than one DHCP server on this network (by accident)?
• Any chance you have a layer 3 router (not the default gateway) attached to this network and it is doing proxy arp?
• Any chance Windows Internet Connection Sharing is enabled on some machines?  This causes Windows to run a DHCP server and returns the client machine itself as the gateway (and in ARP replies)?

[chuyendang]

We have DHCP policy that blocks rogue DHCP servers by default.

On this SSID, we setup client isolation as well.

Wireshark showing that many Macbooks doing arp sniffing, however, when I look at these Macbooks, I don't see any abnormal..

[BrechtSchamp]

When you execute ifconfig in terminal on these MacBooks, does the address show up on one of the interfaces?

[chuyendang]

No, the Mac only shows it's actual IP. 

I think this is a similar case: https://mailman.nanog.org/pipermail/nanog/2019-March/100081.html 

[thangpa]

Hi @chuyendang,

I have the same issue with you.
Did you solved the issue?

[chuyendang]

Hi, No, not yet. Few other people having the same issue.

[thangpa]

Hi @chuyendang,

I opened a ticket with Meraki support.

He advise change to NAT mode ( - currently in Bridge mode ). But I cannot change to NAT mode because of our policy.

My colleague try to disable connectivity in Sleep mode to prevent this issue.

Could you add my skype for further discuss: thangphan205

Regards,

Thang

[chuyendang]

Other guys mention that disabling Wake on LAN could fix the problem. However, we cannot do this as we don't manage the device. Meraki should fix the problem at their end (though they implemented ARP proxy - https://documentation.meraki.com/MR/WiFi_Basics_and_Best_Practices/Broadcast_Suppression_and_Control...

You can check some information here: https://www.reddit.com/r/Cisco/comments/b6eiur/cisco_3802i_waps_change_capwap_gateway_due_to/

[redsector]

Has the DHCP server the correct router address in the Scope Options?

And if you have two DHCP-servers, is the standby server correctly replicated? I had issues when the primary DHCP- server didn´t replicate changes to the backup DHCP server.

[chuyendang]

Hi, 

Yes, the DHCP settings are correct.

Most of Macbooks broadcast ARP when they are in sleep mode, I ran  pmset -a disablesleep 1 to disable wake on Lan on a test machine and the problem have not happened with that Mac.

[chuyendang]

This is Cisco's note regarding this problem: https://www.cisco.com/c/en/us/support/docs/wireless/aironet-3800-series-access-points/214491-arp-res...

Any Meraki staff here? Can you take a look on this?

Thanks

[Christiangelo]

We are having the same issue right now on our wireless network. A lot of Android phones, and a couple of Windows laptops and Chromebooks are affected. Apple devices are fine and can connect without issues. In our case, MacBook Pro OSX 10's are sending their gateway addresses as response to an ARP request. I'm not quite sure if they are on sleep mode, though.. because they have a green icon on the clients page. Did you get any feedback from Meraki Support?

[chuyendang]

Meraki has a beta firmware for this problem as the email I got

Hello again,


Thank you for your patience!


I received an answer from the development team.
There is a test version of the firmware that should solve the issue.


Please, remember this version is not meant for public use, so it may show some unexpected behaviours.
The alternative is to wait for a future stable release of the firmware.


Please, if you wish to test it feel free to call our 24/7 support telephone line during a maintenance window of your network.


Kind regards,

[zd]

Is anyone that is seeing this issue running Avast as an antivirus software? We are experiencing this on Windows devices running Avast, but when we uninstall it completely the issue is resolved.

[DOIT]

We are having the EXACT same problem !!!!

Meraki, please HELP !!!!

I have opened a case !

[zd]

Do you know what the firmware version this is?

[Christiangelo]

Most offenders we see are from Mac OS X 10 (mostly 10.14). Those machines send an ARP reply to anyone (even Apples), but I guess some Windows and Android phones are having problem routing to the right gateway. Based on our traces, those replies were already tagged as duplicate but I not sure why they still route to them.
You might want to try Layer 2 LAN isolation on the SSID to drop those ARP replies.

[cmr]

26.5 has now been released and this is in the fixes section of the release notes:

Gateway IP was being spoofed by certain types of clients resulting in incorrect gateway IP information being propagated to other clients (All MRs)

It may resolve the issues mentioned above, be good to know either way!

[FiberOps]

Can anyone confirm that 26.5 fixed incorrect broadcast MAC address issue? Has anyone done any changes on client side Macbooks to disable broadcasting spoof of Gatway MAC? 

[Rodrigo_]

Hi There

The issue is actually on Apple side, however, Meraki has provided a fix for this not to affect clients.

I can confirm this is present in 26.5, now 26.6

[FiberOps]

Thanks for the update on this. 1 other question about this 26.5 firmware. In the known issues it states, "When upgrading from MR25.x firmware to MR26.x firmware, mesh repeater MRs may be slow to upgrade (802.11ac Wave 2 MRs)." 

Now when it mentions "may be slow to upgrade," how long should I expect for the upgrade to finish on these APs? We currently have 3 repeaters in our environment that meet the criteria for this bug. I would need a maintenance window and want to know how long I would need for the upgrade to complete. We run a 24/7 shop 7 days a week. 

[PhilipDAth]

>may be slow to upgrade

Assuming you have a decent Internet circuit for downloading the firmware, and the MESH repeaters have a decent connection to the wired node, I would think 20 minutes should be enough.

[FiberOps]

I ran command successfully, however, energy saver option still shows "wake for wi-fi network access" as still being enabled. When you ran the command did this option change for you or is it disabled regardless of whats showing in the energy saver menu?