Second attempt failed to upgrade MR52's, packet loss and slowness in switch after upgrade MAC Flap

Thagaman
Comes here often

Second attempt failed to upgrade MR52's, packet loss and slowness in switch after upgrade MAC Flap

Second attempt failed to upgrade MR52's, packet loss in AP's and slowness in switch after upgrade with MAC Flapping.  Everything works great on 25.13, but when I upgrade to 28.5, AP's show 100% loss when I ping them, and my Cisco 3850 switch runs super sluggish.  Tech Support helped me rollback to 25.13, but how to I get to a stable release?  The moment the AP's upgraded, and rebooted.  I see this in my Cisco 3850's, these mac flapping logs, and the MAC address is my MR52's.

 

Jan 10 18:26:08.134 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.2cb0 in vlan 72 is flapping between port Po24 and port Po1
Jan 10 18:26:08.571 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.30e2 in vlan 72 is flapping between port Po1 and port Po24
Jan 10 18:26:09.061 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.357a in vlan 72 is flapping between port Po1 and port Po24
Jan 10 18:26:11.991 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.22a0 in vlan 72 is flapping between port Po1 and port Po24
Jan 10 18:26:14.350 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.2cb0 in vlan 72 is flapping between port Po1 and port Po24
Jan 10 18:26:14.373 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.357a in vlan 72 is flapping between port Po1 and port Po22
Jan 10 18:26:14.375 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.30e2 in vlan 72 is flapping between port Po1 and port Po22
Jan 10 18:26:15.415 MST: %PLATFORM_MATM-4-QUEUE_OVERLIMIT: MATM dropped learn cache batch messages as queue limit has reached.
Jan 10 18:26:16.851 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.34f6 in vlan 72 is flapping between port Po1 and port Po22
Jan 10 18:26:22.065 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.357c in vlan 72 is flapping between port Po22 and port Po24
Jan 10 18:26:29.389 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.357a in vlan 72 is flapping between port Po22 and port Po24
Jan 10 18:26:29.443 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.2cb0 in vlan 72 is flapping between port Po1 and port Po24
Jan 10 18:26:29.622 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.34f6 in vlan 72 is flapping between port Po22 and port Po1
Jan 10 18:26:41.045 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.357a in vlan 72 is flapping between port Po24 and port Po22
Jan 10 18:26:42.033 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.357a in vlan 72 is flapping between port Po22 and port Po24
Jan 10 18:26:42.842 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.22a0 in vlan 72 is flapping between port Po1 and port Po24
Jan 10 18:26:44.041 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.32de in vlan 72 is flapping between port Po1 and port Po24
Jan 10 18:26:44.421 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.357a in vlan 72 is flapping between port Po22 and port Po24
Jan 10 18:26:44.579 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.2cb0 in vlan 72 is flapping between port Po24 and port Po22
Jan 10 18:26:44.615 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.34f6 in vlan 72 is flapping between port Po22 and port Po24
Jan 10 18:26:45.296 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.357c in vlan 72 is flapping between port Po24 and port Po1
Jan 10 18:26:47.057 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.32de in vlan 72 is flapping between port Po22 and port Po1
Jan 10 18:26:49.256 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.22a0 in vlan 72 is flapping between port Po24 and port Po1
Jan 10 18:26:53.641 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.3506 in vlan 72 is flapping between port Po22 and port Po1
Jan 10 18:26:59.503 MST: %SW_MATM-4-MACFLAP_NOTIF: Host e055.3dc0.357c in vlan 72 is flapping between port Po1 and port Po24

10 REPLIES 10
Thagaman
Comes here often

I have read other posts about MAC flapping, but there was nobody in the building, and no mac where roaming between AP's, and the MAC address is my AP.

Brash
A model citizen

Given what you've described, it's quite possibly a firmware.
But under the assumption it's not, if you follow the MAC flaps down through the port-channels and switches, which devices and ports to they isolate to?

Thagaman
Comes here often

The AP go from my Cisco 3850 to my Nexus 9k, and the MAC address I see in the logs is the MAC address of the MR52 AP.

ww
Kind of a big deal
Kind of a big deal

Are you using 1 or 2 lan ports of the ap?

Thagaman
Comes here often

2 Lan ports

ww
Kind of a big deal
Kind of a big deal

Maybe something is wrong with the bonding? I would move 1 or 2 ap's to a new network and try to get that working with meraki support.  try  fw upgrade in that new network and maybe disable 1 switchport to the ap

Thagaman
Comes here often

Thanks for the suggestion, I am  planning on removing the port aggregation from the MR52, and remove LACP from the switch.

Thagaman
Comes here often

I see flapping in 3850 and Nexus core switch.

 

2022 Jan 10 18:33:04 HQDC-N5672-A %STP-2-DISPUTE_CLEARED: Dispute resolved for port port-channel120 on VLAN0126.
2022 Jan 10 18:33:04 HQDC-N5672-A %STP-2-DISPUTE_CLEARED: Dispute resolved for port port-channel130 on VLAN0254.
2022 Jan 10 18:33:05 HQDC-N5672-A %STP-2-DISPUTE_DETECTED: Dispute detected on port port-channel110 on VLAN0112.
2022 Jan 10 18:33:05 HQDC-N5672-A %STP-2-DISPUTE_DETECTED: Dispute detected on port port-channel110 on VLAN0116.
2022 Jan 10 18:33:05 HQDC-N5672-A %STP-2-DISPUTE_DETECTED: Dispute detected on port port-channel110 on VLAN0071.
2022 Jan 10 18:33:05 HQDC-N5672-A %STP-2-DISPUTE_DETECTED: Dispute detected on port port-channel110 on VLAN0072.
2022 Jan 10 18:33:05 HQDC-N5672-A %STP-2-DISPUTE_DETECTED: Dispute detected on port port-channel110 on VLAN0076.
2022 Jan 10 18:33:06 HQDC-N5672-A %STP-2-DISPUTE_DETECTED: Dispute detected on port port-channel110 on VLAN0091.
2022 Jan 10 18:33:06 HQDC-N5672-A %STP-2-DISPUTE_DETECTED: Dispute detected on port port-channel110 on VLAN0081.
2022 Jan 10 18:33:06 HQDC-N5672-A %STP-2-DISPUTE_CLEARED: Dispute resolved for port port-channel110 on VLAN0071.
2022 Jan 10 18:33:06 HQDC-N5672-A %STP-2-DISPUTE_CLEARED: Dispute resolved for port port-channel110 on VLAN0072.
2022 Jan 10 18:33:06 HQDC-N5672-A %STP-2-DISPUTE_CLEARED: Dispute resolved for port port-channel110 on VLAN0076.
2022 Jan 10 18:33:06 HQDC-N5672-A %STP-2-DISPUTE_CLEARED: Dispute resolved for port port-channel110 on VLAN0081.
2022 Jan 10 18:33:06 HQDC-N5672-A %STP-2-DISPUTE_CLEARED: Dispute resolved for port port-channel110 on VLAN0091.
2022 Jan 10 18:33:06 HQDC-N5672-A %STP-2-DISPUTE_CLEARED: Dispute resolved for port port-channel110 on VLAN0112.
2022 Jan 10 18:33:06 HQDC-N5672-A %STP-2-DISPUTE_CLEARED: Dispute resolved for port port-channel110 on VLAN0116.

PhilipDAth
Kind of a big deal

That sounds like an LACP issue, a failure to negotiate.

 

I don't know anyone who uses LACP with the APs.  It's pretty hard to get an AP to deliver even 1Gb/s of wired throughput.

 

I bet if you disconnected those second cables your life would become easier with no performance difference ...

thanks for the input, appreciate the help.  Meraki's website suggested configuration with a 3850 is LACP.  Do AP's do PaGp?

Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.