Spanning-Tree Broadcast Storm from Connecting APs

[ToryDav]

Hi all,

I have a question regarding MR. 

How can I prevent STP topology change issues when deploying new APs. I think this should not even have happened, but here is what occured.

5x MR76 deployed onto the network. Ports are configured as trunk with a native vlan set and all specified for the allowed VLANs. 

Multiple SSIDs in use with differing VLAN tags on a per SSID basis.

APs were deployed, and the network must have looped through the APs as I notice they did come up as a repeater briefly when they first connected.

Then the network went down, had to disconnect all the inter-switch links from the core to stabilize and then slowly bring it back up. No physical looped cables are identified at this time.

Should this even be happening?

Also note:

Network is tied to a template
Swiches are bound to switch profiles

STP root is configured to be the 1st member of the MDF stack via the Profile on the stack, but does not appear to actually set the priority on that switch since it was clearly going through STP election when the APs connected. 

STP root changes between three different switches, and will not lock onto the switch configured in switch settings (via the profile)

[MarcP · Accepted solution]

Within Switch - Switch Settings - STP Configuration is enabled and your correct switch has got the lowest priority? 

Additional info, do not use 1 or something like this as this can be horrible when changing the root bridge.

<edit> Maybe you answered this question in here... https://community.meraki.com/t5/Switching/Configuration-Templates-switch-profiles-and-STP-root-bridg... </edit>

I very often see that new MRs become a bridge for a couple of minutes when first ever connect them to a network. But then they are just normaly online as they should.

Also connecting MRs should not effect the switches to change the RB. Or is one of the MR going to be the RB and the switches just show the direction to the switch, where the MR is connected? - Never saw that and don´t know if this can happen, tbh.

Remembering the case, when I got to a new company I saw the RB was a unmanaged switch for "mobile device scanners"... Meaning, not only "normal" switches can be the RB.

[ToryDav]

Hi @MarcP,

This post is related to the other that you have linked and commented on, so until work arounds are implemented, setting the root bridge isn't happening with the current setup where stacks are in play.

I very often see that new MRs become a bridge for a couple of minutes when first ever connect them to a network.


Yes this happened, I can in the dashboard they came up as a repeater looking for a gateway. This caused a network loop through the WiFi mesh. 

Also connecting MRs should not effect the switches to change the RB.
*Should* not, but we should not get loops though APs either. My thoughts are that since these are TRUNK ports, then link up/down causes a topology change, if they were ACCESS, portfast behavior prevents this. Topology Change then triggers STP election, since all switches are default priority. Thus as switches get overwhelmed, some stay up others go down, links flap, RB bounces around. 

Or is one of the MR going to be the RB and the switches just show the direction to the switch, where the MR is connected? - Never saw that and don´t know if this can happen, tbh.

MR was not RB, it was acting as an uplink. For example, the 8 port MS 120 where the new APs connected shows in the log that the only physical uplink the switch had changed from ROOT -> ALTERNATE

Then I see the switch root port bounce around between the AP ports.... others turning to alternate, some going from DESIGNATED -> DISABLED.

Then I see root bridge bounce between three switches. Loop caused a total outage, and the site had to strip the network down to the core stack and stack it back up one Inter-switch link at a time. There is no physical looped cable that can be identified, so if it is there, we have not found it yet.

This seems a little ridiculous but this is what I am seeing.

@MarcP What are your thoughts on using BPDU guard on the AP ports. Seems a bit unconventional, but should a BPDU cross that port in the future, the AP will be disabled and NO loop will form.

In my lab BPDU Guard has not affected traffic flowing and I rely on that AP for day to day use, but I cannot recreate the issue in my lab, I only have 1 AP. 

My thought is that we put BPDU Guard on all the trunk ports leading to APs, should this ever occur again where a BPDU crosses that link, the AP is shut down, no loop, and we can bring it back up manually.

Thoughts?

[MarcP]

I haven´t work with BPDU Guard so far, because I never had this problem. But why not test it at a hand full of specific ports?

Sounds good to me. 

[ToryDav]

@MarcP I have it on my lab AP  now for over 48 hours. I may suggest this for this one location as a one-off.

I think it's a little unconventional to use it in this way, logically it makes sense though. My only concern is if an AP goes into repeater and BPDUs are seen by ALL APs, then they all go into blocking and if this is during the day it will be a major issue, so I may not pursue it any further.

[cmr]

@ToryDav do you need meshing?  We commonly disable it and I think that would stop the issue occurring?

[ToryDav]

Hi @cmr 

Sometimes yes, and sometimes no. This particular location does not, but others do rely on meshing. This isn't something I can override, and there are several networks tied to configuration templates in this case. 

I can't disable it for this one network, doing so at the template level will affect other locations

[CarlosBernal]

Hello @ToryDav 

I kind of have same issue, the thing is I seeing a lot of STP logs but with one of my switches which I think is the root reason why a lot of DHCP requests are denied, I already tried everything I could but the issue is not solved at all. 

May I know how did you solved this?

[ToryDav]

I don't think STP would cause DHCP request to be denied. The DHCP discover messages may actually be getting lost, causing the DHCP process to fail.

Where is the DHCP service running in relation to the client? (What does the path look like?)
Filter your event logs for STP elections / root bridge changes, if you have an STP problem such as constant topology changes DHCP likely isn't the only thing affected.  Are there any other problems occurring? 

[CarlosBernal]

I have DHCP service running on the meraki MX, the path is: AP > IDF Switches > MDF Switch > MX

It was already solved, a dummy thing happened, I have two separated infrastructures, one with meraki and one with cisco. Before I installed the meraki ones, I had everything running on the Cisco AP but for mistake I was propagating the same SSID in both infras, one (meraki) had complete path to internet, the other one (cisco) was not connected to internet anymore and as clients were connecting randomly between Cisco and Meraki AP I had some having internet connectivity and other not, I just turn off the necessary SSID in cisco and leave it only in meraki. 

Again, it was a dummy thing but I think it is good to share in case anyone else have this issue hahaha