I had an issue this morning where a user created a loop by connecting both ports on the IP phone in the office to the switch. Obviously that's a bad thing and will cause problems, but it essentially took the entire network down, and once I started thinking about it, that surprised me a bit.
This is a fairly large site, they have a 100Mbps WAN circuit, an MX84 and about 25 various MS series switches. We have a total of 6 VLANs configured at the site, 3 data and 3 voice. When I started getting reports from users on multiple VLANs, I suspected the WAN circuit, but I got low latency and no drops pinging the WAN router from my office. Pinging the Uplink IP of the MX84 showed long strings of timeouts with occasional replies. Same for any other IP I tried to PING at the site, regardless of VLAN.
At this point I figured it was a broadcast storm caused by a loop, so I sent someone to the site to track it down. Once we removed the loop everything returned to normal.
At this point, I started wondering why multiple VLANs were affected by a broadcast storm, and it occurred to me that it could have saturated the trunks between the switches. But when I started looking at the trunk links, I discovered that the switch that had the loop on it has a max 100Mbit link to the switch it is connected to, while most of the other trunk links are 1Gbps. I would have thought the <= 100Mbps link between the switch with the loop and the other switches would have reduced the impact.
The switches are running MS 12.28, RSTP is on, and the switch the MX is plugged into is configured with bridge priority 0, all others are 32768. I'm guessing spanning tree didn't help because of the switch in the phone, looking at the logs for the switch the phone was plugged into it looks like both ports would get disabled, then re-enabled around 4 minutes later. That seemed pretty consistent for the entire time the loop was present.
Am I just underestimating the impact of ~100Mbps of broadcast traffic coming from a switch, or is there something else I'm missing?
If you are running MS2x or above switches you can also use storm control to limit BUM traffic to a percentage of bw on a link to better survive these loops or even devices that are spamming the network.
@PhilipDAth The Access ports on the switch are configured with separate voice and data VLANs, so normally the phone would be on VLAN 6 and the data would be on VLAN 5. VLAN 5 is also the Management VLAN.
I don't manage the phones, but I downloaded the manual for the Avaya 9611G phones that we use. There don't appear to be any options for spanning tree, the only reference I found in the manual is the statement "Spanning tree frames are always exchanged between the LAN port and the PC port in all VLAN separation modes."