port disconnect bursts?

[cabricharme]

What do "bursts" of port disconnects and reconnects signify or mean, as opposed to single such events?

(Can't wrap my mind around the idea that multiple port disconnects and reconnects can happen all within one second, and several series of such disconnects - over the course of several seconds.)

Multiple ports on multiple switches (in multiple locations) are reporting to have these "bursts" of disconnects over the course of several seconds, and we can't yet make sense of it. No other errors or issues on the ports - no apparent packet loss, CRC errors or anything else. Seems random - yet given we are not aggregating logs, can't run analytics on these events to check for patterns, rhymes or reasons.

MS120 and 130 switches. Asked Meraki support (case 12875760): "These events seen are related to the port going down/up" and that's it. Is "bursting" significant - they are not saying.

A sample of these events:

Mar 20 07:12:54	R12 Sw2	Port 22	Spanning Tree	Port RSTP role change	Port 22 designated→disabled
Mar 20 07:12:54	R12 Sw2	Port 22	Switch port	Port status change	port: 22, old: 10fdx, new: down
Mar 20 07:12:53	R12 Sw2	Port 22	Spanning Tree	Port RSTP role change	Port 22 disabled→designated
Mar 20 07:12:53	R12 Sw2	Port 22	Switch port	Port status change	port: 22, old: down, new: 10fdx
Mar 20 07:12:34	R12 Sw2	Port 22	Spanning Tree	Port RSTP role change	Port 22 designated→disabled
Mar 20 07:12:34	R12 Sw2	Port 22	Switch port	Port status change	port: 22, old: 10fdx, new: down
Mar 20 07:12:33	R12 Sw2	Port 22	Spanning Tree	Port RSTP role change	Port 22 disabled→designated
Mar 20 07:12:33	R12 Sw2	Port 22	Switch port	Port status change	port: 22, old: down, new: 10fdx
Mar 20 07:12:26	R12 Sw2	Port 22	Spanning Tree	Port RSTP role change	Port 22 designated→disabled
Mar 20 07:12:26	R12 Sw2	Port 22	Switch port	Port status change	port: 22, old: 100fdx, new: down
Mar 20 07:12:25	R12 Sw2	Port 22	Spanning Tree	Port RSTP role change	Port 22 disabled→designated
Mar 20 07:12:25	R12 Sw2	Port 22	Switch port	Port status change	port: 22, old: down, new: 100fdx
Mar 20 07:12:14	R12 Sw2	Port 22	Spanning Tree	Port RSTP role change	Port 22 designated→disabled
Mar 20 07:12:14	R12 Sw2	Port 22	Switch port	Port status change	port: 22, old: 10fdx, new: down
Mar 20 07:12:13	R12 Sw2	Port 22	Spanning Tree	Port RSTP role change	Port 22 disabled→designated
Mar 20 07:12:13	R12 Sw2	Port 22	Switch port	Port status change	port: 22, old: down, new: 10fdx

Thanks!

[alemabrahao]

It's a bit complicated, but basically it's something that could be caused by an STP event, a layer 1 problem, a network loop or even a bug.

What version are you running?

[cabricharme]

MS 16.9, MS 17.1.3.

We got 20+ locations with 1-4 switches in each. Spot checking 5-6 locations: these "bursts" happen in each one, fairly regularly. Do you think we have "STP events", layer 1 problems, network loops in each one?

[alemabrahao]

Yes, it is a possibility, I would start at layer 1, trying to change the cable or the switch port.

[cabricharme]

Already tried - no change.

The question is still: why the burts? I get that a single "flap" (port going down then going up) would generate 4 events - but why 8 or 12 or 16 of them, all in the span of a second or three?

Physically disconnecting a cable from a port, then reconnecting it - it takes a few seconds for the port to negotiate the link and light up. Yet here, the port is somehow able to process several layer 1 disconnects in under a second? Can't wrap my mind around that.

[alemabrahao]

Because the switch port is flopping, that's all.
There's no way to know exactly without knowing more about your network topology. It would be necessary to do a detailed analysis.

[IvanJukic]

Hi @cabricharme ,

Mar 20 07:12:53 R12 Sw2 Port 22 Switch port Port status change port: 22, old: down, new: 10fdx
Mar 20 07:12:26 R12 Sw2 Port 22 Switch port Port status change port: 22, old: 100fdx, new: down

If we focus just on these two event logs. We see that Whatever device is connected to Port 22 on Sw2. The Speed is fluctuating from 10 megabits per second to 100 megabit per second and then back again. This would only be due to Layer 1/Physical conditions. I.e. faulty, cabling, ports or port controller misbehaving.

Judging from my experience and the frequency that this is occurring. I would;

a.) Replace the cable first. Use a known working cable. I.e. already used from another device not having this issue. Then if it is still occurring,
b.) Connect a known working device to the same port. Then if it is still occurring on Port 22. It looks to be the switch port. Raise a case with Support.

If it stops, then move the affected device to another good working port. If the issue occurs, then you know it is the end device.

[michalc]

You are experiencing port flapping on port 22. Please verify physical layer. I recently described the event logs seen in my other post. 
I'd recommend verifying what is connected to that port and what is the behavior like. I see similar events (port: 22, old: down, new: 10fdx) when my desktop turns off and it has a Wake on LAN enabled. 

[cabricharme]

Got it, thanks.

What's connected: POS (point of sale), Polycom VoIP devices, scanners.

All through patch panels.

One would think if there was a layer 1 issue, it'd manifest itself by persistent outages and connectivity issues? Yet we are not seeing this.

Seems specific ports experience this more than others.

Sounds like I need get analytics going across all of our networks to get a sense of this.

[PhilipDAth]

This can also be caused by machines with power saving enabled or having EEE (Energy Efficient Ethernet) enabled.

https://en.wikipedia.org/wiki/Energy-Efficient_Ethernet