Stack going offline

[checkmate1984]

Hi all,

Randomly every 4-6 weeks only on a Wednesday and similar 11am time frame our only 1 stack of 3 switches goes offline.

Only fix is we unplug fibre cable and plug back in 5 seconds and the switches come back online

Happens to one bank of switches only. The rest of network is fine.

Anyone know what can be causing this? The event log is pretty slim sadly doesnt say much information

Have got snipet of logs 

Thanks in advance

[RWelch]

What hardware model are the stacked switches?
What firmware are they running?

If it were me, I'd look first at the uplink port to verify it's able to access the internet to include the native VLAN and allowed VLANs.

[checkmate1984]

We have C9300-48U model for that stack

F/w Version - CS 16.8

There is a update pending but at present we just scared as when the network was setup the stack took hours for it to come online and as we school have to wait for half term. Some stacks took like 3-4 hours!(not sure if this is normal!)

Ip is static assigned and same as all other stacks in terms but ill check the internet just in case its core switch related

[Inderdeep]

Are you using the original stack cables ? we see that kind of issue in earlier post 

[checkmate1984]

Should be original, as the kit all came from Cisco during our project in August. 

[PhilipDAth]

My initial guess is something to do with a protective measure shutting down the port, like spanning tree.

When you say the stack goes offline; what precisely goes offline?  Does it continue switching locally, but goes offline is just the dashboard?  Or does it go offline in some other way?

The fibre cable you pull out, is that the uplink?  What does it go to?  Does the device it goes to have anything interesting in its event log?

Whatever is the "core" switch/switch stack in the company, make sure it is configured as the spanning tree root (such as giving it a priority of 0).

https://documentation.meraki.com/MS/Port_and_VLAN_Configuration/Configuring_Spanning_Tree_on_Meraki_...

[PhilipDAth]

Also - do you have any non-Meraki switches in your environment (even Cisco Catalyst running native IOS-XE)?  If so, make sure they are configured to use "mst" spanning tree.

On a Cisco Catalyst you use this command:

spanning-tree mode mst

[DarrenOC]

Have you had TAC review the status of the switches at all?

Could be a process that gradually grinds to a halt or the reverse causing a spike which knocks the stack offline.

[BlakeRichardson]

If it's happening at the same time it sounds like something is timing out. I would start by talking to support. When you say the stack goes offline is it losing connectivity to the dashboard? 

Re firmware updates why not schedule for the middle of the night or is that not possible?

[checkmate1984]

Offline is including the end devices, with the switch lights turning amber. The problem seems to occur when either fibre cables used as uplinks to the core switch gets pulled out and back in. Oddly enough, if we unplug and then plug back in either fibre, the stack comes back online.

I'll need to check the root priority and understand the process before making any changes. Our network setup exclusively uses Meraki switches, so no non-Meraki devices are involved.

I've sent a TAC request but unfortunately, it's been over 24 hours with no response so far. Additionally, the connection to the dashboard is lost, and the switches show as offline.

Regarding the firmware, when we set up the switches in August, we often had to restart them multiple times to bring the stack online. They never seemed to come online straight away after a reboot without needing a hard reboot once or twice. Sometimes, we let one switch (the root of the stack) come online first, and once it's up, we plug in the others. Because of this, we're hesitant to take any risks in the evening