Reboot switches in stack after firmware upgrade

[rvdtol]

Hi all,

Last weekend we upgraded our switches (MS425 / MS220) to the 9.32 firmware. Most of our switches are part of switch stacks (with stack modules or flexi stack configuartions) and I noticed that after the firmware upgrade all the switches in a stack rebooted at the same time. I was on the assumption that the switches that are part of a stack never rebooted all at the same time but one-by-one so redundant links from my server clusters and storage would stay up and running.

Does some have the same experience? I asked Meraki support and they told me that switches in a stack should reboot one-by-one and never at the same time while doing a firmware upgrade.

Looking forward hearing form you!

[redsector]

Same problem, now afraid to update to 10.6. Working with stacked MS225-48LP. Also had spanning-tree loops while rebooting with aggregated uplink ports over two stacked switches.

[rvdtol]

Did you even contacted Meraki about this? They told me that it was unexpected behaviour if switches in a stack rebooted at the same time

I also asked Meraki support a to send me a report with the exact time of when the switches did reboot (Dasboard is not able to give the precise time). Think of a 'show version' in Cisco classic equipment. Again, they were not able to provide this.....Do you have any tips  trick how to obtain this?

[MerakiDave]

Hey Team, Support is now aware of this issue and actively investigating, and seems to be limited to 220/320 models that have been running for a long time (long uptime) and when upgrading from a 9.x firmware.  You should see updated release notes shortly on this.  But for now, if you need to upgrade, the best way to mitigate this issue is simple but involves a double reboot.  Simply do a reboot of the switch prior to the actual firmware update (so you're first rebooting it with no changes and existing firmware), and then proceed with the firmware update like normal (which will download the firmware and reboot the switch into the new firmware).  Please do open a case with Support to assist with properly tracking and resolving this.

[rvdtol]

Hi MerakiDave,

For us the problems are not limited to 220/320 as you described. We experience these problems on350 and 425 switches as well.

Also we are not upgrading from a 9.x version but from a 8.x version.

Does your proposed mitigation results in rebooting the switches one-by one if configured in a stack?

Ps. I already opened a case with Support but they told me that rebooting the switches in 1 stack at the same time is expected behaviour.

[BrothersTM]

The firmware issues is definitely not resolved.  On this round of updates to my MS320 switches from 9.30 to 9.32, all but 5 of my switches had to be manually rebooted post deployment in order for the firmware to update.  I thought last Friday's snow day was a great day to update all the switches.  I was very wrong, since that required me to come in anyways and go closet t closet pulling power to switches until they were all updated and back online.

[MFuchs]

i'm using staged updates now for this to make sure it work 😉

[MRCUR]

I'm not seeing this issue on the 10.X branch FWIW. 

[MFuchs]

Sounds good, will test in the 10.x branch, too.