New MS 15.14.1 beta firmware - fix for flash corruption on MS390 stacks

cmr
Kind of a big deal
Kind of a big deal

New MS 15.14.1 beta firmware - fix for flash corruption on MS390 stacks

Switch firmware versions MS 15.14.1 changelog

Alerts

  • HTTP proxy is no longer supported on MS 15+. Nodes that use HTTP proxy without any other means to connect to dashboard may fail to connect.
  • Moving or re-provisioning an MS390 stack in dashboard can cause the management plane of at least one member to stay offline until rebooted (present since MS 15.0)
  • MS390 upgrades from MS 15.14 or later will result in minimal impact to client traffic
  • MS390 series switches can only downgrade to MS 12 with an incremental step to MS 14
  • MS390 series switches can only downgrade to MS 12 with an incremental step to MS 14

Branch additions

  • STP anomaly detection support for MS390 series switches
  • IPv6 static routing support for MS390 series switches
  • Additional client analytics added for MS390 series switches
  • Meraki authentication support for MS390 series switches
  • Alternate Management Interface (AMI) support for MS390 series switches
  • UDLD support for MS390 series switches
  • Group policy ACL support for MS390 series switches
  • IPv6 management interface support
  • URL redirect support for MS390 series switches
  • UPoE (802.3bt) support for MS390 series switches
  • Critical/failed authentication support for MS390 series switches
  • Multi-auth with voice VLAN bypass support
  • MAC flap detection support for MS390 series switches
  • Named VLAN support for MS120/125/210/225/250/350/355 series switches
  • Stack power is supported by default for MS390 series switches
  • Netflow and Encrypted Traffic Analytics (ETA) support for MS390s

Bug fixes

  • Larger MS390 stacks may fail to successfully boot into the firmware causing a ruination to the previous image (present since MS 15.14)

Known issues

  • MS390 management plane may encounter infrequent restarts (always present)
  • MS390 series switches in rare instances disconnect from dashboard until rebooted (present since MS 14.33)
  • MS390s in rare circumstances will experience a full system restart (preset since MS 14)
  • When the MS390 management plane experiences a restart, LACP flapping can occur
  • Stack ports may fail to initialize properly upon reboot. A subsequent reboot of the switch would be needed (present since MS 14)
  • MS390s that receive incorrectly flooded CDP packets may incorrectly report VLAN mismatches and SFP port information (present since MS 12)
  • Connecting a stacking cable to a stack that is online may result in a stack member going offline (present since MS 12)
  • In rare instances, MS120 series switches may have empty packet captures until they are rebooted
  • Switch stacks will learn MAC addresses from ports in the STP blocking state which can trigger a constant flood of MAC flaps in the event log
  • Links being established on an MS120 can result in neighboring ports to flap (present since MS 11)
  • MS120 series switches in extremely rare instances will experience reboots every few minutes (present since MS 11)
  • MS390 "Port Up/Down" events will be shown across all members
  • IGMP querier is enabled on all VLANs resulting in IGMP messages being sent every 125 seconds. Layer 2 switches use the management IP as the querier address. (always present)
  • Enabling Combined Power on MS350/355 switches results in events being logged once per minute (present since MS 11)
  • Networks containing a large number of switches may encounter issues saving changes on the Switch Settings page
  • Stack members may experience delays in updating their configuration for up to an hour after a config change (present since MS 9)
  • mGig switches will have an amber light for all physical ports that do not negotiate to the highest supported speed. Dashboard will continue showing a light green status for all ports above 100Mbps. For example, MS355 switch ports will incorrectly show an amber light for 1G, 2.5G, and 5G, but will show a green light for 10G.
  • LACP bundles configured across stack members experience brief outages when a stack member goes offline and again when it comes online (present since MS 10)
  • MS390 stacks may send frequent DHCP requests despite having a valid static IP address, which can result in IP flapping (present since MS 14)
  • LACP bundles spanning multiple stack members can result in a temporary loop if one of the stack members is rebooted (present since MS 11)
  • In rare instances, RADIUS accounting sessions may not be removed after a client disconnects with an EAPoL logoff message (present since MS 11)
  • Broadcast types of traffic can leak into the Guest VLAN if a port that fails authentication has a voice VLAN configured, and dashboard has a Guest VLAN defined (present since MS 11)
  • MS390 series switches do not support loop detection
  • MS390 series switches do not support warm spare/VRRP
  • MS350-24X and MS355 series switches do not negotiate UPoE over LLDP correctly (predates MS 10)
  • AMI IP addresses do not send gratuitous ARP packets which can lead to packet loss if the AMI address has aged out in the network
  • Meraki authentication does not work with guest VLAN
  • Moving or re-provisioning an MS390 stack in dashboard can cause the management plane of at least one member to stay offline until rebooted (present since MS 15.0)
  • Ports with an odd-numbered MTU value fail to initialize for MS120/125 series switches (predates MS 11)
  • When an SFP module is inserted/removed on MS420/425 series switches, BPDUs can be delayed leading to STP transitions in the network (predates MS 12)
  • MS350/450 series switches in a stack configuration will lose dashboard connectivity if a "Deny Any Any" ACL is added without having higher "Allow" rules in place for dashboard connectivity (predates MS 12)
  • Rebooting an MS390 switch in a stack via the UI will result in the entire stack rebooting
  • MS390s with a Group Policy ACL containing spaces or non-alphanumeric characters results in ports being initialized with incorrect configurations of trunk, native VLAN 1, until the system finishes booting
4 REPLIES 4
MiguelMVLA
Here to help

Had multiple stacks of 5+ switches fail to upgrade when rolling out 15.14. Was able to restore functionality by breaking the stacks and upgrading each switch individually, per SE’s recommendation, which took hours considering each switch took 20/30 minutes to boot, download and install the firmware and boot again.

 

This wasn’t the first time we’d been burned by failed stack upgrades so we ended up implementing a 3 switch stack cap since we had the SFP modules, SFP+ units, fiber pairs and a Distribution switch with enough free SFP+ ports. This means we now have two independent stacks of 3 switches in some IDFs and it cleared up the constant Meraki Dashboard drops on the previously 5 or 6 unit stacks…but I’m still seeing single MS390s drop off for no obvious reason and during times when essentially nobody is using the network as shown on the following image:

 

D249CB1A-A07C-4C82-B3DA-0B7EBA2C6EB6.jpeg

Waiting for our Technical Solutions Architect to find more information on 15.14.1 to decide if it’s needed or suggested.

Miguel

I don't like building stacks bigger than 4 either.  5 absolute tops.

Makes sense. Our largest deployments were 6/5 units so 3 just happened to work for us.

 

Our Meraki Dashboard seems healthier after splitting our MS390 stacks:

 

MV 700 Wing MS390sMV 700 Wing MS390s

 

 

 

Miguel
MiguelMVLA
Here to help

Got an answer regarding upgrading to 15.14.1 from 15.14 as it relates to the fix for flash corruption:

 

The MS390 stack corrupting its firmware happens specifically when upgrading TO 15.14 or when a reboot occurs on a switch stack of 8 (<1% of the time).

 

It is advised that with smaller switch stacks it's more applicable to stay on 15.14 unless you’ve run into a specific issue that needs the new maintenance release.

Miguel
Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.
Labels