New MS 14.12 firmware, unexplained reboot and L3 issue upon reboot fixed

[cmr]

I'm really happy about this one as we've seen these issues on a L3 MS210 stack 😎

Switch firmware versions MS 14.12 changelog

Alerts

• MS390 upgrades from MS 14.10 or later will result in minimal impact to client traffic
• MS390 client authentication may fail closed if RADIUS server passes unsupported AVPs
• Auth session state on RADIUS server may be stale following an MS390 CoA update until RADIUS Accounting is supported
• MS210/225/250 switches cannot forward IPv6 Router Advertisement and Neighbor Solicitation packets when flood unknown multicast is disabled (has always been true)

Bug fixes

• Rebooting non-MS390 stack members can produce negative impact on L3 routing
• Dashboard can depict a switch port in green/forwarding state even when port is in access policy discarding state
• Switches can experience an unexpected reboot on weekly basis
• Mac table fetch on MS390 switches can produce incomplete data for module ports and LAG interfaces

Known issues

• Non-MS390 switch stacks using Dynamic ARP Inspection can block ARP packets which should actually be permitted
• MS390 stack members may not report client-level detail
• MS390 series switches Max MTU is limited to 9198 bytes
• MS390 series switches will not display packet details in the DHCP Servers page
• MS390 series switches do not support the 'next-server' or 'bootfile' parameters in DHCP messages
• Rebooting a single switch in an MS390 stack will reboot the entire stack
• LLDP frames from MS390 series switches will use a single system name instead of the Dashboard given name
• MS390 series switches will not display the advertising router ID for OSPF
• MS390 series switches with OSPF enabled will default to an IP MTU of 1500 bytes on every OSPF enabled interface
• MS390 series switches do not support the multicast routing livetool
• MS390 series switches do not support tagged traffic bypass for voice on ports with a Multi-Auth access policy
• Dynamic VLAN assignment from a RADIUS server on MS390 series switches must already be an allowed VLAN on the port for it to function
• MS390 series switches do not currently support the following features: VRRP, SM Sentry, Syslog server, SNMP, Traceroute, IPv6 connectivity to dashboard, Meraki Auth, URL Redirection, MAC Whitelisting, RADIUS Accounting, Loop Detection, UDLD, MAC Flap Detection
• SM Sentry does not support Windows based clients (predates MS 10.x)
• MS425 series switches do not forward frames larger than 9416 bytes (predates MS 10.x)
• MS350-24X and MS355 series switches do not negotiate UPoE over LLDP correctly (predates MS 10.x)

Other

• Various performance and stability improvements

[Mloraditch]

That reboot bug was killer. We had MS210/225 stacks that we needed the 14.5 fix for routing in and we deployed it and then we discovered the reboot bug. Not a fun time with those customers.

Have deployed 14.12 to two test sites and keeping my fingers crossed we've found the one ring...

[cmr]

@Mloraditch we had the exact same and had to revert to 12.32, I've upgraded one site so far and plan to do the others tomorrow. 🤞

[Sabin]

I'm experiensing this same issue.  We just deployed 36 MS390's and have had 20%-40% ping loss on the management IP's.  The larger the stack, the worse the ping loss apears.  We are also experiencing issues with stacks losing access to the dashboard as described above. 

Has anyone seen any improvement if all L3 was moved off the MS390's?  I have a stack of 4 acting as a distribution/CORE.  However I have Palo Alto NGFW's that I could move all L3 functionality to.  That way all stacks will be L2 only, and the "CORE" becomes a distribution switch only.  

We have been considering this move anyway, if doing so could help mitigate some of these issues then it will be easier to push through change management. 

[cmr]

@Sabin I'd move the L3 off and upgrade to 15.9 as every release fixes MS390 issues and they are all beta for that model...  I would also make the stacks as small as is possible, no more than 4 at most.

[Sabin]

@cmr - Thank you for the advise.  Have you moved L3 off with an improvement in stability? 

[cmr]

I haven't personally used any MS390s, but as they are very much still a work in progress, removing any feature is good for their stability.  There is another topic on here where what you are suggesting was done with a very similar setup and it did help.