New MS 16.8 stable firmware - multiple LACP and PoE fixes!

[cmr]

Switch firmware versions MS 16.8 changelog

Ms12x important notes

• If DAI is enabled prior to upgrading the network to MS 16, ensure that trusted ports and/or DAI-allow lists are configured prior to upgrading to avoid potential network outages.

Ms420 important notes

• Devices configured for this version will run MS 15.22

New feature highlights

• ACL hit counter
• DHCPv6 and IPv6 RA guard
• Local Status Page Packet Capture

New ms12x feature highlights

• DAI support

Fixed issues

• A stack containing more than 10 LACP bundles may encounter a brief network loop when a stack member is rebooted (present since MS 15)
• Fixed LACP discarding state on cross-stack member link aggregation ports after active switch reboot
• Improved STP performance by updating STP forward state after link-aggregation interface state is programmed in hardware
• Performance optimization for generating Support-Data-Bundles on offline nodes, reducing completion time

Ms120 fixed issues

• Fixed a rare bug that caused some switches to reboot persistently, fail firmware upgrades, or lose Dashboard connection after a reboot
• Resolved access issues to the Local Status Page of MS120s via Management Port

Ms130 fixed issues

• Fixed a bug preventing MS130Rs from powering more than 6 802.3bt PD devices
• Fixed device reachability during control plane policing tests

Ms350 fixed issues

• Resolved an issue preventing MS350s from providing power to 802.3at Class 4 devices

Ms425 fixed issues

• Addressed an issue where OSPF Adjacency was stuck in the Exchange state during OSPF/BIRD initialization

General known issues

• Multicast routing may stop functioning after a cross-stack link aggregation disruption
• RADIUS authentication failures are possible at boot when Guest VLAN is configured
• RADIUS communications may not recover after an initial failure when Critical Auth is enabled
• Stack port configurations may become isolated or changed upon multiple flaps or when disabled
• Switches move LACP ports to an active forwarding state if configured. This can cause loops when connecting to an MS390 or other Catalyst switches unless the bundles are configured on the MS390/Catalyst switches first. All non-catalyst ports are configured in passive LACP mode so that loops do not occur between Meraki switches (always present)

Ms120 known issues

• Some MS120 switches might not complete firmware upgrades, causing them to revert to previous firmware, and attempt to upgrade every 2 hours
• Switches may fail to provide PoE power to legacy access points (always present)

Ms210 known issues

• Stack ports have an orange LED instead of green (present since MS 11)

Ms250 known issues

• Interruptions to aggregate links initiated by restarts or reconnection may produce unexpected LACP discarding states
• Stack nodes may erroneously report "MAC Table Full" errors

Ms2xx/35x/4xx known issues

• Switch stacks will learn MAC addresses from ports in the STP blocking state which can trigger a constant flood of MAC flaps in the event log

Ms35x known issues

• In rare instances, stack ports fail to initialize after an upgrade (always present)
• Incorrect SFP port mappings may disrupt SFP functionality
• Switches may experience an unexpected reboot (present since MS 15)
• UPoE does not negotiate over LLDP correctly (always present)
• mGig switches will have an amber light for all physical ports that do not negotiate to the highest supported speed. Dashboard will continue showing a light green status for all ports above 100Mbps. For example, MS355 switch ports will incorrectly show an amber light for 1G, 2.5G, and 5G, but will show a green light for 10G.

Ms425 known issues

• MS425 stacks may become unresponsive under high load
• MS425 stacks may fail to add mac addresses for stack member aggregate ports across multicast groups

Ms4xx known issues

• When an SFP module is inserted/removed, BPDUs can be delayed leading to STP transitions in the network (predates MS 12)

[Nef64]

---

@cmr wrote:

Ms120 known issues

• Some MS120 switches might not complete firmware upgrades, causing them to revert to previous firmware, and attempt to upgrade every 2 hours

Any further info on this one or has anyone seen it in the field? 

[dcatiller]

The upgrade was pretty smooth for me overall. I updated a total of 43 switches for this site, models include MS425, MS355, MS120, MS220. Everything worked great (including one MS120) except for three MS220s I have, which basically all needed a factory reset and two of those needed to be goosed by support. I'm not sure the overall fix I was looking for is there, but I'll have to do some internal testing to verify. 

I did not have the PoE issues others were wanting to address, so I can't speak to those. It's pretty stable, so I'll roll it to other sites soon.

[Nef64]

Thanks for the update.

[cmr]

Which MS220 models trashed themselves?  I'm running it on an MS220-8P and that upgraded well.  Were your MS355s stacked?

[dcatiller]

They are all MS220-8P and the only ones I have. They are older and we've had some issues like this with them before.

I've got 5 stacks of MS355s, each with 6 to 8 switches, which updated smoothly.

[Sugarfree]

If the MS120 firmware upgrade fails and the MS120 attempts to upgrade every 2 hours, what is the fix for getting it to stop?

[ITMiltWhitOCSNC]

This update solved a huge headache for us, we recently installed MS355 48 port switches at a location and it killed some of our Yealink VoIP devices causing them not to boot PoE. After a ton of research and back and fourth with Meraki Support, it was determined from a Technician that worked on a case very similar that the T31P was deemed not compliant for proper wiring from the UPoE MS 355 and denied PoE to that device even after the spec sheet claims it is! Yealink refuses to believe this and wanted logs stating such but once 16.8 was pushed out to those switches, everything worked! 😬

[cmr]

Excellent news!  We have some MS355s with a few recalcitrant PoE devices, I'll see if it fixes them as well...

[dcatiller]

I'll know Saturday night, as I'm pushing MS 16.8 to a good size site with MS425, MS355 and MS120 switches. I'll report here.

16.8 addresses a specific issue I'm having with LACP links. When I would reboot a stack in an IDF or even interrupt the trunk to our security appliance stack the core switches would go orange for 10 to 15 minutes. 

[ITMiltWhitOCSNC]

I'll be following!!

[dcatiller]

I'm not one for applying updates this early, but in this case looking to fix a long running annoyance.

Famous last words: "Dude, hold my beer, watch this!"

Will report one way or another.

[Sugarfree]

How did the upgrade go for ya?

[HendrikvdM3]

MS425 OSPF flapping after upgrading to firmware version MS16.8

We have a stack 2 x MS425 with OSPF setup as connection to one of our datacenters, Hello1 Dead4.

This has been configured like this for the last 3 years and the OSPF neighbors' adjacencies has been stable with no alerts.

After upgrading the Stack to the latest stable version there is now alerts on the MS425 stack and the neighbors they are connected to.

There is no notable impact as the negotiation to reestablish the neighbors is sub 10ms.

Has anyone experience the same?