New 18.211.5 Release Out for MXs

[Mloraditch]

This appears remarkably similar to the never available 18.212 release notes with perhaps a couple of changes and 18.212 is gone now and they have I think the same release dates so appears may have just been renamed, there is an interesting new "Other" note at the bottom as well.

Important notice

• USB modems with MX/Z series devices running firmware MX 18 or newer will be limited to best effort support and will not be receiving any future firmware fixes or improvements.

Executive summary

• This is a maintenance release for MX 18.211 containing only bug fixes.
• The fixes are mostly focused on resolving rare, but potentially disruptive issues that could be encountered while using VPN (AutoVPN, Non-Meraki site-to-site VPN, and Client VPN).
• Another noteworthy fix restores the proper functioning of factory resets for MX95, MX105, MX250, and MX450 appliances.
• Additional fixes are also present, so please read through the full details below.

Bug fixes - general fixes

• Fixed a very rare issue that could result MX appliances operating as VPN concentrators encountering a device reboot or a failure to connect to new AutoVPN peers. This was most likely to occur on VPN concentrators with thousands of AutoVPN peers connecting and disconnecting over a period of months.
• Resolved an issue that could result in MX appliances incorrectly dropping traffic destined for AutoVPN peers for traffic that was received through the WAN interface. This occurred when the MX appliance was configured in passthrough mode and a default route was learned through eBGP.
• MX appliances will now more gracefully apply firewall rule configuration changes. This will resolve several instances where updating large sets of L3 or site-to-site VPN firewall rules could impact packet processing and network control traffic.
• Corrected a regression that caused traffic sourced by the MX to incorrectly follow the client routing table when a default route was advertised and multiple AutoVPN hubs were configured. This affected the MX's ability to establish an iBGP connection over AutoVPN, as well as impacting its ability to correctly route traffic such as NetFlow and syslog.
• Corrected an MTU issue that could result in MX appliances erroneously performing fragmentation of AutoVPN traffic after encapsulation had been performed. This occurs when the MX appliance is using an MTU other than 1500, such as 1230 or 1486.
• Resolved a very rare issue that could result in MX appliances encountering a device reboot when non-Meraki VPN or IKE-based client VPN were in use. This did not affect devices using AnyConnect VPN.
• Resolved an issue that resulted in MX appliances incorrectly fragmenting non-Meraki VPN traffic after encryption and encapsulation had been performed, rather than before it.
• Corrected an issue that could result in an unexpected reboot when an IPv6 DHCPv6-PD packet advertised a prefix of larger than /64.
• Resolved an issue that resulted in traffic shaping priorities not being applied correctly when default traffic shaping rules were enabled.

Bug fixes - limited platforms

• Corrected an MX 18.2 regression that resulted in it not being possible to perform a factory rest on MX95, MX105, MX250, and MX450 appliances.
• Corrected an MX 18.211.4 regression that could result in MX75, MX85, MX95, MX105, MX250, and MX450 appliances encountering an unexpected device reboot when processing a large volume of network flows.
• Fixed a rare issue that could result in AMP incorrectly blocking traffic on MX75, MX85, MX95, MX105, MX250, and MX450 appliances.
• Fixed an additional issue that could result in MX75, MX85, MX95, MX105, MX250, and MX450 appliances reporting an erroneous spike in network traffic usage.
• Fixed an issue that resulted in vMX appliances running on Cisco UCS servers failing to upgrade to MX 18.2XX versions when the vMX appliance was running a version older than MX 17.
• Fixed a rare issue that could result in unexpected device reboots for MX67(W), and MX68(W,CW) appliances operating under heavy network load.
• Further resolved cases where MX68(W,CW) and MX85 appliances could improperly attempt to provide PoE power to connected devices.
• Fixed an issue that resulted in Z4(C) appliances failing to properly forward STP frames received on its LAN interfaces.
• Corrected an issue that resulted in ​​Z4(C) appliances failing to forward ARP messages that had a VLAN tag, even when the VLAN tagging correctly matched with the Z4(C)'s port configuration.

Legacy products notice

• When configured for this version, Z1 devices will run MX 14.56.
• When configured for this version, MX400 and MX600 devices will run MX 16.16.9.
• When configured for this version, MX64(W), MX65(W), MX84, MX100, and vMX100 devices will run MX 18.107.12.

Known issues status

• This list is being reviewed and updated.

Known issues

• Trusted traffic exclusions will not function on Z4(C) appliances if AMP is configured.
• Due to a rare issue, MX appliances may fail to initiate non-Meraki site-to-site VPN connections when IKEv2 is used. This is most likely to occur when there are mismatched VPN subnets configured between the MX and the non-Meraki VPN peer. This will be resolved in MX 19.1 releases.
• Due to an issue under investigation, VMX-XL appliances fail to add local networks into the routing table.
• Due to an issue under investigation, MX appliances may incorrectly report 100% loss on the SD-WAN monitoring page.
• In rare cases MX75, MX85, MX95, MX105, MX250, and MX450 appliances may encounter an unexpected device reboot.
• Due to an issue under investigation MX75, MX85, MX95, MX105, MX250, and MX450 appliances may report an erroneous spike in network traffic usage.
• Z4(C) appliances fail to forward ARP messages that have a VLAN tag, even if the VLAN tagging correctly matches with the Z4(C)'s port configuration.
• Due to issues under investigation, MX75 and MX85 appliances may encounter unexpected device reboots.
• Z4(C) appliances fail to properly forward STP frames received on its LAN interfaces.

Other

• Added proactive mitigations that should prevent new or unforeseen internal packet routing issues from resulting in MX75, MX85, MX95, MX105, MX250, and MX450 appliances continuously operating at 100% device utilization.

[RaphaelL]

This is the diff between 18.212 and 18.211.5 : 

• MX appliances will now more gracefully apply firewall rule configuration changes. This will resolve several instances where updating large sets of L3 or site-to-site VPN firewall rules could impact packet processing and network control traffic.

• Fixed an additional issue that could result in MX75, MX85, MX95, MX105, MX250, and MX450 appliances reporting an erroneous spike in network traffic usage.
• Fixed an issue that resulted in Z4(C) appliances failing to properly forward STP frames received on its LAN interfaces.
• Corrected an issue that resulted in ​​Z4(C) appliances failing to forward ARP messages that had a VLAN tag, even when the VLAN tagging correctly matched with the Z4(C)'s port configuration.
• Added proactive mitigations that should prevent new or unforeseen internal packet routing issues from resulting in MX75, MX85, MX95, MX105, MX250, and MX450 appliances continuously operating at 100% device utilization.
•  

[thomasthomsen]

Yeah ... 18.212 is gone from "release notes" it seems ... 🙂

But then again, I was never able to select that software for upgrade.

[PhilipDAth]

I tried 18.211.5 and found it broke AnyConnect with SAML authentication.  I have had to roll back.

[PhilipDAth]

A further update - my issue turned out to be time synchronisation on the VMX.  It was too far out from the SAML provider.

Just in case anyone else runs into this issue, about once a year when running in Amazon AWS, I see this event:

msg: SAML: Assertion is expired or not valid

The VMX and the SAML provider have to have their time synchronised to within 3 minutes of each other.  Anymore than that, and you get the above error.  When the VMX runs in Amazon it seems to source its time from the host (rather than NTP).

The solution to fix the issue to to shutdown the VMX and then start it again.  This moves you to a new host, which 99.999% of the time has the correct time.

[Ren1]

I was told by Meraki support that this update fixes issues with traffic analytics not showing in the dashboard and also insufficient data error when looking at individual clients.

any one know first hand if this update fixes those issues?

[Stephen_P]

We've been getting weekly random reboots on our MX450 with the 18.211.4 firmware.  I wonder if this will fix the issue.  We have a case opened with support who are actively troubleshooting the issue.  Has anyone else been experiencing the same issue?

[DonaldB]

We deployed 18.211.5 on Thursday (1/23/25), upgrading from 18.211.2. After the upgrade, we ran into high latency, jitter, and packet loss issues and had to roll back last night (1/27/25). It was negatively impacting our MS Teams and Talk Desk applications.

[annmarie24us]

 

• As of MX 19.1, Cisco Meraki will no longer support USB-based Cellular Failover on the MX and Z platforms.

Legacy products notice

• When configured for this version, Z1 devices will run MX 14.56.
• When configured for this version, MX400 and MX600 devices will run MX 16.16.9.
• When configured for this version, MX64(W), MX65(W), MX84, MX100, and vMX100 devices will run MX 18.107.11.

Bug fixes - general fixes

• Corrected a rare device reboot that could occur when BGP was in use.
• Resolved an issue that could cause eBGP routes learned over non-Meraki VPN tunnels to become unreachable after other device configuration changes were made.
• Fixed an issue that resulted in failover between non-Meraki VPN tunnels not occurring in some cases.
• Resolved an issue that could result in MX appliances incorrectly dropping traffic destined for AutoVPN peers for traffic that was received through the WAN interface. This occurred when the MX appliance was configured in passthrough mode and a default route was learned through eBGP.
• Resolved an issue that could result in MX appliances failing to establish an eBGP session with a peer on the LAN. This happened in configurations that used a route overlapping the IP address of the eBGP peer for failover and failback of non-Meraki VPN peers.
• Corrected a regression that caused traffic sourced by the MX to incorrectly follow the client routing table when a default route was advertised and multiple AutoVPN hubs were configured. This affected the MX's ability to establish an iBGP connection over AutoVPN, as well as impacting its ability to correctly route traffic such as NetFlow and syslog.
• Corrected an MTU issue that could result in MX appliances erroneously performing fragmentation of AutoVPN traffic after encapsulation had been performed. This occurs when the MX appliance is using an MTU other than 1500, such as 1230 or 1486.
• Resolved an issue that resulted in traffic shaping priorities not being applied correctly when default traffic shaping rules were enabled.
• Corrected an issue that could result in an unexpected reboot when an IPv6 DHCPv6-PD packet advertised a prefix of larger than /64.

Bug fixes - limited platform fixes

• Corrected an MX 19.1.4 regression that could result in MX75, MX85, MX95, MX105, MX250, and MX450 appliances encountering an unexpected device reboot when processing a large volume of network flows.
• Fixed other circumstances that could result in MX75, MX85, MX95, MX105, MX250, and MX450 appliances continually operating at 100% device utilization.
• Corrected a regression that resulted in being unable to perform a factory rest on MX95, MX105, MX250, and MX450 appliances.
• Resolved an issue that resulted in MX75, MX85, MX95, MX105, MX250, and MX450 appliances dropping traffic destined to itself when the traffic was received over AutoVPN. This could cause a lack of response to things like an SNMP walk sent over AutoVPN.
• Fixed an issue that could result in GRE traffic being incorrectly dropped by MX75, MX85, MX95, MX105, MX250, and MX450 appliances when using a 1:1 NAT and the traffic path was a “hairpinned” NAT.
• Corrected an issue that resulted in ​​Z4(C) appliances failing to forward ARP messages that had a VLAN tag, even when the VLAN tagging correctly matched with the Z4(C)'s port configuration.
• Fixed a rare issue that could result in unexpected device reboots for MX67(W), and MX68(W,CW) appliances operating under heavy network load.
• Fixed an issue that could result in the ping live tool failing when trying to ping an IPv6 IP address over a cellular uplink.
• Fixed an issue that could result in port forwarding rules for integrated cellular modems not working after a device reboot was performed.
• Fixed an issue that resulted in vMX appliances running on Cisco UCS servers failing to upgrade to MX 19.1 versions when the vMX appliance was running a version older than MX 17.
• Further resolved cases where MX68(W,CW) and MX85 appliances could improperly attempt to provide PoE power to connected devices.

Known issues status

• This list is being reviewed and updated. Many existing issue reports have not been confirmed to affect MX 19.1 firmware versions.

Known issues

• During the upgrade process, MX appliances upgrading from versions prior to MX 19 may experience a failure to properly classify traffic. This issue will be resolved once the appliance has completed the upgrade to MX 19.
• Z4(C) appliances fail to properly forward STP frames received on its LAN interfaces.

Known issues - december 9th update

• Due to an issue under investigation MX75, MX85, MX95, MX105, MX250, and MX450 appliances may report an erroneous spike in network traffic usage.
• Due to a rare issue without a known method of reproduction, AMP may incorrectly block traffic on MX75, MX85, MX95, MX105, MX250, and MX450.
• Due to an issue under investigation, MX appliances may incorrectly route traffic destined to subnets learned through eBGP over a Non-Meraki VPN connection.

Other

• MX appliances will now use the IP address of their highest number VLAN as the NAS-IP in RADIUS Access-Request messages.
• The local status page will no longer display fields related to the cellular modem when no SIM card is present in the MX.
• Reduced the power consumption of MX67(W), MX68(W,CW), MX75, and MX85 appliances in some cases.

Have you tried the MX 19.1.6 that has fixed the reboot that occurs?

[Stephen_P]

Just wanted to add that we upgraded to 18.211.5.  Like @DonaldB mentioned, we started experiencing bad latency spikes after the upgrade.  We will be performing a downgrade tonight.  Random reboots is better than the latency I guess.  I just talked to Meraki support today, and they are processing an RMA for our device.  If that doesn't solve the issue, I'm not sure what we're going to do.  This has been the worst security appliance I've ever had to support.  I would rather not upgrade to a release candidate, but at this point, if the RMA doesn't work then we may be forced to do just that. 

[DonaldB]

I would be interested to know how your RMA swap goes, @Stephen_P . Meraki support had suggested that for us at one point, but we resisted because our issues were present regardless of which member of our HA pair was acting as the primary unit.

We haven't run into the MX device reboot issue, though the issues that have been plaguing us for almost the past year seem just as bad sometimes. We have been seeing random drops of our WAN connections at seemingly random times during the day (all WAN traffic stops for 1-3 minutes). We've also been fighting a problem where the MX stops passing certain traffic, and we have to modify a firewall rule (any firewall rule) to get it to start passing traffic again. At least for that issue, I can proactively "poke the firewall" each morning, and that seems to keep that issue at bay for the most part. I haven't found a way to prevent the WAN drops, which can be nearly as challenging as a device reboot, but "thankfully" only happen a few times a week.

[Stephen_P]

The RMA swap did not resolve our issues.  We swapped out our original MX450 with the RMA replacement on Sunday.  Two days later (today) the new MX450 just rebooted interrupting internet across our campus.  I'm contacting support yet again, but this is unacceptable.  We need a reliable firewall, not one that is now rebooting every two days.  We are running 18.211.4 on the new firewall as well.