Security appliance MX 18.211 Release

[TyShawn]

Security appliance firmware versions MX 18.211 changelog

Important notice

• USB modems with MX/Z series devices running firmware MX 18 or newer will be limited to best effort support and will not be receiving any future firmware fixes or improvements.

Bug fixes

• Resolved an MX 18.2 regression that resulted in the WAN2 being unable to pass traffic if 1) WAN1 was not in use and 2) cellular was enabled.
• Fixed inconsistencies with the cellular active uplink feature. WAN 2 cannot be used as a functioning WAN interface when cellular active uplink is enabled.
• Fixed a MX 18.2 regression that resulted in MX75, MX85, MX95, MX105, MX250, and MX450 appliances being unable to successfully establish IPSec VPN connections when NAT-T was required to establish the connection.
• Corrected an MX 18.2 regression that resulted in MX75, MX85, MX95, MX105, MX250, and MX450 appliances failing to form AutoVPN or teleworker tunnels with other peers via their LAN interfaces.
• Resolved an issue that made MX75, MX85, MX95, MX105, MX250, and MX450 appliances more likely to rewrite the source port of traffic being NAT'ed out a WAN interface.
• Fixed an issue that resulted in the VPN status information for non-Meraki VPN peers being shown incorrectly on the VPN status page in Dashboard.
• Fixed a rare issue that could result in the AnyConnect VPN process becoming unresponsive on MX75 and MX85 appliances.
• Resolved an issue that could result in AutoVPN tunnel instability on both MX uplinks when packet loss and intermittent connectivity occurred on one uplink.
• Corrected an issue that could result in Z4C appliances being unable to successfully pass cellular traffic when using a Telstra SIM.
• Fixed an issue that resulted in MX75, MX85, MX95, MX105, MX250, and MX450 appliances tracking information about upstream WAN addresses as if they were local clients if 1:1 or 1:M NAT were configured.
• Resolved an issue that resulted in uplink connectivity tests for IPv6 being routed incorrectly.
• Fixed an issue that could result in an increased level of jitter and latency for AutoVPN traffic on Z3(C) appliances. That would specifically occur during periods of low and infrequent AutoVPN traffic.
• Stability improvements for MX75, MX85, MX95, MX105, MX250, and MX450 appliances.
• Resolved an issue that could result in MX appliances with adaptive policy configured encountering frequent connectivity state changes for AutoVPN tunnels.
• Corrected a MX 18.2 regression that resulted in the SIM and APN configuration being shown on the device local status page for devices without integrated cellular modems.
• Reduced the potential for existing traffic flows to be disrupted from configuration changes on MX75, MX85, MX95, MX105, MX250, and MX450 appliances.
• Resolved a rare issue that could result in the AnyConnect client VPN process crashing.
• Corrected a rare issue that could result in an IPv6 delegated prefix not being visible in Dashboard.
• Fixed a MX 18.2 regression that could result in MX appliances not performing ARP for virtual IP addresses, 1:1 NAT IP addresses, and 1:M NAT IP addresses when 1) the MX was configured in high availability and 2) had WAN1 disconnected or disabled.

Legacy products notice

• When configured for this version, Z1 devices will run MX 14.56.
• When configured for this version, MX400 and MX600 devices will run MX 16.16.9.
• When configured for this version, MX64(W), MX65(W), MX84, MX100, and vMX100 devices will run MX 18.107.10.

Known issues status

• This list is being reviewed and updated.

Known issues

• In rare cases, MX67C, MX68CW, and Z3C appliances may fail to enter into a "Ready" state despite being able to register to a cellular network and obtain an IP address for the modem.
• The Non-Meraki VPN service may fail to properly establish IKEv2 tunnels when the MX appliance is acting as the IKEv2 responder and many allowed subnets are configured.
• Due to an MX 18.2 regression, the link light LED for WAN2 on MX75 appliances will not light up if WAN2 is the only wired interface in use.
• Due to an issue with no known method of reproduction, the IDS and IPS process may unexpectedly restart.
• When a WAN failover occurs, Non-Meraki VPN tunnels will persist on the backup, non-primary uplink after a failback to the primary WAN interface if the WAN interface uses IPv6.
• Due to an issue still under investigation, MX appliances may experience an unexpected reboot when ThreatGrid is enabled.
• MX AutoVPN tunnels fail to generate new connections when the AutoVPN flow has been blocked or filtered unidirectionally by an upstream or intermediary device. This prevents appliances from automatically working around this partially connected state.
• MX appliances may experience unstable eBGP connections when 1) the MX appliance is configured in Routed mode and 2) the MX learns a large number of routes from its eBGP neighbor. This may result in eBGP-learned routes being inaccessible.

Other

• Improved AutoVPN failover times for VPN connections between MX appliances running MX 18 or higher.

[CptnCrnch]

That's a lot of fixes! 😎

Upgrading just now.

Edit: looking good so far!

[cvsmith122]

Do you do any VPN or 1:1 I have rescheduled my update because the last time Meraki did a auto update it broke VPN and 1:1. 

[plumdev]

I updated two networks from 18.208 to 18.211 last night and one of them lost autovpn site to site connectivity completely. Rolled it back this morning.

[cmr]

Still a lot of stability issues, hopefully the fix list won't shrink...

[RaphaelL]

MX18.2.11 was supposed to be the first 'stable' version. Wonder what happened.

[jimmyt234]

It needs to reach at least 15% global network saturation prior to being promoted to stable (Meraki Firmware Release Process - Cisco Meraki Documentation). We've seen at least a dozen customer orgs have an auto-upgrade scheduled for this version overnight, so it seems Meraki will push this one out quite aggressively to reach this limit in order to get it to stable.

[RaphaelL]

Yes you are right. forgot about that part. 

Which also means that MX18.2.11 will be the only stable version.  Other fixes will only be included in patches ( 18.2.11.X )

[jimmyt234]

Yes 😄

18.210 has been pretty solid for us, hoping for the same from .211!

[FRover]

We still have a large number of devices that are 4G only on 16.16.9 due to mobile connectivity issues on 17.xxx 

Is that resolved here or does it fall under this known issue?

• In rare cases, MX67C, MX68CW, and Z3C appliances may fail to enter into a "Ready" state despite being able to register to a cellular network and obtain an IP address for the modem.

[DMD]

I updated one of my test sites that had this similar issue on 18x running the 4G SIMs.  Are you using static or dynamic APNs?  I updated mine to 18.211 using a static APN and associated with the proper location and worked perfect.  the 18.210 broke this 

·         West: WE01.VZWSTATIC

·         Midwest: MW01.VZWSTATIC

·         Northeast: NE01.VZWSTATIC

·         South: SO01.VZWSTATIC

[jbright]

I had a customer that had to rollback to 18.210 this morning because 18.211 broke site-to-site AutoVPN.

He told me Meraki TAC told him that they have a number of customers experiencing issues with this version today.

I installed it on all of my company's MX and it is working fine for us and we have one AutoVPN connection that is also working

fine. My advice is to test it on non-production networks first to see if your environment is impacted.

[cmr]

I've installed it on an MX68 that is acting as an AutoVPN spoke and it seems to work okay, being stable and performing normally.  It is only running the enterprise feature set, so issues might be related to advanced licensing?

[BHC_RESORTS]

We YOLO'd this update to our "test" site which is our corp office. MX95. So far so good...we got burned hard on the last "stable" RC so we're taking a bit of a gamble here but I liked the fix list.

[TyShawn]

For me, SNMP stopped working with this release. 

[Lauzon_C]

What version of firmware were you using prior? I heard mention of possible snmp issues here. We are on 18.210.

[Lauzon_C]

Had to roll back to 18.210, had vpn issues on the site I tested it on.

[TyShawn]

I was on 18.210 and many other older versions. 18.211 is the version that broke SNMP for me. I have a ticket opened up on this version. 

[bdeen]

I have a scheduled upgrade for 4 sites for this weekend, are there any known issues with this present release MX 18.211 apart the one mentioned above. Thanks 

[cmr]

@bdeen from the release notes the below are all known issues, hopefully there aren't too many more:

• In rare cases, MX67C, MX68CW, and Z3C appliances may fail to enter into a "Ready" state despite being able to register to a cellular network and obtain an IP address for the modem.
• The Non-Meraki VPN service may fail to properly establish IKEv2 tunnels when the MX appliance is acting as the IKEv2 responder and many allowed subnets are configured.
• Due to an MX 18.2 regression, the link light LED for WAN2 on MX75 appliances will not light up if WAN2 is the only wired interface in use.
• Due to an issue with no known method of reproduction, the IDS and IPS process may unexpectedly restart.
• When a WAN failover occurs, Non-Meraki VPN tunnels will persist on the backup, non-primary uplink after a failback to the primary WAN interface if the WAN interface uses IPv6.
• Due to an issue still under investigation, MX appliances may experience an unexpected reboot when ThreatGrid is enabled.
• MX AutoVPN tunnels fail to generate new connections when the AutoVPN flow has been blocked or filtered unidirectionally by an upstream or intermediary device. This prevents appliances from automatically working around this partially connected state.
• MX appliances may experience unstable eBGP connections when 1) the MX appliance is configured in Routed mode and 2) the MX learns a large number of routes from its eBGP neighbor. This may result in eBGP-learned routes being inaccessible.

[scc_sysadmin]

Ever since the upgrade to this version, all our Meraki switches' DNS entries were changed to 1.1.1.1 and 8.8.8.8. Causing intermittent disconnections across our switches now. Anyone have similar issues or how can I revert back to the older version?