MX 19.1.7 Stable release candidate

[Fabian1]

Security appliance firmware versions MX 19.1.7 changelog

Important notice

• As of MX 19.1, Cisco Meraki will no longer support USB-based Cellular Failover on the MX and Z platforms.

Executive summary

• This is a maintenance release for MX 19.1 containing primarily bug fixes.
• There are fixes in a variety of areas, including changes that address known issues on MX75, MX85, MX95, MX105, MX250, and MX450 appliances.
• Additional fixes are also present, so please read through the full details below.
• With this release, we are piloting the inclusion of bug identification numbers for known issues. There will also be changes made to the previous MX 19.1 versions to include this information as well.

Legacy products notice

• When configured for this version, Z1 devices will run MX 14.56.
• When configured for this version, MX400 and MX600 devices will run MX 16.16.9.
• When configured for this version, MX64(W), MX65(W), MX84, MX100, and vMX100 devices will run MX 18.107.12.

Bug fixes - general fixes

• Fixed a rare issue that could result in MX appliances encountering an unexpected reboot when servicing many clients with a large number of network flows. This was more likely to occur on MX450 appliances supporting 10,000 or more active clients and 500,000 or more concurrent flows. This resolves known issue MX-35210.
• Corrected an issue that could result in non-Meraki VPN traffic being routed incorrectly when 1) IPsec VPN failover was configured and 2) VPN configuration changes were made.
• MX appliances will now more gracefully apply firewall rule configuration changes. This will resolve several instances where updating large sets of L3 or site-to-site VPN firewall rules could impact packet processing and network control traffic. This resolves known issue MX-35524.

Bug fixes - limited platform fixes

• Fixed an additional issue that could result in MX75, MX85, MX95, MX105, MX250, and MX450 appliances reporting an erroneous spike in network traffic usage. This resolves known issue MX-32538.
• Fixed a rare issue that could result in AMP incorrectly blocking traffic on MX75, MX85, MX95, MX105, MX250, and MX450 appliances. This resolves known issue MX-34038.
• Resolved an issue that could prevent AutoVPN tunnels from forming over cellular interfaces when the Cellular Active Uplink configurations are changed.
• Corrected an issue that resulted in MX appliances failing to establish PPPoE connectivity on the WAN3 interface.
• Fixed an issue that resulted in Z4(C) appliances failing to properly forward STP frames received on its LAN interfaces. This resolves known issue MX-34639.
• Fixed a rare issue that could result in VMX appliances going offline 11 months after first upgrading to an MX 19.1 release.

Known issues status

• This list is being reviewed and updated. Many existing issue reports have not been confirmed to affect MX 19.1 firmware versions.

Known issues

• During the upgrade process, MX appliances upgrading from version prior to MX 19 may experience a failure to properly classify traffic. This issue will be resolved once the appliance has completed the upgrade to MX 19. (MX-36307)
• Due to an issue under investigation, MX appliances may incorrectly route traffic destined to subnets learned through eBGP over a Non-Meraki VPN connection. (MX-34803)
• Duplicate retrospective “malware download detected” emails may be erroneously sent. (MX-30111)
• Due to an issue under investigation, making certain configuration changes to WAN interfaces (such as disabling or enabling an interface) can cause the IDPS process to fail. This issue may also cause high device utilization. The issue can be worked around by rebooting the MX appliance or disabling and then re-enabling IDPS. (MX-34504)
• Due to an MX 19.1.5 regression, Z4(C) appliances may fail to provide PoE power to connected devices. (MX-34938)
• Due to an MX 19.1 regression, traffic will fail to route over AutoVPN when the only active uplink is a cellular connection. (MX-35703)
• Due to an issue under investigation, MX75, MX85, MX95, MX105, MX250, and MX450 appliances may experience an unexpected device reboot when VPN NAT is configured. (MX-36180)
• When failover is configured between non-Meraki VPN tunnels, the Route Table page on Dashboard may incorrectly show the route for the primary VPN tunnel is inactive. (MX-36316)
• During the upgrade process, MX appliances upgrading from versions prior to MX 19 will experience a failure to connect to non-Meraki VPN peers if any VPN peer names contain a space. This issue will be resolved once the appliance has completed the upgrade to MX 19. (MX-36312)

Other

• When upgrading to MX 19.1.7 or higher, Z4C appliances will perform an upgrade of the integrated cellular modem. This may result in Z4Cs taking a longer time to complete the upgrade process.
• Improved the consistency of syslog output for firewall rule decisions. All log messages should now say “allow” or “deny.” Previously, an inconsistent mix of deny/allow and 0/1 were used.

[Fabian1]

• Due to an MX 19.1 regression, traffic will fail to route over AutoVPN when the only active uplink is a cellular connection. (MX-35703)

This is a problem for some of our networks... I hope this will be fixed as soon as possible

[Chad_CSB]

Hey Fabian1, can you clarify on what the impact is? 

The bug seems straight forward - does it mean AutoVPN no longer works over Cellular at all?

What if WAN1 is connected, but not seeing internet traffic?

Trying to determine if I can move forward. All of my sites are using MX67C's with internal cellular backup.

[Fabian1]

Unfortunately I'm also wondering what the impact would be.

I would guess that if you have anything connected to WAN1 (must be active), you'll be fine, but if the WAN1 goes down physically, AutoVPN doesn't work too. So for us, because we have some MX with LTE only, we are not able to update to this version...

I'm waiting to get this bug fixed

[Chad_CSB]

Tested this yesterday. It was as feared. If WAN 1 or 2 are not connected, and in 'active' status, cellular will not autoVPN. This update effectively makes cellular purposeless. 

This FW version is required in order to integrate with CiscoXDR, part of Cisco's Breach suite we just purchased so to say I'm disappointed is an understatement. Fingers crossed on a speedy resolution.

[cmr]

Bug IDs now included 🎉🎇

[RWelch]

👏 👏 👏  🎉 🎉 🎉

[thomasthomsen]

"MX appliances will now more gracefully apply firewall rule configuration changes. This will resolve several instances where updating large sets of L3 or site-to-site VPN firewall rules could impact packet processing and network control traffic. This resolves known issue MX-35524."

Wahooo ... perhaps it will actually now forward traffic when I apply rules.

Upgrading now ... 🙂

[ww]

Hmm or it gets more sketchy when it apply the rules. This fix is also in 18.211.5 but it took a long time to take effect for a vpn fw rule, that i ended up rebooting.

I wonder what it exactly  does.

[thomasthomsen]

So far, no problems... but I will test some more.

[thomasthomsen]

On a side note. I have a MX95 running latest 18 that did not quite like having ::0 as the gateway on the WAN side, after upgrading to this MX19 (and perhaps it would have worked on other 19 releases) it is now happy.

[CptnCrnch]

Same here. MX85 running perfectly fine.

[jimmyt234]

• Fixed a rare issue that could result in VMX appliances going offline 11 months after first upgrading to an MX 19.1 release.

Had to play the long game to find that one out 🤣

[TyShawn]

Updating my MX75 let see if the random reboot comes back.

[RaphaelL]

For those not aware of the most recent known issue : 

Known issues - january 31st update

• Due to an MX 19.1.7 regression, MX75, MX85, MX95, MX105, MX250, and MX450 appliances servicing many clients may experience intermittent periods of heightened packet drops and latency. (MX-36338)

[cmr]

19.1.7.1 has now been released to fix MX-36338