New MX 19.2.2 stable release candidate: active/active IPSEC VPN and cellular providers

[cmr]

Security appliance firmware versions MX 19.2.2 changelog

Important notice

• As of MX 19.1, Cisco Meraki will no longer support USB-based Cellular Failover on the MX and Z platforms.

Executive summary

• This is the first generally available release for the MX 19.2 release. It contains a substantial amount of valuable fixes across many different areas and new feature functionality. We anticipate almost all devices having valuable fixes in this release.
• There are a significant amount of fixes related to routing, VPN, and connectivity issues, including enhancements for BGP, IPsec, AutoVPN, and stability improvements for some models. Please read through the full details below.

What's new

• Added support for Active-Active Non-Meraki VPN peer connections.
• Added HTTPS support for the device local status page.
• Updated the list of built-in APNs for the Bell provider to include “mnet.bell.ca.ioe” and “mcorp.bell.ca.ioe” for Z3C, Z4C, MX67C, and MX68CW appliances.
• Updated the list of built-in APNs for Verizon to include “V5GA01INTERNET” for Z3C, Z4C, MX67C, and MX68CW appliances.
• Updated the list of built-in APNs for the Telus Mobility operator to include “m2m.telus.iot” for Z3C, Z4C, MX67C, and MX68CW appliances.
• Added support for the Transatel provider for Z3C, Z4C, MX67C, and MX68CW appliances.
• Updated the list of built-in APNs for the Mint provider to include “wholesale” for Z3C, Z4C, MX67C, and MX68CW appliances.

Bug fixes - general fixes

• Resolved an issue where MX appliances operating in passthrough mode would incorrectly advertise local IPv6 subnets to EBGP peers. The behavior is now consistent between IPv4 and IPv6 subnets after this change.
• Fixed an issue where packets would follow a less specific static IPsec route even if there was a more specific EBGP over IPsec learned route.
• Fixed an issue that resulted in MX appliances failing to properly install a 0.0.0.0/0 route into the routing table when it was learned through BGP over an IPSec VPN connection.
• Fixed an issue that could result in traffic sourced from clients in non-VPN-joined VLANs being incorrectly routed over an IPSec VPN connection. This could occur when the destination for that traffic matched a subnet that the MX appliance had learned about through an EBGP session with an IPSec VPN peer.
• Fixed an issue that resulted in the Non-Meraki VPN peer status always reflecting an inactive state when the remote VPN peer had a local ID that included the word “remote”.
• Corrected an issue that resulted in TCP MSS clamping not being performed on traffic routed across non-Meraki VPN peer connections.
• Fixed an issue that resulted in the AutoVPN MTU not being updated when SGT was enabled. The maximum AutoVPN MTU is now 1426 bytes when SGT is enabled.
• Fixed a rare issue that could result in packet loss and unstable AutoVPN connections when complex NAT schemes were used between peers. This typically occurred when Carrier Grade NAT was used by the Internet Service Provider.
• Corrected an issue that caused MX AutoVPN hubs to incorrectly advertise IPv6 IP addresses over AutoVPN. This could cause instability in AutoVPN routing and disruptions to clients that relied on the IPv6 connectivity.
• Resolved an MX 18.1 regression that resulted in MX appliances improperly routing traffic in the following configuration: 1) the spoke is configured in passthrough mode, 2) the spoke is configured with at least two AutoVPN hubs, 3) OSPF route advertisement is configured, and 4) there is at least one route being advertised from both hubs.
• Updated the AnyConnect VPN service.
• Fixed an issue that resulted in MX appliances configured in passthrough mode incorrectly routing ICMP fragmentation needed messages that were generated by the MX itself.
• MX appliances will now consider source-based routes when routing packets generated from the MX itself, such as ICMP ping messages.
• Fixed an issue that resulted in CoS tags not being set on traffic egressing a PPPoE uplink.
• Corrected an issue that caused No-NAT uplink overrides to not apply if the L2 PCP field was set to a value other than 0.
• Fixed an issue that caused MX appliances to incorrectly assess whether the WAN connection was working when the MX’s connectivity checks were redirected to a splash page. Appliances will now treat an HTTP 302 response as a failure, rather than a success when performing WAN health checks.
• Corrected an issue that could result in TCP flows being removed from flow tables sooner than expected under high volumes of network flows.
• Resolved an issue that could result in excessive CPU utilization when SD-WAN policies for Internet traffic were enabled and at least 8 tracking destinations were configured.
• Corrected an issue that could result in TCP flows being removed from flow tables sooner than expected under high volumes of network flows.
• Corrected a very rare issue that could result in web traffic being incorrectly denied when URLs were defined in the custom pie chart option through the Network-Wide>General section of the Meraki Dashboard.

Bug fixes - limited platform fixes

• Resolved an issue that resulted in various types of unexpected device reboots on MX75 appliances.
• Resolved an issue that could cause the IDPS process to fail when making certain configuration changes to WAN interfaces (such as disabling or enabling an interface). This issue may have also contributed to high device utilization.
• Resolved an issue that resulted in MX appliances operating in warm spare with a virtual IP Address (VIP) configured erroneously sending ICMP error messages using the WAN interface IP address, as opposed to the VIP.
• Corrected an issue that caused traffic to the Thousand Eyes Cloud to be incorrectly dropped If an MX appliance had a WAN uplink with an MTU lower than 1500 bytes.
• Fixed an issue that could result in ICMP fragmentation needed messages sent from client VPN clients being incorrectly routed on MX75, MX85, MX95, MX105, MX250, and MX450 appliances.
• Corrected an issue that resulted in MX75, MX85, MX95, MX105, MX250, and MX450 appliances failing to forward GRE traffic.
• Fixed an issue that could result in MX95, MX105, MX250, and MX450 appliances incorrectly forwarding CDP frames.
• Fixed an issue that caused MX75, MX85, MX95, MX105, MX250, and MX450 appliances to send more ICMP pings than intended for uplink statistic monitoring.
• Corrected an issue that resulted in MX75, MX85, MX95, MX105, MX250, and MX450 appliances reporting erroneously higher device utilization on the Meraki Dashboard.
• Resolved an issue that could result in IBGP sessions failing to form between AutoVPN peers when 1:M VPN NAT was configured on MX75, MX85, MX95, MX105, MX250, and MX450 appliances.
• Resolved a very rare issue that could result in firmware upgrades from MX 18.1 failing to complete successfully on MX75, MX85, MX95, MX105, MX250, and MX450 appliances.
• Corrected an issue that could result in duplicate Client VPN and AutoVPN Event Log entries on MX75, MX85, MX95, MX105, MX250, and MX450 appliances.
• Fixed an MX 19.2.1 regression that could result in MX75, MX85, MX95, MX105, MX250, and MX450 appliances servicing many clients to experience intermittent periods of heightened packet drops and latency.
• Resolved a rare issue that could result in MX250 and MX450 appliances becoming temporarily unresponsive after making VLAN configuration changes.
• Resolved a rare issue on MX67(C,W) and MX68(W,CW) appliances that could result in ports configured for 802.1X authentication getting stuck in an unauthorized state. This would result in a lack of connectivity for clients connected to the port.
• When the content filtering lookup queue is full, new flows that need lookup will fail open or closed (based on configuration). This lookup queue is generally expected to be full during error states, such as a lack of connectivity to the content filtering lookup service.
• Corrected a rare issue that could result in a URL persisting in the content filtering allow list if the URL was reconfigured from being included in the allow list to being included in the block list.
• Corrected an issue that could result in Z3C appliances failing to successfully form cellular connections using the integrated cellular modem.
• Z4C appliances will no longer include a blank APN in the list of potential APN options for connecting to Verizon.
• Fixed an issue that resulted in LAN packet captures failing for vMX appliances configured in routed mode.
• Resolved an issue that resulted in HTTP traffic being routed improperly when HTTP Content Caching was enabled.

Legacy products notice

• When configured for this version, MX64(W), MX65(W), MX84, MX100, and vMX100 devices will run MX 18.107.13.

Known issues status

• This list is being reviewed and updated.

Other

• MX appliances will now expire or age out DNS flow more rapidly than flows for other types of network traffic.
• Updated the default login credentials for the device local status page. The new login is a username of "admin" and the password is the device's serial number in upper case, with dashes included. If custom login credentials have already been configured on the Meraki Dashboard, those will continue to be used.

[Esorniloc]

Is there any documentation on "Added support for Active-Active Non-Meraki VPN peer connections." and what this actually means???

[GIdenJoe]

I haven't found any mention in the site to site documentation pages.

However active-active non-Meraki VPN means you need a route based VPN and you will be able to do load balancing and perhaps SD-WAN like features over the non-Meraki VPN tunnel.

However there is something strange that will need clarification.

- At this time you can only have route-based non-Meraki VPN if you use BGP.
- The documentation says that you can't have a secondary tunnel of you use BGP.

So either they will now finally allow route-baed VPN with static routes OR they will allow BGP over both tunnels.

These VPN adaptations surely have to do with their Secure Access integration because we didn't see such quick updates on VPN tech throughout the years.

[tushar01_01]

meraki is suggesting to upgrade to this version is there any recent issues observed for the people who have upgraded to the version.

Just to be on safe side🙂

[IFST]

Dear All,

We have established a site-to-site VPN between a non-Meraki peer, specifically a Cisco 800 series, and a Cisco 900 series device. The VPN is configured using IKEv1 with dynamic names, as we do not have static IPs. Our headquarters is identified by the dynamic name MX105, and it is also using a cloud dynamic name. Both sites connect perfectly; however, after approximately 10 hours, the tunnel drops.

When I checked the status with the command "show crypto isakmp sa," I noticed that the source and destination are in QM-IDLE, with both phases 1 and 2 still up. It appears that the Meraki device is not sending any traffic. After I disabled the crypto map and reinserted it, the tunnel re-established successfully.

Meraki advised me to upgrade the firmware to the most current stable version, which is MX 19.1.10, but the tunnel still went down after the same 10-hour period. I have since raised a follow-up ticket for further assistance.As my router does not have NAT TRAVERSAL and NAT T is automatic on Meraki i believe disabling no nat or nat exemption will solve the problem.

please let me know if anyone has resolved the problem.

[DMD]

Has anyone experienced Active Directory DNS entries not updating after upgrading to this version?   The time stamps on our DNS were being scavenged after 14 days of inactivity.   This was the only thing that changed.   We downgraded to the stable release of 19.1.10 and fixed the issue and DHCP entries began to populate correctly.  I believe 19.2.2 has a hidden bug or its the way they changed the DNS stuff.