New MX 16.14 stable release candidate - not beta any more and assorted reliability fixes

[cmr]

Security appliance firmware versions MX 16.14 changelog

Important notice

• While Meraki appliances have traditionally relied on UDP port 7351 for cloud communication and TCP ports 80 and 443 for backup communications, with MX 16 we are beginning a transition to using TCP port 443 as the primary means for cloud connectivity. In order to ensure proper connectivity to the Meraki cloud after this upgrade, please ensure that traffic using TCP port 443 between 209.206.48.0/20 is allowed through any firewalls that may be deployed upstream of your Meraki appliances.
• HTTP proxy, which allows default management traffic from MX appliances to be sent through a proxy, is deprecated on MX 16 and higher firmware versions.

Legacy products notice

• When configured for this version, Z1, MX60, MX60W, MX80, and MX90 devices will run MX 14.56.

Bug fixes

• Updated the AnyConnect VPN service
• Resolved a case that could incorrectly result in brief periods of heightened device utilization.
• Corrected an issue that could result in communication issues between Z3(C) appliances and client devices connected through a port that was configured to operate at 100Mbps link speed.
• Fixed a case that could result in group policies not correctly applying to client devices.
• Corrected an issue that could result in layer 7 firewall rules still taking effect for clients with the whitelisted group policy applied.
• Resolved an issue that resulted in MX appliances operating in a backup/spare/standby capacity in a high availability pair improperly attempting to establish non-Meraki site-to-site VPN connections.
• Fixed an MX 15.12 regression that could result in some Macbook clients running some older software versions being unable to reconnect to the client VPN shortly after disconnecting.
• Corrected an issue that resulted in "Ethernet Port Carrier Change” Event Log messages referencing an incorrect port number on MX95 and MX105 appliances.
• Resolved a rare case that could cause a device reboot when processing fragmented traffic.
• Added the ability for MX65(W) appliances to automatically recover when an error causes PoE to stop being provided.
• Fixed an issue that resulted in the PoE power indicator on the Appliance Status page displaying on the incorrect port for MX75 appliances.
• Resolved an issue that resulted in Z3(C) appliances that upgraded to MX 16 versions being unable to directly downgrade to MX 14 releases.
• Corrected a case that could theoretically result in MX250 and MX450 appliances not properly rebooting after a crash had occurred.
• Upgrade reliability improvements for MX64(W) and MX65(W) appliances.

Known issues

• After making some configuration changes on MX84 appliances, a brief period of packet loss may occur. This will affect all MX84 appliances on all MX firmware versions
• Due to MX 15 regressions, USB cellular connectivity may be less reliable on some modems
• Due to an MX 15 regression, the management port on MX84 appliances does not provide access to the local status page
• Client traffic will be dropped by MX65(W), MX67(C,W), and MX68(W,CW) appliances if 1) The client is connected to a LAN port with 802.1X authentication enabled and 2) The VLAN ID of the port is configured to 16, 32, 48, 64, 80, 96, 112, 128, 144, 160, 176, 192, 208, 224, or 240.
• BGP-learned routes may not be properly reflected in the Route Table page on the Meraki Dashboard, despite BGP and packet routing operating correctly.
• There is an increased risk of encountering device stability issues on all platforms and across all configurations.

[PhilipDAth]

This is awesome news!

[ViniciusFranca]

Wonderful version.
For my company it is very stable.

Alias, prediction of having it as stable?

[cmr]

It will be once a certain percentage of us use it, we run v16 on all our MXs bar one that runs v17 to test the IPv6 features.

[GIdenJoe]

Does that mean we're getting closer to paid Anyconnect?

[PhilipDAth]

>Does that mean we're getting closer to paid Anyconnect?

It's the same AnyConnect licences you already need to use it ... it's just on an honesty basis.

[CptnCrnch]

As per that BU, there won't be any license validation on the Dashboard, so as @PhilipDAth already said, it's still the trust based model.

Of course, this could change in the fututure. 😉

[danguita]

Hello, we have had a problem with the last upgrade to this version (MX 15.43 → MX 16.14).

Later than the upgrade we had several problems with the traffic between this SPOKE and the HUB. Concretly, the problems were found in Lan2Lan traffic, we didn't have traffic from LAN of Spoke to the LAN (servers) in the HUB.

Another problem that we found was with the NBAR applicattion: we don't know why this application recognized the local destination with IP 192.168.1.X as a Facebook traffic. We had to delete the Facebook layer 7 rule.

Finally, to resolve the problem, we had to do the rollback. Is it possible that 16.14 version has another bug?

for the time being, we have the automatic updates stopped.

Kind regards

[Eric123]

Was 16.14 pulled?  I no longer see it as an option to upgrade our devices.

[cmr]

@Eric123 still showing up here for an appliance on 16.12:

[danguita]

Hello, it's still on stable release candidate.

[danguita]

Hello, it's still on stable release candidate.

[jay_b]

Hello @cmr 

On MX15.x release notes I can see this :

"

• Due to underlying changes present in MX 15, MX appliances will now strictly validate the remote ID parameter during VPN tunnel formation. If you notice issues with non-Meraki VPN tunnel connectivity after upgrading to MX 15 for the first time, please ensure the remote ID configured in the site-to-site VPN page for a given non-Meraki peer matches what is configured as the local ID on that device."

But I don't see this in 16.14, does it mean it doesn't validate the remote ID parameter strictly any more ?

[cmr]

@jay_b I don't know for sure, but I'd be willing to bet that this has carried forward and is only not noted due to it not being a new feature.

[AlexP]

This is correct - between MX14 and MX15, we changed to a new IKE/IPsec management process, which lead to the stricter requirements

[GIdenJoe]

Any idea you could implement a debugging feature for the authentication part of IKE and perhaps more debugging for re-key phases?  We can't use packet captures for this because the authentication messages are encrypted so we don't know how the remote IKE ID arrives and how we send the local IKE ID.

[AlexP]

Yes, talks have been in the works for a while about this. Trust me, Support wants this as much as anyone, but we also need to make sure it's done right.

[PhilipDAth]

Oh god, yes, please!

I suspect this isn't as sexy as writing new features, or mundane as fixing bugs, so is less attractive for the developers to do.

[ViniciusFranca]

Hello!

What's the difference between 16.14 and 16.14?

[cmr]

The beta simply moved to stable release candidate after enough stable deployments were seen

[ViniciusFranca]

yes, but all my environment were with 16.14 version, and suddenly the new version appears and old version is excluded!

Its complicated.

[cmr]

@ViniciusFranca you can get support to install a non-current release, but why would you?

[Net-Wrok]

We have observed the issue after upgrading our MX400 (hub) on verison16.14. Post upgrade we had observed problem where the  MX looks to performed a software initiated reboot intermittently. we also make our spare MX as primary but It did not resolve the problem. At last we had to roll back to version 14.53 and device seems to stable.

Does any one faced same issue ? or 16.14 version has a bug ?