New MR 29.4.1 stable release candidate - fixes bug where can't download config if it can't reach NTP

[cmr]

Wireless firmware versions MR 29.4.1 changelog

Important note

• Meraki APs use UDP port 7351 for cloud communication and TCP ports 80 and 443 for backup communications when running MR 27 and older firmware. When running MR 28+ firmware, Meraki APs will now use TCP port 443 as the primary means for cloud connectivity. In order to maintain connectivity to the Meraki cloud on MR 28+ ensure that TCP port 443 is allowed to communicate with 209.206.48.0/20 on firewalls that are deployed upstream of your Meraki APs. (Wi-Fi 6 and Wi-Fi 6E APs)

Legacy product notice

• When configured for this version, MR12, MR16, MR18, MR24, MR26, MR32, MR34, MR62, MR66, and MR72 will run MR 26.8.3.

Bug fixes

• General stability and performance improvements
• AP may be unable to download configuration when NTP servers are inaccessible

Known issues

• Sporadic packet loss & instability on Layer 3 roaming & Teleworker VPN SSID's (Wi-Fi 5 Wave 2 and Wi-Fi 6 APs)

[cmr]

We've seen the bug on some MR52s so will apply and test. 

[RaphaelL]

Out of curiosity , what kind of bug ? Was it under 29.4 ?

[cmr]

With 29.4 the clients on the two APs kept being kicked off every 2 hours or so. I logged a ticket with support and they said they couldn't get the config due to being unable to reach the configured NTP server. Oddly the other MR52s and MR 42s etc. on the same network were fine. Downgrading to 28.7.1 stopped the client kick offs but on one AP there is still an NTP warning every couple of hours.

I'm going to take the network to 29.4.1 and see what happens. 

[BlakeRichardson]

Thanks for sharing, great work as always @cmr 

[cmr]

I can confirm that 29.4.1 fixed the two misbehaving APs, see below where the red arrow points to where I upgraded:

[europa]

This latest firmware is so far working with the quality I expected. I installed it on my 183xMR56 about 36 hours ago. Since then, I have been observing carefully and have not experienced any trouble, and the terminals (Windows PC or iPad) connected to the APs continue to operate well.
Based on these results, the next step will be to install it on the 107xMR52.
Note: Prior to installing it, all APs were using MR 28.7.1.

[europa]

Following the recent installation, I have installed the latest firmware to my 107xMR52 as scheduled. It has now been about 36 hours since the installation. And as with the installation on the MR56, the connection is good.

[Adoos]

We have two issues - 

1. APs randomly disconnecting from the cloud. 
2. Roaming issue, when you roam between APs on the same floor (L3 enabled) you can roam to the next AP but no longer have the IP and see the DHCP error. 

Meraki could only suggest upgrading to the latest stable release, which we have done.

Can anyone confirm what "Sporadic packet loss & instability on Layer 3 roaming & Teleworker VPN SSID's" scenario means in real life? 

[Adoos]

**Update**

Upgraded to 29.4.1 and 802.11w protection has a bug since 29.2 which causes clients to not connect. 

Disable protection on the SSID and the clients can connect again.

Basically we saw the following and Meraki support noted the bug:  auth_mode='wpa3-802.1x' vlan_id='208' radius_proto='ipv4' radius_ip='10.39.7.5' reason='eapol_timeout' radio='1' vap='0' channel='60' rssi='31'

[WB]

How have you got security type set? WPA2/3 transition mode w/ 802.11w enabled or WPA3 only with 802.11w required?

[bigben386]

We are also seeing .11w issues with 29.4.1 on transition mode networks with enforced .11w.

[RaphaelL]

Same here. Seeing couple of these also : 

xtra='\<0600D5000A4D564D542D434F52504F01070C9824B048606C2102000C2432240128012C013001340138013C014001640168016C017001740178017C018001840188018C019001950199019D01A101A5012D1AEF0917FFFF00000000000000000000000000000000000000000030260100000FAC040100000FAC040100000FAC033C000100F3DD9DDBCA895701D3EA6D5D0AD20BD136033059003B1380515354737475767778797A7B7C7D7E7F8081460572000000007F0A04008880014000C00000BF0CB2799103FAFF0000FAFF0020DD070050F20200010002001067EB0551955CAF25D607FF05B9179D1203001641432D31372D43382D31312D45312D37353A7661703004001641432D31372D43382D31312D45312D37353A76617030050006AC17C811E175>' auth_mode='wpa2-802.1x' 11k='1' 11v='1' ft='1' reassoc='1' reason='invalid_pmkid' radio='1' vap='0' channel='153' rssi='37'

Station is a Win11 HP laptop. None of these were seen before upgrade to 29.4.1

[pjc]

I noticed that 29.4.1 has moved to stable in the last day or so.

Are there any outstanding issues with this version, any ongoing calls in with support ?

I like to keep up to date as new versions move to stable but if there are outstanding issues, I'll hold back until resolved

Thanks

[cmr]

@pjc we are running it on 125 APs of assorted types and it has been stable for us.  However at home I'm running 29.5 which has some unspecified bug fixes...

[Shawn_Kingston]

We typically won't upgrade if there are no current bugs/issues in the current configuration until the stable release has been out for at least 60 days with no major issues. 

[pjc]

I guess this is always the dilemma, upgrade as soon as production stable is available and take advantage of the new features/enhancements/bug fixes/security which may fix issues that are not always obvious (but may be unreported performance affecting), or wait longer until the majority of customers have been using it in the production environment.

I think there are quite a lot of fixes and new features in 29.x to warrant upgrading sooner rather than later.  My take is that by the time the version has gone from beta, to stable release candidate, to stable there is a period of time and and a number of deployments using it in the live environment to offer a level of assurance. 

However, having said that, I have personally seen in my deployment issues with firmware from Meraki that have caused us issues in the past, but that firmware had been available for some time before we upgraded, so it was not that we were early adopters then.  I think things have moved on from those times a few years ago, and that the development, testing, and staged deployment has improved greatly

[WB]

Echo both @pjc and @Shawn_Kingston 's comments really. We are coming from a Cisco world and so waiting for new releases to be tested and reported on by early adopters is a built in mechanism for us.

That being said, even with a stable release that's been out for a while you never know what edge cases your own envrionment will uncover! We're lucky enough to have a standalone test environment to pilot our upgrades with.

If we didn't then we'd likely wait until the automated Meraki 'we're going to upgrade your MR firmware for you' email comes through and we then get somewhat nudged to then upgrade

[pjc]

Forgot to mention, I always deploy a new version to a couple of AP's (temporarily move those aps into a defined 'firmware test network' running the new code) serving one of our high use area with typically IT dept. end users for a week and monitor before rolling out further.  

[RaphaelL]

Well 29.5 is out..

[bigben386]

Have you tried enforced .11w and transition mode with 29.5?