MR 26.7 Firmware

[NolanHerring]

New

• Initial stable firmware for MR36/MR46/MR56
• Client association syslog messages now use the verbiage "last_known_client_ip" for association and WPA authentication messages to better represent the IP being reported (All MRs)

Bug fixes

• Air Marshal considered VAP (SSID) numbers 5-14 as non-Meraki and would report as spoofs or contain BSSID based on Air Marshal configuration (MR45/MR55)
• 802.11ax information elements broadcasted after disabling in RF Profiles (MR45/MR55)
• Incorrect WMM parameters (ECWmin/max, CWmin/max, TXOP) after 802.11 PHY mode change (802.11ac Wave 2 MRs)
• Invalid memory reference leading to reboot for mesh repeater (802.11n MRs)
• MR was no longer sending ARP probes to check for passive client associations (All MRs)
• SSIDs configured with Umbrella integration did exclude domains configured exclusion if DNS query was TCP (All MRs)
• Walled garden domains not updating for SSIDs configured with Umbrella Integration (802.11ac Wave 2 / 802.11ax MRs)
• Mesh repeaters flooded DHCP Discovers when Client Isolation was enabled (All MRs)
• Beacon rate did not honor the per-SSID configuration if multiple SSIDs had differing rates configured (802.11 ac Wave 2 MRs)
• SSIDs matching a blacklist policy were not contained if the blacklisted SSID used a non-ASCII character (All MRs)
• Default RX-SOP configuration could contribute to degraded 2.4 GHz performance as compared to MR24.x firmware (MR42/MR52/MR53/MR84)
• DFS event detection was overly sensitive (MR42/MR42E/MR52/MR53/MR53E/MR84)
• Wireless Health Latency only displayed Best Effort traffic (802.11ac Wave 2 MRs)
• Channel utilization graph on the "RF" tab of the Access Point details page did not populate data reliably (802.11ac Wave 2 MRs)
• MRs in Costa Rica were unable to operate with 80 MHz channel widths (802.11ac Wave 2 MRs)

Known issues

• VLAN override via RADIUS authentication is not properly applied (MR36/MR46/MR56)
• Downstream VOIP RTP Packet Loss (MR42/MR42E/MR52/MR53/MR53E/MR84)
• Sporadic packet loss & instability on Layer 3 roaming & Teleworker VPN SSID's (All MRs)
• Reduced aggregate upload throughput on 2.4GHz radio for Windows clients (802.11ax MRs)
• EAPOL Key 3 is not sent from AP in the CE regulatory domain to 802.11b/g clients on PSK SSIDs (802.11ax MRs)

[nscheffer]

Hi,

Applied to 5x MR52 and one MR84, solving Wireless Health bug dashboard with only Best Effort traffic !

Good update, thanks.

[PhilipDAth]

I've just upgraded our MR45 to it as well.

[CptnCrnch]

Looking good so far on MR45

[jrhop]

Upgraded a few sites with MR33's and can also confirm the Wireless Health is now working and not just showing as Best Effort. Took them a couple of years but fixed it in the end!

[pjc]

Interesting that 2 of my (newish) networks running 26.6.1 (small test networks) have automatically been scheduled to be updated to 26.7 despite me having 'try beta firmware' option disabled.  It looks like Meraki are pushing out 'stable release candidate' firmware as part of the standard firmware updates (unless you have requested, as I have for my other networks, that I don't have automatic updates)...anyone else seeing this?

Is this an attempt by Meraki to push out firmware 'under the radar' quicker so a greater uptake on the live enviroment (so they can gauge issues quicker before they move it to full stable) than if relying on us proactively upgrading....

[jrhop]

Hi @pjc We have had this a couple of times in the past and also have try beta firmware disabled. When I have contacted support they say that a stable release candidate is supported the same as a stable release. They have always emailed me before doing it and saying when it is going to take place and you always have the option to cancel the upgrade. Did you receive an email/notification about the upgrade being scheduled.

[NolanHerring]

---

@pjc wrote:

Interesting that 2 of my (newish) networks running 26.6.1 (small test networks) have automatically been scheduled to be updated to 26.7 despite me having 'try beta firmware' option disabled.  It looks like Meraki are pushing out 'stable release candidate' firmware as part of the standard firmware updates (unless you have requested, as I have for my other networks, that I don't have automatic updates)...anyone else seeing this?

Is this an attempt by Meraki to push out firmware 'under the radar' quicker so a greater uptake on the live enviroment (so they can gauge issues quicker before they move it to full stable) than if relying on us proactively upgrading....

---

Had this happen yesterday with two sites that are still on 26.6 (not 26.6.1), wanting me to get them to 26.7, which I plan on doing.

[PawelG]

As far as I understand beta is not release candidate.

You have separate "beta" software releases (currently not for MRs). RC software is treated as release one in case of automatic updates.

Br, Pawel. 

[cmr]

As far as I'm aware, with Meraki:

Beta = new major release

Release candidate = new minor release

Once more than ~10-15% use it, it becomes release.

[pjc]

@jrhop  I did receive an email telling me they had scheduled the update, and those scheduled networks were also displaying the banner, so in fairness to Meraki, yes the relevent notification and opportunity to cancel or change the date if desired

It was just my surprise that they had scheduled SRC at all, but upon reading the documentation, it would appear to be the standard practice...

Stable Release Candidate

As a new firmware version matures from beta, it has the opportunity to graduate into a stable release candidate. A formal review of the beta firmware success is conducted by our software and product teams. KPIs for quantifying firmware quality are analyzed including open support cases & engineering issues, firmware adoption, and stability metrics. After the formal review a beta may be reclassified as a "Stable Release Candidate". At this point the firmware version will be indicated as such in the firmware upgrade tool. Once a new stable release candidate is available, Engineering will begin scheduling a limited set of customers for upgrade. These upgrades can be canceled, modified, or reverted using the firmware upgrade tool on dashboard.

[PawelG]

Upgraded all MR45's. Working well although the bug with incorrect power level for non-maraki AP is still there. A lot of neighbor SSIDs with 160dB signal reports. Annoying.

Br, Pawel. 

[cmr]

Running it on a WLAN with 1xMR32 and 2xMR34, when connected to the MR32 I am NOT seeing the packet loss/timeout issues that some others complained about with 26.6.1.   I haven't yet had the chance to test our MR32s on 26.6.1.

[NolanHerring]

---

@cmr wrote:

when connected to the MR32 I am NOT seeing the packet loss/timeout issues that some others complained about with 26.6.1

---

What issues were people having? I don't recall reading anything negative yet for 26.6.1

[cmr]

@NolanHerring In this post

https://community.meraki.com/t5/Wireless-LAN/Is-MR-26-6-1-stable-release-safe-to-upgrade/m-p/75110#M...

It was being discussed that that MR32 particularly had issues with 26.6.1.

[NolanHerring]

Yikes, didn't realize there were issues. Thanks for the link.  I'm about to implement some MR42's and that last poster claimed he had issues with it coming from 25.13

@jdsilva - i know you've said you have lots of MR42s.  You rocking 26.6.1 by chance?

[cmr]

We have about 60x MR42s running 26.6.1 and haven't seen any particular issues.

[jdsilva]

---

@NolanHerring wrote:

@jdsilva - i know you've said you have lots of MR42s.  You rocking 26.6.1 by chance?

---

Nope. 

[NolanHerring]

---

@jdsilva wrote:

---

@NolanHerring wrote:

@jdsilva - i know you've said you have lots of MR42s.  You rocking 26.6.1 by chance?

---

Nope. 

---

We need you to take one for the team

Upgrade all your device and let us know how it goes 😃

[jdsilva]

---

@NolanHerring wrote:

---

---

We need you to take one for the team

Upgrade all your device and let us know how it goes 😃

---

Nope.

[pjc]

@cmr  That's good to know that no timeout issues with MR32's and this version 26.7.  Will subscribe to this thread.  No mention in the release notes of any bug fixes with MR32's about connection timeouts on this new version, despite talk of a known issue on the other thread.

I've held off updating my network as most are MR32's and the issues reported with 26.6.1 from those on the other thread scared me off.  Had my fingers burnt too many times before as an early adopted on flaky Meraki firmware

[antonis_sp]

26.7 seems to solve issues we were seeing in MR33s.

So far it seems good to deploy.

[NolanHerring]

---

@antonis_sp wrote:

26.7 seems to solve issues we were seeing in MR33s.

So far it seems good to deploy.

---

Which issues if you don't mind me asking

[cmr]

@pjcI finally managed to test one of our MR32s on 26.6.1 and I don't get the timeout issues that @randhall was seeing.

[NolanHerring]

---

@cmr wrote:

@pjcI finally managed to test one of our MR32s on 26.6.1 and I don't get the timeout issues that @randhall was seeing.

---

I have a feeling the 'bug' doesn't show its head immediately. I think it takes days until it crops up. Could be wrong but that has been my experience in the past with other firmware bugs where things are fine and then 2 weeks later a 5GHz radio stops responding to association requests, forcing me to reboot it etc.

[cmr]

@NolanHerringI think there must be something else involved as the particular AP I tested today was upgraded a month ago and has been up continuously serving 5-20 clients at any one time since...  Currently it has two on 2.4GHz and fourteen on 5GHz.

[pjc]

@Randal  wrote on the 26.6.1 thread about the connectivity issues with his/hers MR32's

"We weren't given a bug number by support but they said it was a known issue. It extends a couple of revs back. Only on MR32s (150) out of our 750+ total."

Just need Randal to confirm if the bug was fixed or still outstanding , and/or Meraki engineer to advise further ( calling @davidvan @Noah_Salzman )

cheers

[ether]

Anyone else have feedback on this firmware? I'm considering jumping from 25.13 to 26.7. We have about 5 MR32 APs and 65 MR33 APs.

[NolanHerring]

So far I've got about 300 access points on it across about 10 sites, no issues yet 😃

[jrhop]

We have 116 MR33's across 35 networks and its been rock solid.

[cmr]

We have ~120 APs across ten sites running 26.7 and so far it looks good.  Models include 32,33,34,36,42,52,55,72  About 20 are MR32s and eight are MR33s

[Sufyan-Samar]

We have MR-55 in our environment, we have DHCP scope on a L3 switch, I upgraded to 26.7 and suddenly started getting so many logs like client declined IP address, every client would decline about 10 to 15 IPs and than take one, so support team suggested to role back to 26.6.1, in 26.6.1 a lot of 2.4 Ghz clients face issues, it works and in a day or 2 without any changes any where it stops working, 2.4 Ghz clients are not able to associate only on the network.

[NolanHerring]

---

@Sufyan-Samar wrote:

We have MR-55 in our environment, we have DHCP scope on a L3 switch, I upgraded to 26.7 and suddenly started getting so many logs like client declined IP address, every client would decline about 10 to 15 IPs and than take one, so support team suggested to role back to 26.6.1, in 26.6.1 a lot of 2.4 Ghz clients face issues, it works and in a day or 2 without any changes any where it stops working, 2.4 Ghz clients are not able to associate only on the network.

---

To be fair, I would quarantine MR45/MR55 model issues as unique because those have been having issues since inception.

[Sufyan-Samar]

So what is the solution, we recently migrated from on premise wlc to meraki and selected the latest model assuming it will be good but there are lots of issues with radio, we never had any such issues in our previous setup of cisco access points with wlc

[cmr]

Which model do you have?

What firmware are you running?

Which issues are affecting you most?  Someone here can usually help.

[Sufyan-Samar]

Wireless model is MR55, we are running 26.6.1, the problem is the devices which only supports 2.4 Ghz usually doesn't get associate, and lots of problems for sticky clients, clients are not getting connected on the nearest AP even though strong signal is available with different channel number, check the coverage heatmap as well and it seems fine

[cmanglin]

Wanted to join in here.  We are seeing similar situation at two locations with DHCP issues as well. 

Overview

Cisco 2960XR - Core Switch

Microsoft Windows 2012 - DHCP Server

Site1: MR42E/MR45

Site2: MR55s

We upgraded to 26.6.1 and had issues with barcode printers connecting on 2.4 at Site1.  When we upgraded to 27.7, this fixed that issue but seems to have introduced a new issue with DHCP BAD_ADDRESS.   We are seeing DHCP Logs fill up with this coming from multiple SSID's on different VLANs.   We have this happening at two sites and the common factor is Meraki 26.7. Site2 does not have barcode printers so we will test downgrading this site.

We have window tentatively scheduled for this weekend or early morning next week so I'll post another update once testing is completed.  Do not feel I'm getting much support on ticket opened with Meraki.

[cmr]

We have some of these dhcp bad address issues in the logs, but haven't seen an actual issue, @cmanglin are you seeing a real issue?

[cmanglin]

@cmr   Yes, this BAD_ADDRESS.  We keep seeing the clients try consecutive IPs utilizing DHCP addresses so maxing out our scope.  We have to keep clearing them out but now have a script to do this for us so we do not get calls while we troubleshoot the issue. 

[cmr]

We use Cisco IOS switches for the DHCP and it doesn't have the same effect, we only get the log entry...

[Doug100]

Hi we are seeing exactly the same issue in a similar setup. Rolled back firmware fixed the issue. Meraki support cast opened, I wish they would be more pro-active on these issues, anyway, waiting for a resolution. 

[cmanglin]

Exactly... Here is what support told me.

It could potentially be due to layer 3 roaming being enabled as devices will hold on to their IP address while roaming.
You could try one of 2 things:
1. Roll this network back to 26.6.1 and review the DHCP logs
2. Disable L3 roaming for the SSIDs on this network and review the logs.

It seems they know rollback fixes issue and should only be suggesting this and need to quite offering 26.7 as a "stable release candidate".  

[cmanglin]

We did try option 2 to no avail. Rolling back firmware to 26.6.1 did resolve our BAD_ADDRESS issue on Windows Server.

[NolanHerring]

So I have several sites, hundreds of users, running on 26.7
No issues yet, seems as stable as 25.13 is/was
I also do not see any of those DHCP issues (Windows Server 2016)
Normal bridge-mode dropping off onto a VLAN, so none of that L3 roaming stuff

Not sure if this is directly related to the L3 roaming (which honestly I've always strayed away from, and over the years the vibe I always got from that via the community was its a bit finicky), or the new MR45/MR55 models specifically having issues (would not surprise me based on community feedback so far).

[pjc]

@NolanHerring  Hi Nolan, thanks for the feedback.  Presumably the customers you support who are not reporting issues are running various AP models on v26.7 ?  Have you a number running MR32 and MR33 without issues?

Thanks

[NolanHerring]

I have MR33/MR42/MR42E/MR52. I have older sites on MR32 still, but they are still on 25.13

[cmr]

We have 20x MR32s on 26.7 and when I went to one of the sites today the SSIDs were nearly all set to layer 3 roaming, but they were working fine...  I only changed them after being there for an hour and did some heavy data work for 30 minutes while connected to an MR32 first...

[NolanHerring]

Are we talking 'Layer 3 Roaming' or 'Layer 3 Roaming with a Concentrator' ?
Try the TEST button for 'Layer 3 Roaming' to make sure everything works right too

[cmr]

Without a concentrator, effectively just a crappy version of bridged mode for us as the APs have the same VLANs/subnets.  A leftover from when we hadn't figured out what it really meant! 🤫

[cmanglin]

great point but we had tested "layer 3 roaming" - successful

[cmanglin]

Bridge mode with 802.11r did not work for us (legacy client issues). We have went back from 26.6.1 to 26.7 twice to prove the issue out

[cmr]

We haven't used r for a while as we use WPA personal and there is a security risk if combined 

[Brambo]

Hi guys,

We also had the bad addresses issue. We solved it by disabling the mesh feature on the wireless. This was also confirmed by meraki that this is an issue on the new mr45/55  models. Hope this helps!

[NolanHerring]

Thanks for the response @Brambo

I had a feeling this was directly related to the new first gen 802.11ax MR45/MR55 models.

[cmr]

Ah, that is why we didn't get the actual problem, just some alerts, we had disabled meshing!

[cmanglin]

Mesh was disabled on our site and we continued to see BAD_ADDRESS.

[Brambo]

We also disabled 802.1r but that was not related as to my understanding and what the engineer told us.

[cmanglin]

Meraki support recently came back to us with the following.  We will pass on this for now and wait for it to be fully tested but FYI out there to others who may not be able to afford to wait around. 

Greetings, The product team reached out and mentioned they have recently received reports of issues of this nature occurring on 26.7.

They have provided a test firmware that needs to be applied on the affected networks.

As this test firmware cannot be scheduled, you would need to call in during a maintenance window (as your APs will reboot on the firmware upgrade) so that we can apply it for you while on the call.

[NolanHerring]

So I think the meshing was a red herring, and the real issue is the MR45/MR55 =/

[RumorConsumer]

Red herring or @NolanHerring, you decide 😆

[Brambo]

Hi guys,

Bad news this morning we had again bad addresses in our dhcp scope so yes red herring :-).

Good news meraki installed a patch and we havent had the issue anymore!

So spam meraki support for the patch if you really need it! 😉

Greets

bram

[NolanHerring]

---

@Brambo wrote:

Hi guys,

Bad news this morning we had again bad addresses in our dhcp scope so yes red herring :-).

Good news meraki installed a patch and we havent had the issue anymore!

So spam meraki support for the patch if you really need it! 😉

Greets

bram

---

What are your specific conditions that are having you run into this issue?

MR45/MR55 model?

L3 Roaming on the SSID?

Just want to validate what it is causing it since I'm not having the issue.

Did support tell you what version they pushed you to? 26.8? Did they mention what the specific bug was?

[Brambo]

MR45 and we are using bridging not l3 roaming .We run our layer 3 on a MS355 switch and dhcp relay to a microsoft dhcp. We had the same issue with the DHCP running on the switch(pool was running full) at first thats why we migrated to a microsoft dhcp server.

[NolanHerring]

Thanks for the feedback @Brambo

I think this just confirms the MR45 and MR55 are still having a continuation of issues since their conception.