Meraki MR 25.9 Firmware now Stable?!

[BBFred]

Seriously... I canceled the firmware upgrade, but how can firmware 25.9 with the following release notes be considered a stable release?!

Known issues

• Condition under investigation causes 2.4GHz radios to become unresponsive (MR32/MR72)
• Condition under investigation causes radios to become unresponsive for 5 seconds in high-density networks (MR34/MR32/MR72/MR26)
• Condition under investigation causes lower than expected throughput on the 2.4GHz radio (MR26/MR34)
• APs lose connectivity to dashboard until reboot in certain cases (MR33/74)

This is NOT okay!

[ww]

well yeah zero would be better but i guess not possible , but to get a better understanding of this knows issues, how many times this issues would be seen..?

what version where you running?

this one at least has less known issues then 24.12.

[redsector]

We are running 25.9 since a few weeks and I am happy with that release.

We use MR32, MR33, MR42, MR52 in different locations.

[Adam]

25.9 is the latest stable wireless firmware.  We usually upgrade one or two sites to verify there are no issues then we do the rest of our sites in phases.  All have been complete for a couple of weeks and we haven't had any issues. 

[PhilipDAth]

If a vender tells you there is no bugs in their software they are probably lying. Software is complicated and it is just a fact of life.

25.9 had been good for our clients.

[pjc]

Those known issues have been indentified as ongoing issues going back various versions of stable firmware, they just haven't been fixed yet.  I suspect the version that doesn't have those issues (if there is one, more likely have always had the issue) is very old

[MDS-tech]

Our 40 APs updated this week. We've been noticing some weirdensses - at random times several of the MR16s will go offline for no apparent reason. Timeframe ranges from 30 - 120 seconds, then POOF they are back on.

It's causing some consternation in the classrooms [high school].

Anyone else seeing anything weird?

[Adam]

Are you getting notices in the dashboard that the APs are going offline or just observed to those within range?

[MDS-tech]

Both...

A few APs are showing up as disconnecting in the dashboard.

The one in my office has glitched a few times, with no indication on the dashboard.

So far today tings have been normal [well, "what passes for normal" is more accurate!]

[MDS-tech]

Oooops... spoke too soon. Still seeing the same issue......

[Krewal]

After our upgrade to new firmware, we have noticed plenty of problems with connection stability. 

Load balancing does not work (clients are disassociated from holding AP but all other units are rejecting the client) which cause laptops adapters to freq-out and disconnect until they are manually turn off/on (wifi adapters). 

Also, clients have plenty of problem holding to SSID for a long time: random drops, without option to connect with SSID, client holds the IP even tho it is already disconnected, only previously connected SSID is visible).

So far turning off load balancing improved the situation but this is no fix, although it might be a case where there is fewer people in the office so I don't get that many tickets compare to recent days. 

In the meantime wired network works perfectly (wire & wifi are on the same subnet).

Anyone experiencing similar issues? Support is helpless right now... 

[PhilipDAth]

@Krewal I haven't had any reports of roaming issues amongst any of our clients using 25.9.  It has been pretty solid for us.

A couple of thoughts; have you got 802.11r enabled?  This really makes roaming a lot better.  We would have it enabled (and I mean "enabled" rather than "adaptive") on the vast majority of our clients.

[Krewal]

We decided to downgrade firmware to version >25.
All the problems are gone. @meraki_ please let us know how we can figure where is the heart of the issue.
The worst that might happen is "surprising" security update which is going to force us where we do not want to be.

[Uberseehandel]

---

@Krewalwrote:
We decided to downgrade firmware to version >25.
All the problems are gone. @meraki_ please let us know how we can figure where is the heart of the issue.
The worst that might happen is "surprising" security update which is going to force us where we do not want to be.

---

Real men don't test, strictly for the punters.

Whilst this comment is sardonic, not (properly)testing is inevitable in a marketplace that embraces rolling upgrades. Most (all?) of the big guys are at it. It is the world we live in.

[MRCUR]

@Krewal We're also seeing issues with clients either randomly being disconnected or not able to associate (and not seeing the SSID to associate to). More details in this thread (it would be helpful if you could post there as well with what you were seeing while on 25.9): https://community.meraki.com/t5/Wireless-LAN/Windows-laptops-Losing-Wireless-Connection-Firmware-25-...

[InfinityDrop]

We're seeing the exact same thing you are. We have 4 AP, and two of them are trying to service all the subscribers while the other two constantly point subscribers to the overloaded pair. Very weird, Meraki support worked on it some today, but couldn't find anything wrong with the configuration.

I've disabled Client Load Balancing for now, I'll update tomorrow to see if the problem units start behaving as expected. I may also revert to pre-v25 if I'm still having issues, or maybe the 25.10 SRC. *shrug*

[MDS-tech]

Any of you behind a SonicWALL? We noticed out connection counts were sky high and dropping connections due to the max number allowed. After meraki firmware downgrade and clearing the connections, they seem to be back to normal fingers crossed. We'll be monitoring it today.

[Krewal]

Bingo - yes we are behind the SonicWall. That sounds super weird tho because weird part of the Network is totally fine. 

How about the other experiencing the problem? 

I would test it further but I do not think we will do the test lab on 150 people office ;(

[MDS-tech]

watch your logs for a "full cache" system error. One of our "full cache" errors pointed to one of the access points so we put two and two together.  Our network seemed fine also, just dropping connections occasionally, a user may have just seen something "buffering" for a couple of moments a few times a day while the connection is regained. Anyway, worth a look.

[InfinityDrop]

We're behind an MX84 here. We have an MS220-48 that the AP aren't running off of, because it's not PoE. We adopted this network, and they put in a basic HP PoE Switch in a flat config to hook up the APs.

Turning off Client Load Balancing seems to have helped the issues we're having. People are able to subscribe again, but without the load balancing, two of the APs now have 26 and 29 clients while the other two have 6 and 7. On previous versions of the firmware where client load balancing was working better, this split looked much better. I may roll back tonight just to confirm and test.

[InfinityDrop]

Okay guys, try this on for size.

Apparently the APs were updated to 25.9 4 weeks ago. We didn't have any problems that anyone noticed until I updated our MX84 to 13.28 from 12.24. I got confused because I triggered an update for the MX and MS, and considering the issues we've had, assumed it had to do with one for the MRs as well.

This is a relatively new client for me, so it's hard to say for certain if there were issues when the update to MS 25.9 went off, but I'm putting my settings back and rolling the MX back to 12.24 to see what happens.

Weird, y'all. I'll report back tomorrow.

[MRCUR]

@InfinityDrop Are you just seeing issues with client load balancing or other issues as well? The MR 25.9 release notes do specifically call out changes to client load balancing. 

[InfinityDrop]

@MRCUR Well, 2 of the APs are taking on very few to no clients when client load balancing is enabled. Even when they do take a client it will stay connected for about 30 seconds then disconnect. Some clients managed to stay, but then they would disconnect at 30 minutes, or 2 hours. It was weird.

I saw the basic release notes, which always contain several super vague potential issues, but didn't see anything specifically about CLB.

"

Bug fixes
A corner case would cause AP to stop serving clients. (MR42/52/53/84)
A corner case where APs could be potentially re-flashed upon boot up (All MR’s)
APs would lose connectivity dashboard and reboot (MR16/66)
Performance improvements for Distributed L3 roaming (MR42/52/53/84)
Erroneous DFS detection on some APs under certain conditions (MR26/32/34/72)
Regulatory domain support for Curacao
Outdoor APs in the IC Regulatory Domain to not broadcast channels on frequencies higher than 5.35 GHz (Channel 64)
Event log Improvements
Client Steering performance improvements (All MRs)
BLE Performance Improvements (MR33)
Tuned MCS EVM to improve rate vs range (MR42)
Bridge Mode Wireless Client isolation prevents access to splash pages
Known issues
Condition under investigation causes 2.4GHz radios to become unresponsive (MR32/MR72)
Condition under investigation causes radios to become unresponsive for 5 seconds in high-density networks (MR34/MR32/MR72/MR26)
Condition under investigation causes lower than expected throughput on the 2.4GHz radio (MR26/MR34)
APs lose connectivity to dashboard until reboot in certain cases (MR33/74)
Other
General Performance Improvements

"

At any rate, I've rolled back my MS and MX, and the APs are doing the same thing. I'm not sure what triggered the issues we're seeing currently, but I've turned client load balancing back off and will update teh APs to 25.10 tonight to see if that changes anything.

[AReimann]

Several days later, how is going now @InfinityDrop?

I've noticed here some issues with load balance too, no matter if running 25.09 or 25.11.

[Lachlan]

Agreed, our MR32 has never crashed or had any problems. In the last 3 days it's crashed and needed a hardware reboot twice. Please either roll back firmware version or fix the errors!