30.7 Update Breaks RADIUS

jpeterprarch
Here to help

30.7 Update Breaks RADIUS

We just recently upgraded our eligible APs to 30.7 from 30.5.  Immediately afterward all 802.11 auth attempts to our RADIUS servers failed.  Thus, we rolled back to 30.5.  Has anyone else encountered this issue?  I see in the 30.7 release notes a reference to a fix for RADIUS requests when if they had exited the primary management interface if an AMI was defined.  In our environment we don't have a AMI defined.

45 Replies 45
Brash
Kind of a big deal
Kind of a big deal

What are you using for your RADIUS server?

We have upgraded our fleet to 30.7 and are using RADIUS authentication without issue.

jpeterprarch
Here to help

Windows NPS

ammahend
Building a reputation

I have 30.7 in production and a test lab on 31.1.2 no issues with radius, I am using Cisco ISE as radius.

 

can you share more information like, if you are seeing logs on radius and share the logs to understand why radius authentication is failing.

RaphaelL
Kind of a big deal
Kind of a big deal

Exact same setup as @ammahend , running without any issues. 

 

WPA2-Enterprise , Cisco ISE reachable via AutoVPN.

jpeterprarch
Here to help

We're using Windows NPS.  The requests never hit the NPS servers therefore there's nothing to see in the logs.  

ammahend
Building a reputation

out of curiosity re-enter shared secret again on both side and see if it changes anything.

Mark_S
Meraki Employee
Meraki Employee

Hi jpeterprarch,

 

There is an issue in the 30.7 firmware, under very specific configurations, whereby client balancing rejects clients. This may be what you were seeing when you upgraded.

If you have not already, I would recommend raising a case with Meraki support so they can assist with troubleshooting this issue to help determine whether you were indeed affected by this.
Details on how to raise a case through dashboard or to call the support line can be found within the 'Get help & cases' page which is accessible in the ? icon menu in dashboard.

If you found this post helpful, please give it kudos. If my answer solved your problem, click "accept as solution" so that others can benefit from it.
jpeterprarch
Here to help

Ah, thanks.  I have opened a case and waiting to hear back.

ChristianHenn
Conversationalist

Meraki support said there is no known bug. Difficult to troubleshoot this topic when the manufacture can't confirme the bug.

jpeterprarch
Here to help

Well.  @Meraki.  Look again.  I received a single response from support asking the obvious questions.  Haven't heard back.  

redsector
Head in the Cloud

I did open a case but they didn´t really understand the problem.

skunkytoots
New here

30.7 broke Radius for my org as well

Brad39
Here to help

Hi ,

Adding to this as well, we have upgraded to 30.7 and have seen two networks with the same issue, generic Association errors with SSIDs with Radius Auth. Running Windows NPS. Will Raise a support ticket to investigate. 

jpeterprarch
Here to help

I'd be curious to hear what you hear from support.  So far I have yet to hear back.

pjc
A model citizen

@skunkytoots and @Brad39 do you both have client balancing enabled on the affected networks ?  I've scheduled 30.7 upgrades for next week and we run NPS Radius but we don't use client balancing

Thanks

jpeterprarch
Here to help

Yes we have Client Balancing enabled.  Perhaps we fall under "very specific configurations".

Brad39
Here to help

We have it on, however we tested turning this off to try and fix the issue but didn’t help

ChristianHenn
Conversationalist

We had the exact same issue. The only solution was to rollback.

pjc
A model citizen

I've decided to postpone my planned rollout of 30.7.  Although we don't use client balancing, @Brad39 stating that turning off client balancing didn't solve the issue is enough for me to be nervous about this, despite me running 30.7 in a small live environment (2 x MR44's) for the past week without issues with radius auth (my normal live test firmware update environment)

Brad39
Here to help

It’s a strange one, in the end this looks to have only affected 2 out of around 30 networks in our org, all very similarly configured. We are just going to hold off on 30.7 for those two and possibly jump to the next release.

redsector
Head in the Cloud

I have got the same issue

redsector
Head in the Cloud

I have got also issues with Cisco ISE 3.2 Errors occured since ugrading ISE from 2.7 to 3.2 and Meraki MR30.6 to MR30.7

AAABBBCCC
New here

Had this exact issue going from 29.X to 30.7.  Any SSID using radius was broken.  We backend with ISE.  Rolled back to fix for now.

user61762
Conversationalist

Similar experience here after seven networks with MR42 (and some MR86) APs upgraded from MR 29.5.1 to MR 30.7 overnight.

 

We have various SSIDs configured across the seven networks in question.

 

We experienced at least two networks where an SSID configured for Identity PSK with RADIUS did not permit clients to move past the association step.

 

Event logs showed clients attempting to associate but never moving to the wpa_auth step. After not too long, associations from clients on the SSID in question were failing with reason code 17.

 

Finding this thread (and another thread on Reddit), we disabled client balancing and the symptoms subsided.

 

The remaining networks that have an SSID configured for Identity PSK with RADIUS do not currently have active clients that would be attempting to use that SSID, so we don't yet know if they're affected.

 

Like others, we've opened a case with Meraki support to report the issue and seek more details.

pjc
A model citizen

Being reported on Reddit

https://www.reddit.com/r/meraki/comments/1e35rty/mr_307_causes_association_error_17/

I'm gonna hold of upgrading until confirmed fixed in future release

jpeterprarch
Here to help

This might be it.  Our affected SSID is in Slot9.

mags1892
Here to help

 this seems specific to the AP model , we have MR57 and 30.7 with no issues . However other offices with mr33 do have issues. Meraki have not been very forthcoming with support 

BWIT
New here

Hi

We had the issue with MR46's with NPS

 

One of my networks I applied the workaround of diabling the client balencing, the others I chose to roll back.

 

The best part is meraki support said

'a fix has been developed by our engineering team and should be available in the next firmware releases of MR 30.8 & MR 31.1.2'

However 30.8 is not yet avaible and 31.1.2 has already been superseeded by 31.1.3 in beta. 🙄

 

Even worse than that, I asked why this known issue was not noted on the release notes and could the add it, they said

'While it is true that there is a known issue with SSID and client balancing in version 30.7, not all customers utilize these specific features, so they will not be impacted by this issue.' ...'Furthermore, the request to update the release notes of firmware upgrades immediately upon issue identification cannot be fulfilled by Meraki Support directly. Nevertheless, we have the means to relay feedback and information to our internal development team. Notably, utilizing the "leave feedback" function'.👏

 

The point being, I dont know why they can't retrospectivly add a note the release notes of 30.7 to mention this known issue, its not a secret.

 

Finally, the best part, even though I have an active support case about this issue, Meraki's automation decided to schedule updates to 30.7 for my remaing networks last week. 😒

 

JMI
Here to help

As per the recent findings in the Reddit thread mentioned earlier - avoid SSID slot #9.

jpeterprarch
Here to help

Just read it (pun intended).  You have got to be kidding me.  Our affected SSID is in slot 9.  Now I haven't fully tested it yet, however, if this is the reason, this has to be one of the most absurd bugs encountered in my entire 30 year IT career.  I echo the sentiment of another post prior "Doesn't anyone test anything anymore?"

ChristianHenn
Conversationalist

Our affected SSID is in different templates in different slots...

JMI
Here to help

I now found they actually updated the 30.7 release notes a couple weeks back. Not that it would have made much of a difference to us, we only have three active SSIDs. But at  our main site (of course) one of them was in #9, due to the presence of various disabled legacy networks.  No RADIUS on that SSID, by the way.

 

One might be forgiven for thinking Meraki could have issued a warning to accounts where they saw this particular configuration, them having access to global inventories and all.

 

JMI_0-1725289288378.png

 

Brad39
Here to help

So looks to be an update in the "Known Issues". 

 

None of our affected networks are in slot 9 SSID, however they are in Slot 10. Meraki has updated the known issue to now be SSID 10. We are going to look at moving these from that SSID slot to another one in the next week or so and re-test 30.7...

unnamed (1).png

redsector
Head in the Cloud

Hello, where can I see the "slot"-number?

Brad39
Here to help

Under the network, Wireless > Configure > SSIDs and then click show all my SSIDs. Left to Right is 1 to 15

TBHPTL
A model citizen

No, in GUI left to right it is 0 , 1, 2 ,3 

TBHPTL
A model citizen

https://developer.cisco.com/meraki/api-v1/get-network-wireless-ssids/

 

Left to right in GUI is slot 0 , slot 1, slot 2, etc

Brad39
Here to help

Sorry you are correct!

 

Pulling it via API it’s “Number” 0-14 and via the GUI “Unconfigured SSID” 1-15. 

The terminology is prob not the best.

 

Sorry!

mags1892
Here to help

we only have 4 ssids and the radius issues is there . we downgraded firmware and used meraki proxy on the MRs but also had to upgrade firmware on the MX 

redsector
Head in the Cloud

Which MX firmware had you bevore and after upgrading?

 

mags1892
Here to help

MX 18.211.2 - 18.211.3 

MX - 75

redsector
Head in the Cloud

Thank you, very interessting. we will check it here, we also have MX75 and firmware MX 18.211.2

mags1892
Here to help

it does seem like this had a bearing whereas our mx100 worked without issues. 

JMI
Here to help

That's weird, so this bug is moving across the configured SSIDs? In our case it was definitely the 9th SSID that was affected (Default name "Unconfigured SSID 8" as they start counting from zero).

scpswifi
Here to help

Same issue here, I was using slot 10 with radius and it broke.  Moved the network to slot 8 and clients are connecting again.

Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.
Labels