Strange Behavior with MR34

[vassallon]

So we were having issues in one of our classrooms with iPads (iPad Air) not connecting other than the teacher's iPad (iPad Air 2). We reset the MR34 and even went so far to replace it with a Spare MR34. Now the student iPad Airs are connecting but teacher iPad Air 2 are not. Error Log for Teacher iPad

Even when trying to MacBook on the AP for troubleshooting we see the same type of messages. Both MR34 are running the Current version: MR 25.9 firmware. We are only broadcasting on 5 and have 2.4 turned off (Power 0). A cable test to the AP is also showing no issues.

Cable Test

Any thoughts as to what may be causing issues in this haunted room of ours? 

[Adam]

The devices work when connected to other access points on the same network (in the same building) just not one of them?  Because the error would seem to indicate an 802.11 issue.  You could make a temporary SSID with WPA2 to test and verify.  

[vassallon]

@Adam It's the same SSIDs that are broadcast and working everywhere else in the building. It just seems to be specific to this one classroom and there has been strange behavior with now two different MR34 in the room. 

[Asavoy]

It's easier to troubleshoot one device not connecting, rather than multiple devices.

Since it's currently just the one Air2 that's not connecting, have you done a network settings reset on it? 

The only difference between the Air and Air2 in terms of their wireless is that the Air2 is capable of 802.11ac, so that's a good place to look at the Meraki settings.  I had a wireless docu-cam (2.4ghz) that wouldn't even see my SSIDs with bandwidth steering turned on.  I've also seen some newer devices just not wanting to connect to the 5ghz band, and with bandwidth steering on, they couldn't even see the 2.4ghz band with inSSIDer.

[vassallon]

@Asavoy It is multiple iPad Air 2 having issues and also my test MacBook. We have tried resetting Network Settings on the teacher iPad and even tried a second iPad for her with the same issues. The iPads are running iOS 11.3 and 11.3.1 and the MacBook is OSX 10.13.4.

[Asavoy]

@vassallon If you're also having the issue on a Macbook, then definitely run a program like inSSIDer to see what bands the Macbook is seeing.  Depending on your setup, you should see each of your SSIDs and WAPs listed for each band that are broadcasting.  I had a problem room like that where the Chromebooks wouldn't connect to the staff network (or would have poor connectivity) but would connect to the open public network.  When I scanned the room for signal strength, I could not see one of the bands in that room, but could see the adjacent room.  Once I turned bandwidth steering off on the SSID I was able to see all bands.  It hadn't been an issue previously, but it's possible the forced update I had to go through did something.  (Yes, Meraki forced me to update and I couldn't opt out).

[vassallon]

Okay so this is even stranger now. The APs around the problem AP are showing a spoof with a MAC address that is remarkably similar to the MAC on the AP having issues.

[Asavoy]

That's not really out of the norm.  I take most of the dashboard results like that as being 'false-positives', especially with that being around the time you added it into the network.  (I'm assuming that anyhow).

Can you post a screenshot of a scan from a program like inSSIDer?

[PhilipDAth]

I have seen this before.  The "spoof" in the report is the key.  What happens is Meraki thinks one of its own access points is a rogue, and sends de-authenticate frames to prevent people attaching to it.

If you are not running 25.11 - upgrade to that first.  Failing that try rebooting all the Meraki APs in the area.

[vassallon]

@PhilipDAth Right now our APs are reporting up to date on 25.9 and I have a support case open with Meraki so I want to see what they have to say before rolling 25.11.

Thanks for the information though it does confirm that I was headed down the right path on the issue.

[Asavoy]

@PhilipDAth But, if that's the case, then how are the iPad Air connecting to it just fine?  Unless they're picking up signals from other rooms, but those devices tend to try to attach to the strongest signal.

[PhilipDAth]

Could be using a different band, could respond to de-auth frames differently, who knows.  All I know is I would be upgrading to 25.11 (as 99% of our customers already are using).

[Asavoy]

Personally, I'm not trusting my wireless infrastructure to Beta firmware when the 'stable' firmware is buggy.  The first answer to a problem should never be 'try beta firmware' unless the company is paying you to do it.

Besides, introducing a Meraki WAP into an existing Meraki infrastructure should never cause a 'containment' issue.  I mix 34, 42, and 52 and have never seen this.  Heck, I'm about to introduce an Ubiquiti device into my infrastructure for testing purposes.  This will be fun!

[PhilipDAth · Accepted solution]

Meraki firmware release cycles is not like what you are used to.

This is the general process:

https://documentation.meraki.com/zGeneral_Administration/Firmware_Upgrades/Meraki_Firmware_Release_P...

The bit to note is:
"When Meraki's install-base hit a specified threshold for a major version (roughly 10-20% of nodes), that firmware revision will be promoted to GA, pending a final formal review."

So you can have a "stable" release with bugs.  A "beta" firmware version may have these bugs fixed and been out for quite sometime, and all round be far superior. But until 10% to 20% of the nodes in the world deploy that beta release it can't be considered to be declared a "stable" release.

So in the worst case, if everyone in the world said they were only ever going to run the "stable" release, you would never get a beta image promoted to replace it.

Change to 25.11.  You wont regret it.  25.11 is all round much better than 25.9.

[Asavoy]

@PhilipDAth I don't have much of a choice as Meraki is now forcing an update to 25.11 on one of my networks.  Teaches me to come here and say anything bad about their software, right?

And I understand that the process of how they release the firmware isn't standard alpha/beta/gold type of thing, but I still look at their product as buggy as hell, especially updates that aren't "stable".  I would rather not be forced to be a guinea pig for things of this nature- I can screw up my wireless infrastructure on my own, TYVM.

[Asavoy]

@vassallon So, I think I ran into your exact same problem.  I installed a wireless temperature sensor into the room that has my main equipment but the wi-fi was spotty in the room.  I grabbed a MR34 that had been pulled from a site that's in the middle of a total refurb and placed it into the network.  It showed up in the spoof list in Air Marshal and I had nothing but issues with trying to get the temp sensor to connect stably- would maybe be on for a minute tops before dropping off.  I thought it was the temp sensor.

I installed another manufacturers Wi-fi device that had been recently donated- one that I've been begging my Director to get me a demo of to play with.  Of course, it shows up in Air Marshal on the spoof list.  However, the temp sensor has stayed connected to the network for hours with no issue.

As an aside, this part is the most eye-opening- the MR34 only showed up on 3 other MR34s as a spoof.  The other device showed up on 12 other MR34s.  That's some eye opening range difference!

I'm very interested in whether the upgrade to 'stable candidate' solves your issue, or if something else does.

[vassallon]

@Asavoy I'm still waiting for Meraki to determine why the mis-identification is occurring to begin with. I've avoided restarting APs around it to allow Meraki to troubleshoot. The strangest part is that it only flagged one of the two SSIDs that are broadcasting as a Spoof. The SSID for student devices is working with no issues, so it is allowing us to be flexible and leave the AP alone so they can troubleshoot and investigate the issue.

[Asavoy]

@vassallon You know, now that you mention it..... same!  My open public network did not show up as a spoof.  So, only the WPA2-PSK encrypted private network is being naughty.

[vassallon]

So after the upgrade to 25.11 the spoof has disappeared, still strange it occurred to begin with but at least all is working correctly now.

[Asavoy]

@vassallon So, has that also corrected the issue with the iPads not connecting?  Good to know that I can solve the same issue in my network, whenever 25.11 is available to select.  I got the push notification on a different one of my networks, so I let it run to see if there's other improvements.

[vassallon]

Yes, the 25.11 upgrade cleared all of the issues we were seeing with this AP being a "spoof." I see several devices connecting to the SSID was being considered a spoof previously.

[VascoFCosta]

Hi,

We're getting this issue in 25.13. Anyone else with this issue?

[Asavoy]

@VascoFCosta 

I'm usually not a fan at all of the beta firmware, but I can say that 26.2 has solved many issues I have.

I've also learned that when I have an AP or two that's being problematic compared to the majority of them, a hardware reset is in order.  I think sometimes doing incremental upgrades causes issues.

[VascoFCosta]

Hi@Asavoy  , thanks for the tip.

We reseted the APs during last night and apparently it's all stable for now.

[Lucasrg]

Hi,

I am getting a lot of problems with this AP now. I am seeing a mix of all the problems listed above.

The main thing is that it is disconnecting all the clients at once. After some customers get disconnected they have troubles to reconnect again,   DHCP issues with customers not being able to get an IP or it takes a long time to get one.

My question is: I would like to know if there is a version that is currently working. I just downgraded to version 25.13 and its being working for a couple of hours.

Please share what are your experiences with different versions.

Best Regards.

LRG

[vassallon]

@Lucasrg Did you try the 26.2 beta?

I'm interested in hearing if that resolves your issues, we have been having quite a bit of odd behavior lately that does not appear if we put a test MR42 in the same location. We're trying to determine if the random disconnects/slowness is due to the 25.13 firmware on the MR34 or something else being a gremlin in the system.

[ewwhite]

Was this ever resolved?

[vassallon]

@ewwhite The 26.2 release fixed quite a few of the issues we were seeing. Apple has also said there was an issue in previous versions of iOS which is supposed to be corrected in iOS 13, so we'll see what happens. 

[rowell]

How do you have Air Marshal configured?

[vassallon]

Just as an update. Meraki support has confirmed this is unexpected behavior and investigating why this is occurring on the AP. At this point it is only having limited impact. The strange part is that it is only affecting one of the two SSID we are broadcasting. I will wait to see what Meraki support can determine as I'm sure being able to troubleshoot an issue like this provides them feedback for improving the product in the future. 

Here is a screenshot of the Air Marshal settings for reference.

[rowell]

Does the problem go away if you change Air Marshal to allow clients to connect to rogue?

If you do a sniff of the beacons over the air, do you see your SSID being broadcasted from a rogue AP?

[vassallon]

@rowell I would suspect that if I changed the setting yes everything would work correctly. Meraki support has confirmed though that the "rogue" AP is actually our AP and not anything else. They are actively researching why the other APs are flagging this replacement MR34 as a rogue AP.

[rowell]

Ok. I missed the part where you mentioned your own AP is being marked as a rogue.

[redsector]

I have got eight wireless networks with MR32, MR33, MR42, MR52 (not MR34) on firmware version 25.11 and they are working well.

[Asavoy]

@redsector I guarrantee that I can find at least one post that makes that claim about 25.9