Wireless devices "jitter" and fast roam even to distant APs, mostly Realtek but now a Dell also

[RJordan-CCS]

Several months ago we started having severe wifi connectivity problems that affected customers using HP ProBooks (windows 10) at sites with either a combination of mostly MR42s and a couple of MR44s or a site with mostly MR44s and a couple of MR32s filling in dead spots.  We are not certain that the mix of APs is a problem but there is a lot of correlation; we have had far fewer problems at one of the customers alternate sites that only has MR42s.   AP firmware has been kept up to date.  I'd have to dig to find out where they started and when they were updated over the months.

Running HP latest drivers made no difference.  Rolling back to a 2021 Lenovo driver on the HPs actually did reduce the problems substantially (and re-updating to current HP brought them back with a vengeance).

The customer sees horrible performance.  The timeline in the dashboard shows it connecting and disconnecting every few seconds, sometimes switching from 2.4 to 5ghz and back again, often roaming to different APs, often distant ones with horrible s/n ratio, staying there for a few seconds, then roaming back.  Sometimes they will maintain one connection for several hours, then start this mess again for a few minutes to over an hour, even if the laptop is not actually moved.

In our office with two MR32s and an MR44 we were able to duplicate this.  Using an SSID with tags so we could run it just on the central MR44 or the front/back MR32s we confirmed that the problem occurred far more often when we walked the office with the SSID on all three APs, but still happened occasionally when roaming the two MR32s, and didn't happen with just the MR44 as long as we ran the 2021 Lenovo driver.

When it occurred, we would sometimes get the insane timelines, and when we didn't we could still see the event log entries showing most of the connect/disconnects  (there is or was a dashboard problem that Meraki is aware of when you just don't get timeline entries, or they are delayed showing up).

Now we have a Dell at one of the same sites, with a controller ID'd as a Windows 10 box with CHONGQING FUGUI controller.  It does not move when there, but is now doing the same thing.  It keeps jittering, disconnect/reconnect, roaming to distant APs then failing back, but this one adds a new wrinkle; we are seeing many repeating

Code 2 

multiple invalid authentication   (this is not them messing up the SSID passphrase)

Code 1

unspecified failure

Code 53

multiple invalid PMKID status

and

code 30

multiple Refused Temporarily

We were on the dashboard checking when the last bit of this was happening.  The two nearest APs (MR42s) were not heavily loaded and no other clients on them were having issues.   This one PC roamed to both and also 4 other APs further away during its tantrums.

The HP/Realtek fails got the code 1 and 2, and rarely code 30 events.  We never saw a code 53.  This one machine generated 1300 connection attempts and over 800 fails in a few hours.  It had an hour of problems, then ran fine for several hours, then another 45 minutes of problems before the user gave up.

That Dell is not a machine we have access to or control over (nor does our customer, we have to go through the end user); we have asked them to try to get up to date drivers and such installed.

I don't recall problems like this happening prior to about 8-10 months ago (Realtek) but the customers were much quieter then as many still weren't going in to the office sites.

Any thoughts?  

[alemabrahao]

Well, I think the first thing you can try is to give something different than in this article, just to review what best practices:

https://documentation.meraki.com/MR/Wi-Fi_Basics_and_Best_Practices/Channel_Planning_Best_Practices

https://documentation.meraki.com/MR/Wi-Fi_Basics_and_Best_Practices/Multi-SSID_Deployment_Considerat...

You can also try creating a new SSID on 5Ghz only, to validate if the behavior is the same. Review your Rf profile configuration, and SSID configuration.
On Windows machine you can try modify agressive roaming configuration.

[RJordan-CCS]

Thank you for replying.

I know we looked at the aggressive roaming option, I'll talk to the tech who was working the multiple complaint tickets at the time.  It did not make a difference with the really bad Realtek issues. 

We do not have that level of access to the current Dell problem system; belongs to a customer of our customer, but we can ask them to try that the next time they come in, assuming they have sufficient access to their PC (its also a company box for whichever company they are from).

We have discussed setting up separate 2.4 and 5GHz SSIDs with our customer and they really do not want to do that because they want everything set up as straightforward and simple as possible for their walk-in customers.   And this started fairly recently (the Dell started within the last week, the Realteks early this year)

[Andrew_Vinson]

I am dealing with this issue right now as well. I have tested with a different model of HP laptop since our issues seem to be related to the HP ProBook 445 G7 with a Realtek wireless adapter/driver. The other HP and other devices that are connected seem to be Intel wireless adapters/drivers. After troubleshooting on site today and getting the driver updated with the latest package from HP that device stopped roaming to different ap's for no reason and also stopped the constant connecting and disconnecting that you are experiencing. The Meraki model we have deployed is the MR36. All of the other steps of changing radio channels and transmit power and the suggestion above have unfortunately provided no help.

[RJordan-CCS]

Thanks for replying.  I guess we can check to see if there's a newer HP driver available; the last time we checked and updated was early September.

The Lenovo driver we're using with considerable improvement is labeled as "Realtek wifi RTK8822BE-CE Ver 2024_0_8_134 and came via Lenovo.  I will ask one of the folks in the office to see if htere is a newer HP driver worth testing.

[RJordan-CCS]

BTW if you don't mind can you post version info for the realtek driver that worked for you?  We're going to get a test system going tomorrow and seeing what HP, Microsoft, and Realtek direct have as current.  

[Andrew_Vinson]

@RJordan-CCS I don't recall the exact version number but the first 4 digits is 3000 or something like that I believe. I only saw success on our HP 445 G7 models. We have a few HP 455 G7's that gets a driver update from the latest package from HP but we are still seeing issues there. We are looking into either getting new laptops or replacing the Realtek wireless card with an intel one. The wireless card you mention is the one I am still having issues with. But these issues are related directly to our MR36 ap deployments. Previous to this rollout we haven't had issues and the affected laptops have no issues connecting to a personal hotspot or wireless outside of our campus.

[RJordan-CCS]

THanks for replying.  Our test box is a ProBoook 455 G8, though I know a couple of the customer boxes were G7s.  My managers say they have actually determined intel chip based cards are usable on at least some of the problem devices (not sure which versions) but they are 'backordered forever' so we are still waiting for them.  That does not help with our customers' clients though other than being able to inform them.

As you noted we have no complaints about usage of the Probook/Realtek combo from any sites running other wireless APs  (we thought we had one at a Ubiquiti site but just doing the HP driver updates fixed that one permanently and completely).   Whatever the issue is between the Realtek cards and the Meraki APs  (and now possibly that one Dell device) is becoming a significant issue; we had a couple of cases opened, with both leading to reduction of symptoms  but not complete fixes.

And now the Dell doing almost the same thing.  That user also states they have no problems anywhere else except at our customer's sites.

[Andrew_Vinson]

We are in the process of trying some usb adapters to see if those work any better. If this works, although not a great solution, the client devices will at least be able to be used until a better solution can be provided. I am attempting to open a case with HP so I will see how that goes. As you stated any device with an Intel chip doesn't seem to have this issue.

[RJordan-CCS]

I believe the PC techs provided some USB-wifi adapters as a temporary workaround; the users (our customers) did not find that to be a satisfactory solution; it was slow, though consistent since it didn't have the periodic flameouts.   But we really can't tell them to tell their own customers to plug in a foreign device just to get service.

Our test machine got updated to Win10 21H2, fully updated via HP image assistant and support assistant, but it didn't touch the 2021 Lenovo driver and claimed there were no newer drivers.  Since the HP website showed a May 2022 driver (same one we had all the fails with) and nothing newer, there's no magic solution for this machine.

I ran it back and forth with the lenovo driver in our office, no issues for 30 minutes.  Installed the new HP driver and same.  However it is a sporadic problem, might come back any time, so I'm not assuming anything.  I'll ask someone in the office to use the test PC on and off and check its timeline periodically to see if the problems recur.


And we just found out that at least two of the 'other' machines on this network that the dashboard identifies as "CHONGQING FUGUI..."  are actually probooks with Realtek controllers.  One a 450 G6 and one a 455 G7... so the device reporting is now suspect.

[Andrew_Vinson]

@RJordan-CCS We had no success with the USB wireless adapters. Had only one show some stability. We were able to find an Intel AX200 M.2 that we used in one of the affected laptops and had great success with it. I think our next step is to get a few more of those and see if it resolves issues for our other users. This replaced the Realtek adapter and drivers. That option is probably the best solution other than buying new laptops.

[RJordan-CCS]

I don't have info on the USB adapters they tried, but the reports were they were stable, just very slow.  I've asked if the AX200 you referenced are the intel adapters that my manager said worked but were not readily obtainable.  I certainly see them on ebay and amazon, so maybe they are a different option.   If I can talk them into ordering one we will give them a try,

Thanks!

[cmr]

@RJordan-CCS the AX200 / AX201 is usually found as a small M2 card that can be used to replace the existing M2 WiFi chip within a laptop. We did it a few years ago for some Dell Latitude and XPS models that had an Intel WiFi5 chip (8260/9260 I think).  It is an easy task to swap them and the ones we bought using those marketplaces worked well.

[Andrew_Vinson]

@RJordan-CCS we had great success with changing those adapters out to the Intel AX200. They were about $25 a piece that I think we sourced from a local tech store. I highly recommend going this route if possible. We have finished this upgrade over the last couple days and have see significant performance increases on these devices. 

[Macguy]

So I finally figured out the issue this morning.  It was a Realtek driver update via HP Support Assistant.  The model laptop is HP ProBook 455R G6.  The driver version from Realtek is 2024.10.227.2 and dated 8/18/22.  I'm seeing solid stable connectivity and ping times perfect.  No more thousands of logs saying client has roamed to AP X every few seconds.  Attached screenshot.  Prior to this ping times would constantly time out.  The laptop has stayed connected for hours since driver update (not seconds like before). 

[RJordan-CCS]

Thank you for posting that.  HP Support Assistant is NOT giving us that driver version on the last couple of G6 machines we have at customer sites.  I'll try to find that version to check.  I know the G7 and G8 systems we tried get 

Our problem customer's customer machine that the dashboard ID'd as a "CHONGQING FUGUI...", probable Dell turned out to also be a Probook with Realtek controller.  With their permission we installed the Lenovo-provided driver I listed above and its behavior improved dramatically.   

The dashboard still doesn't recognize it as a realtek device though; still says its a "CHONGQING FUGUI..."

Thanks again.  I'll report back results if we can get the driver version you listed for the G7/G8 machines we have ready access to and can test with it.

[Talhaturkdogan]

We had a similar problem, we contacted HP and asked them to work on the driver side.

[RJordan-CCS]

We reported the problem to HP also but have heard nothing back as expected.  So far the best band-aid seems to be the Lenovo driver mentioned above.  It still does not eliminate the problem but reduces it quite a lot.