Client VPN Error After January Windows Updates

SOLVED
Darcon
Here to help

Client VPN Error After January Windows Updates

After patching a couple machines with the January Windows Updates I found that our Meraki Client VPN no longer works. I get error 789 in Windows Event Viewer, and nothing at all in the Meraki Event Log.

 

This is the error displayed when attempting to connect. It errors out within a second of hitting connect, doesn't even prompt for creds.

 

Untitled.png

 

We've had AssumeUDPEncapsulationContextOnSendRule set to 2 for some years. I tried removing / rebooting / re-adding this key; same error. I also tried switching our PSK to a key without special characters, same error.

 

I have confirmed this on 2 different machines. Unpatched machines continue to connect without issue.

 

Anyone else run into this today? 

1 ACCEPTED SOLUTION
MarkChan
Here to help

Microsoft released an Out-of-band patch:

 

Windows 10 - KB5010793

January 17, 2022—KB5010793 (OS Builds 19042.1469, 19043.1469, and 19044.1469) Out-of-band (microsoft...

 

Option 1:

Run Windows Update, KB5010793 will appear under optional download.

 

Option 2:

Download the patch from there: Microsoft Update Catalog

Please download the matching Windows 10 Version.

 

Windows 11 - KB5010795

January 17, 2022—KB5010795 (OS Build 22000.438) Out-of-band (microsoft.com)

 

Option 1:

Run Windows Update, KB5010795 will appear under optional download.

 

Option 2:

Download the patch from there: Microsoft Update Catalog

View solution in original post

162 REPLIES 162
BrandonS
Kind of a big deal

Not today (yet), but pretty much every Windows update brings me service tickets about broken client VPN and 100% of the time they are resolved by deleting and creating the VPN again.  There is probably a quicker way and an underlying reason, but I have not cared to dig further personally..  

 

EDIT: Apparently this time requires removing a Windows update.  Keep scrolling..

I gave re-creating the connections a try, no luck. I also tried running through my mobile hotspot to no avail.  

 

*OS is Windows 10 20H2

 

**After removing the updates (and doing nothing else) the VPN works again. 

Which KB updates did you uninstall? Having same issue now. Thank you.

I uninstalled KB5009543 and KB5008876. I'll see if I can narrow it down tomorrow. 

Thanks. I believe it's KB5009543 but waiting for a response from my users to confirm. Once I do, I will post here as well.

 

Running this to uninstall:

 

wusa /uninstall /kb:5009543

I've seen KB5009543 as well. Other platforms that use L2TP are being impacted too. 

worked for me as well. 
The Win11 Update should be KB5009566

 
 
Jawson
Conversationalist

Woke up here in UK with same slowly hitting all our users. Seems to be anything using PAP auth. Other L2TP VPNs we have using MSCHAP seem fine. Removing update fixed for now.

This is working for me. Confirmed on a few laptops this morning.


@RobertMiranda wrote:

 

wusa /uninstall /kb:5009543


This is a life saver, thanks! We had over 15 people and counting affected this morning.

Make sure you run your prompt as admin.

It is KB5009543. I just spent 4 hours cleaning up the mess from the installation of that update.

You are a lifesaver! Just got the Client VPN set up and it was working, then boom... nothing. Thanks!

The uninstall did not work for me. I received an error stating KB5009543 is required by your computer and cannot be uninstalled.  Help!

 

BigMtnIT
Conversationalist

Having the same issue. KB5009543 2022-01 Cumulative Update breaks L2TP on Win10 computers.  I have about 15 VPNs and they all give the same error now as Darcon's image above.  Recreating does NOT fix this.  Uninstalling KB5009543 does "fix" it but at some point we will all want this update and probably get it whether we want it or not.  Maybe Meraki can start lobbying Microsoft now since this will begin to propagate very soon and show up on more and more customers.  If anyone finds a workaround please post.  Thanks.

Thanks for confirming. Have 3 users now, but tomorrow morning it will go 10X. Fun times.

BlakeRichardson
Kind of a big deal

I dealt with a similar issue last year, I have no idea why MS seem to like breaking VPN connections. 

Meraki CMNO, Ruckus WISE, Sonicwall CSSA, Allied Telesis CASE & CAI
KumarKaliappan
New here

Same issue for my clients. VPN connection is not working after installing the 2022-01 CU.

Nikola
Conversationalist

We've been also affected. Both win10 and win11. 

Recreating connections doesn't help.

Is Cisco doing anything to resolve the issue? We cannot go forever pausing and removing updates.

GiacomoS
Meraki Employee

Good morning awesome people of the Community,

 

We have observed this as well throughout the day. As you have rightfully identified, this seems to be related to KB5009543. As this is a Microsoft update that is breaking the Windows VPN adapter, we are unlikely going to be able to affect it at this stage, but we are investigating internally nonetheless. 

 

The recommendation at the moment is to uninstall that update and let Microsoft know.

 

I've seen mention of other vendors being affected as well; @Darcon could you confirm if you have experienced this directly?

 

Many thanks!

Giac

Please keep in mind that what I post here is my personal knowledge and opinion. Don't take anything I say for the Holy Grail, but try and see!
Appreciate who helps and be respectful of every opinion and every solution offered.
Share the love, especially the Meraki one!

Hi all,

 

We have the same issue. I can confirm uninstalling KB5009543 fixes the issue.

I've just opened a support ticket to Microsoft, #‎29198981‎.

 

Hello,

 

I have a virtual windows 10 that i use for VPN connection to different clients, update broke the VPN and i uninstalled the update and its all good, however i noticed that VPN clent connections to MX80 is working fine only MX64 is broken!! could it be Meraki OS issue??

Avenir
Conversationalist

We have issue with MX100 and virtual MX100

@vhovan it sounds like you may be encountering a different problem here. I would recommend following the Troubleshooting Client VPN KB in the first place, and if you are still having issue afterwards, follow up with a Dashboard case with details of the error, troubleshooting done, packet captures if possible and timestamps of the attempts, so we can help further.

 

Many thanks!

Giac

Please keep in mind that what I post here is my personal knowledge and opinion. Don't take anything I say for the Holy Grail, but try and see!
Appreciate who helps and be respectful of every opinion and every solution offered.
Share the love, especially the Meraki one!

@GiacomoS I am internal IT and we only use Meraki gear, so no first-hand knowledge. I've just seen other affected platforms mentioned online. 

Thank you @Darcon , I appreciate your input!

Please keep in mind that what I post here is my personal knowledge and opinion. Don't take anything I say for the Holy Grail, but try and see!
Appreciate who helps and be respectful of every opinion and every solution offered.
Share the love, especially the Meraki one!
iReidy
Here to help

It worked for me, uninstalled

Windows 10
(KB5009543)
(KB5008876)

 

Windows 11
KB5009566

I have Windows 11 and I get this error 0x800f0905

JimmyM
Getting noticed

Same Problem with multiple customer. Uninstall KB5009543.

kkwok
Conversationalist

Did it fixed the issue, i just tried with a user no dice.

Users are not administrator on Device, and they are outside the network for verify with DC our administrator cred.

There is a way to terminate the Meraki VPN without the windows built-in client? OpenVPN works on it? 

Netmgr-24
Conversationalist

Exact same issue here on Windows 11.  Updated machine first thing this am, and have been trouble shooting since.  Led me here.

 

Event log:
The user SYSTEM dialed a connection named [XXXX] which has failed. The error code returned on failure is 789

 

Have tried the usuals - resetting all network connections, delete and recreate vpn client, etc...

 

January 11, 2022—KB5009566 (OS Build 22000.434) (microsoft.com)

 

Removed update with command:

 

DISM /remove-package /online /PackageName:Package_for_RollupFix~31bf3856ad364e35~amd64~~22000.434.1.4

After removing on Win 11, connected to Meraki (could see attempt in logs), but still failing with 720 error.  To fix this, had to uninstall WAN Miniport (L2TP) under Network Adapters in Device Manager.  Then scan for changes and it will reinstall.  Then connected fine.

 

Having some issues with routing to internet once connected on the Meraki side, but fixed by updating routes so non-vpn traffic uses local ISP, not VPN.

 

Working again!

dylan-mn
New here

We are having the same issue, I can confirm wusa /uninstall /kb:5009543 fixes it for now at least.

KennyH
Conversationalist

We had the same issue, thanks for the direction.  Also, anyone know of a status page for Meraki services or a updates page to subscribe to?

Hey @KennyH , 

The Service Notices board is a good place to start!

 

Hope this helps!

Giac

Please keep in mind that what I post here is my personal knowledge and opinion. Don't take anything I say for the Holy Grail, but try and see!
Appreciate who helps and be respectful of every opinion and every solution offered.
Share the love, especially the Meraki one!
jcgvt
Conversationalist

Any eta...this will turn in a big deal for compliance reasons quickly.  The answer CANNOT simply be rollback the MSFT patch or don't install it.

 

I think this is more of a MS problem than a Meraki problem. Given how widespread this issue is I would expect new patches to be released within a week or so. 

jcgvt
Conversationalist

Agreed.  Bigger than Meraki. 

It's only a Microsoft problem because cisco won't roll out a vpn client for meraki. The free windows vpn is really only for devices you get free from isps where its not worth their while to do anything better. To have systems with 1000s of users not having a vendor client is a joke tbh.

MX100 and upper isn't for manage tens people. The joke is that for 10k+ $ firewall we don't have Meraki client or a workaround to use third-party client. Use only a single built-in client is a point of failure that Cisco have to resolve if want to sell to bigger players. 

AnyConnect IS available on the Meraki platform, currently without a fee (because you need to be running 16.xx) but there will be a charge once the 16-series because stable. 

Specially with so many working from home due to Covid...

LM-TECH
New here

wusa /uninstall /kb:5009543

worked for me on Win10

Love from Microsoft...Love from Microsoft...

Parsoli
New here

What's interesting to us is, we have two different companies using Meraki MX's, both experiencing the same issue as everyone else above, but none of the impacted workstations/laptops have those patches installed.  Validated using the wusa /uninstall /kb:5009543 and the patches are not found, nor seen in Update history or via the Ivanti patch deployment tools we use.

The KB's may be different depending on which OS you are running. Try going into Windows Settings on an affected machine, then Windows Update, click on "View Update History". Write down whatever KB's were installed yesterday or today. 

mikeTNYC
Conversationalist

Same problem on a few systems today. Uninstalling KB5009543 fixed it. 

delfuego
Here to help

This is a mess. MS keeps screwing up. Starting to feel like the old days with Windows Updates.

Thanks for the help. Removing  KB5009543 working on a few machines already.

Marsrock2021
Getting noticed

Find it strange that people blame Microsoft. The built in vpn is no solution for business level vpn. The problem is that cisco bought meraki in 2012 and 10 years later they have no vpn client of their own. The whole cisco anyconnect debate going on for as long. The blame for this lies solely with the guys selling the hardware. Windows client is for $200 devices not 10 grand devices. The anyconnect is in beta for god knows how long on meraki and still no dates for GA. 

+1 on this,    in my case 100K on Meraki HW  and have to deal with this kind of BS, really sucks...

ShadowoftheDark
Here to help

gawddammit, i knew this was related to the windows update. one of our machines borked when i uninstalled the updates. fkn hell all of our vpn users are using windows 10 laptops

Mike6116
Here to help

Greetings from Mexico,  same issue here, some PC´s have the KB5009543  installed  , all having the same error,  we pulled back that update and everything works fine now....

JoshGesite
New here

Uninstalled KB5009543 on one machine and worked properly. Have another machine where I uninstalled KB5009543, which removed the Error 789, but now getting Error809. Anyone else running into this issue?

Try a reboot for the 809 error. I tested to see if the rollback worked without restart (even after prompted) on a machine and got the 809 event.

Tried a reboot. Also did the steps on this page: https://documentation.meraki.com/MX/Client_VPN/Guided_Client_VPN_Troubleshooting -- such as the regedit. Also disabled the Xbox Live Network Service and other steps here: https://www.thewindowsclub.com/troubleshoot-vpn-error-809-on-windows 

but still have not resolved the issue.

Pharkurnell
Here to help

like millions of others have been struck with this one on many laptops. Great issue to have with 80 laptops around Australia...

 

My issue is uninstalling the update.

Logged in as local admin  I right click on KB5009543 and there is no uninstall option ???

 

I also tried wusa /uninstall /kb:5009543 but it tells me its a required update you cannot uninstall it...

 

Thoughts?

must be a policy m8, wusa /uninstall worked for me. i'm on my 5th today lol

Lourdes
Here to help

We are also having an issue with the VPN connection but some users did not update their windows but were suddenly unable to connect to VPN. We are using windows 10 and there is no KB5009543 at KB5008876. 

 

What to do?

is the authentication set to username / password?

in the security tab of the vpn adater is it set to required - only thing checked is PAP?

Yes, the authentication is set to username/password.

VPN type is L2TP/IPsec with a pre-shared key. 

The PAP and CHAP ver 2 is actually checked.

i think its only PAP, W10-CVPN-6.pngchap is unchecked  

 

Client VPN OS Configuration - Cisco Meraki

 

 

for some reason it both connects and if you try to change it and put it as meraki says and check only pap it deletes instanly the user name and pass so you cant  do anything ... tyried to unistall the updates nothing still the same mc is the best ... 

MateTamas
New here

I had the same problem, uninstalling that update + restart solved it.

Where can I raise a ticket to MS? The more opened ticket the sooner solution... 🙂

Thank you!

Hi,

I guess if you don't have access/account to admin.microsoft.com site, then this is the way:

https://support.microsoft.com/en-us/windows/send-feedback-to-microsoft-with-the-feedback-hub-app-f59...

delfuego
Here to help

Update downloaded and installed itself again on one machine today (after removal yesterday).

follow this article to block (hide) the update:  https://www.maketecheasier.com/hide-updates-in-windows-10/

I believe this is the case here too... but running that powershell solution below (Id never seen before but bookmarked it!) to block update on 60-80 remote computers isn't an option for me.

Nikola
Conversationalist

I think this is a serious issue for many users and Cisco should handle this with Microsoft, not ask users to submit tickets. 

JimmyM
Getting noticed

Does anyone know if Meraki work with MS with that or if someone has opened a case with Microsoft.

 

I can open an MS ticket for that but I don't want to do a thing than someone is already doing.

 

Regards,

delfuego
Here to help

Thanks again for the help. If anyone has a permanent solution or what exactly the update changes, please let us know. This has been happening all over for us.

JimmyM
Getting noticed

I opened a support case with Microsoft, i will give you a follow up.

360IT
Conversationalist

Thanks to everyone for your combined efforts in troubleshooting this latest vpn client issue, what a fiasco.

I can confirm that removing KB5009566 on Win11 resolved the issue for me. I lost 13vpn configs on my machine yesterday, a pain to say the lease. Now off to do a regression patch on a mind numbing number of client systems, Gack! 

I have been dealing with random windows client vpn connection issues on MX devices for over a year now. Have tried all the methods to resolve, not one consistently is the fix however. One method I have tried which is partially documented is the removal of all wan miniports, but before scanning for hardware changes to re-install I would configure the required vpn's and when finished I would then re-install the miniports. 

Have no idea why, but that process has worked on quite few win 10 machines over the past several months.

 

MarkChan
Here to help

Microsoft acknowledged the issue. I am not sure if the workaround exist for Meraki MX.

 

The description is as below:

 

After installing KB5009543, IP Security (IPSEC) connections which contain a Vendor ID might fail. VPN connections using Layer 2 Tunneling Protocol (L2TP) or IP security Internet Key Exchange (IPSEC IKE) might also be affected.
 
Workaround: To mitigate the issue for some VPNs, you can disable Vendor ID within the server-side settings. Note: Not all VPN servers have the option to disable Vendor ID from being used.
 
Next steps: We are presently investigating and will provide an update in an upcoming release.
 
Affected platforms:
  • Client: Windows 11, version 21H2; Windows 10, version 21H2; Windows 10, version 21H1; Windows 10, version 20H2; Windows 10, version 1909; Windows 10, version 1809; Windows 10 Enterprise LTSC 2019; Windows 10 Enterprise LTSC 2016; Windows 10, version 1607; Windows 10 Enterprise 2015 LTSB
  • Server: Windows Server 2022; Windows Server, version 20H2; Windows Server 2019; Windows Server 2016

 

Windows 10, version 21H2 | Microsoft Docs