Client VPN Error After January Windows Updates

[Darcon]

After patching a couple machines with the January Windows Updates I found that our Meraki Client VPN no longer works. I get error 789 in Windows Event Viewer, and nothing at all in the Meraki Event Log.

This is the error displayed when attempting to connect. It errors out within a second of hitting connect, doesn't even prompt for creds.

We've had AssumeUDPEncapsulationContextOnSendRule set to 2 for some years. I tried removing / rebooting / re-adding this key; same error. I also tried switching our PSK to a key without special characters, same error.

I have confirmed this on 2 different machines. Unpatched machines continue to connect without issue.

Anyone else run into this today? 

[BrandonS]

Not today (yet), but pretty much every Windows update brings me service tickets about broken client VPN and 100% of the time they are resolved by deleting and creating the VPN again.  There is probably a quicker way and an underlying reason, but I have not cared to dig further personally..  

EDIT: Apparently this time requires removing a Windows update.  Keep scrolling..

[Darcon]

I gave re-creating the connections a try, no luck. I also tried running through my mobile hotspot to no avail.  

*OS is Windows 10 20H2

**After removing the updates (and doing nothing else) the VPN works again. 

[RobertMiranda]

Which KB updates did you uninstall? Having same issue now. Thank you.

[Darcon]

I uninstalled KB5009543 and KB5008876. I'll see if I can narrow it down tomorrow. 

[RobertMiranda]

Thanks. I believe it's KB5009543 but waiting for a response from my users to confirm. Once I do, I will post here as well.

Running this to uninstall:

wusa /uninstall /kb:5009543

[Darcon]

I've seen KB5009543 as well. Other platforms that use L2TP are being impacted too. 

[Ben0391]

worked for me as well. 
The Win11 Update should be KB5009566

[Jawson]

Woke up here in UK with same slowly hitting all our users. Seems to be anything using PAP auth. Other L2TP VPNs we have using MSCHAP seem fine. Removing update fixed for now.

[RobertMiranda]

This is working for me. Confirmed on a few laptops this morning.

[RobNYC7]

---

@RobertMiranda wrote:

wusa /uninstall /kb:5009543

---

This is a life saver, thanks! We had over 15 people and counting affected this morning.

Make sure you run your prompt as admin.

[BeckerIT]

It is KB5009543. I just spent 4 hours cleaning up the mess from the installation of that update.

[BreckpointIT]

You are a lifesaver! Just got the Client VPN set up and it was working, then boom... nothing. Thanks!

[April_Fulton]

The uninstall did not work for me. I received an error stating KB5009543 is required by your computer and cannot be uninstalled.  Help!

[BigMtnIT]

Having the same issue. KB5009543 2022-01 Cumulative Update breaks L2TP on Win10 computers.  I have about 15 VPNs and they all give the same error now as Darcon's image above.  Recreating does NOT fix this.  Uninstalling KB5009543 does "fix" it but at some point we will all want this update and probably get it whether we want it or not.  Maybe Meraki can start lobbying Microsoft now since this will begin to propagate very soon and show up on more and more customers.  If anyone finds a workaround please post.  Thanks.

[RobertMiranda]

Thanks for confirming. Have 3 users now, but tomorrow morning it will go 10X. Fun times.

[BlakeRichardson]

I dealt with a similar issue last year, I have no idea why MS seem to like breaking VPN connections. 

[KumarKaliappan]

Same issue for my clients. VPN connection is not working after installing the 2022-01 CU.

[Nikola]

We've been also affected. Both win10 and win11. 

Recreating connections doesn't help.

Is Cisco doing anything to resolve the issue? We cannot go forever pausing and removing updates.

[GiacomoS]

Good morning awesome people of the Community,

We have observed this as well throughout the day. As you have rightfully identified, this seems to be related to KB5009543. As this is a Microsoft update that is breaking the Windows VPN adapter, we are unlikely going to be able to affect it at this stage, but we are investigating internally nonetheless. 

The recommendation at the moment is to uninstall that update and let Microsoft know.

I've seen mention of other vendors being affected as well; @Darcon could you confirm if you have experienced this directly?

Many thanks!

Giac

[Tiainen]

Hi all,

We have the same issue. I can confirm uninstalling KB5009543 fixes the issue.

I've just opened a support ticket to Microsoft, #‎29198981‎.

[vhovan]

Hello,

I have a virtual windows 10 that i use for VPN connection to different clients, update broke the VPN and i uninstalled the update and its all good, however i noticed that VPN clent connections to MX80 is working fine only MX64 is broken!! could it be Meraki OS issue??

[Avenir]

We have issue with MX100 and virtual MX100

[GiacomoS]

@vhovan it sounds like you may be encountering a different problem here. I would recommend following the Troubleshooting Client VPN KB in the first place, and if you are still having issue afterwards, follow up with a Dashboard case with details of the error, troubleshooting done, packet captures if possible and timestamps of the attempts, so we can help further.

Many thanks!

Giac

[Darcon]

@GiacomoS I am internal IT and we only use Meraki gear, so no first-hand knowledge. I've just seen other affected platforms mentioned online. 

[GiacomoS]

Thank you @Darcon , I appreciate your input!

[iReidy]

It worked for me, uninstalled

Windows 10
(KB5009543)
(KB5008876)

Windows 11
KB5009566

[MatiRufa]

I have Windows 11 and I get this error 0x800f0905

[JimmyM]

Same Problem with multiple customer. Uninstall KB5009543.

[kkwok]

Did it fixed the issue, i just tried with a user no dice.

[AntonioGuerrisi]

Users are not administrator on Device, and they are outside the network for verify with DC our administrator cred.

There is a way to terminate the Meraki VPN without the windows built-in client? OpenVPN works on it? 

[Netmgr-24]

Exact same issue here on Windows 11.  Updated machine first thing this am, and have been trouble shooting since.  Led me here.

Event log:
The user SYSTEM dialed a connection named [XXXX] which has failed. The error code returned on failure is 789

Have tried the usuals - resetting all network connections, delete and recreate vpn client, etc...

January 11, 2022—KB5009566 (OS Build 22000.434) (microsoft.com)

Removed update with command:

DISM /remove-package /online /PackageName:Package_for_RollupFix~31bf3856ad364e35~amd64~~22000.434.1.4

[Netmgr-24]

After removing on Win 11, connected to Meraki (could see attempt in logs), but still failing with 720 error.  To fix this, had to uninstall WAN Miniport (L2TP) under Network Adapters in Device Manager.  Then scan for changes and it will reinstall.  Then connected fine.

Having some issues with routing to internet once connected on the Meraki side, but fixed by updating routes so non-vpn traffic uses local ISP, not VPN.

Working again!

[dylan-mn]

We are having the same issue, I can confirm wusa /uninstall /kb:5009543 fixes it for now at least.

[KennyH]

We had the same issue, thanks for the direction.  Also, anyone know of a status page for Meraki services or a updates page to subscribe to?

[GiacomoS]

Hey @KennyH , 

The Service Notices board is a good place to start!

Hope this helps!

Giac

[jcgvt]

Any eta...this will turn in a big deal for compliance reasons quickly.  The answer CANNOT simply be rollback the MSFT patch or don't install it.

[Darcon]

I think this is more of a MS problem than a Meraki problem. Given how widespread this issue is I would expect new patches to be released within a week or so. 

[jcgvt]

Agreed.  Bigger than Meraki. 

[Marsrock2021]

It's only a Microsoft problem because cisco won't roll out a vpn client for meraki. The free windows vpn is really only for devices you get free from isps where its not worth their while to do anything better. To have systems with 1000s of users not having a vendor client is a joke tbh.

[AntonioGuerrisi]

MX100 and upper isn't for manage tens people. The joke is that for 10k+ $ firewall we don't have Meraki client or a workaround to use third-party client. Use only a single built-in client is a point of failure that Cisco have to resolve if want to sell to bigger players. 

[OVERKILL]

AnyConnect IS available on the Meraki platform, currently without a fee (because you need to be running 16.xx) but there will be a charge once the 16-series because stable. 

[Pharkurnell]

Specially with so many working from home due to Covid...

[LM-TECH]

wusa /uninstall /kb:5009543

worked for me on Win10

[Pharkurnell]

Love from Microsoft...

[Parsoli]

What's interesting to us is, we have two different companies using Meraki MX's, both experiencing the same issue as everyone else above, but none of the impacted workstations/laptops have those patches installed.  Validated using the wusa /uninstall /kb:5009543 and the patches are not found, nor seen in Update history or via the Ivanti patch deployment tools we use.

[Darcon]

The KB's may be different depending on which OS you are running. Try going into Windows Settings on an affected machine, then Windows Update, click on "View Update History". Write down whatever KB's were installed yesterday or today. 

[mikeTNYC]

Same problem on a few systems today. Uninstalling KB5009543 fixed it. 

[delfuego]

This is a mess. MS keeps screwing up. Starting to feel like the old days with Windows Updates.

Thanks for the help. Removing  KB5009543 working on a few machines already.

[Marsrock2021]

Find it strange that people blame Microsoft. The built in vpn is no solution for business level vpn. The problem is that cisco bought meraki in 2012 and 10 years later they have no vpn client of their own. The whole cisco anyconnect debate going on for as long. The blame for this lies solely with the guys selling the hardware. Windows client is for $200 devices not 10 grand devices. The anyconnect is in beta for god knows how long on meraki and still no dates for GA. 

[Mike6116]

+1 on this,    in my case 100K on Meraki HW  and have to deal with this kind of BS, really sucks...

[ShadowoftheDark]

gawddammit, i knew this was related to the windows update. one of our machines borked when i uninstalled the updates. fkn hell all of our vpn users are using windows 10 laptops

[Mike6116]

Greetings from Mexico,  same issue here, some PC´s have the KB5009543  installed  , all having the same error,  we pulled back that update and everything works fine now....

[JoshGesite]

Uninstalled KB5009543 on one machine and worked properly. Have another machine where I uninstalled KB5009543, which removed the Error 789, but now getting Error809. Anyone else running into this issue?

[delfuego]

Try a reboot for the 809 error. I tested to see if the rollback worked without restart (even after prompted) on a machine and got the 809 event.

[JoshGesite]

Tried a reboot. Also did the steps on this page: https://documentation.meraki.com/MX/Client_VPN/Guided_Client_VPN_Troubleshooting -- such as the regedit. Also disabled the Xbox Live Network Service and other steps here: https://www.thewindowsclub.com/troubleshoot-vpn-error-809-on-windows 

but still have not resolved the issue.

[Pharkurnell]

like millions of others have been struck with this one on many laptops. Great issue to have with 80 laptops around Australia...

My issue is uninstalling the update.

Logged in as local admin  I right click on KB5009543 and there is no uninstall option ???

I also tried wusa /uninstall /kb:5009543 but it tells me its a required update you cannot uninstall it...

Thoughts?

[ShadowoftheDark]

must be a policy m8, wusa /uninstall worked for me. i'm on my 5th today lol

[Lourdes]

We are also having an issue with the VPN connection but some users did not update their windows but were suddenly unable to connect to VPN. We are using windows 10 and there is no KB5009543 at KB5008876. 

What to do?

[ShadowoftheDark]

is the authentication set to username / password?

in the security tab of the vpn adater is it set to required - only thing checked is PAP?

[Lourdes]

Yes, the authentication is set to username/password.

VPN type is L2TP/IPsec with a pre-shared key. 

The PAP and CHAP ver 2 is actually checked.

[ShadowoftheDark]

i think its only PAP, chap is unchecked  

Client VPN OS Configuration - Cisco Meraki

[Itchriskentas]

for some reason it both connects and if you try to change it and put it as meraki says and check only pap it deletes instanly the user name and pass so you cant  do anything ... tyried to unistall the updates nothing still the same mc is the best ... 

[MateTamas]

I had the same problem, uninstalling that update + restart solved it.

Where can I raise a ticket to MS? The more opened ticket the sooner solution... 🙂

Thank you!

[Tiainen]

Hi,

I guess if you don't have access/account to admin.microsoft.com site, then this is the way:

https://support.microsoft.com/en-us/windows/send-feedback-to-microsoft-with-the-feedback-hub-app-f59...

[delfuego]

Update downloaded and installed itself again on one machine today (after removal yesterday).

[crabwebs]

follow this article to block (hide) the update:  https://www.maketecheasier.com/hide-updates-in-windows-10/

[Pharkurnell]

I believe this is the case here too... but running that powershell solution below (Id never seen before but bookmarked it!) to block update on 60-80 remote computers isn't an option for me.

[Nikola]

I think this is a serious issue for many users and Cisco should handle this with Microsoft, not ask users to submit tickets. 

[JimmyM]

Does anyone know if Meraki work with MS with that or if someone has opened a case with Microsoft.

I can open an MS ticket for that but I don't want to do a thing than someone is already doing.

Regards,

[delfuego]

Thanks again for the help. If anyone has a permanent solution or what exactly the update changes, please let us know. This has been happening all over for us.

[JimmyM]

I opened a support case with Microsoft, i will give you a follow up.