Cisco Anyconnect - SAML using OneLogin for MFA

[JRMcGee]

We are using CISCO Anyconnect VPN client software to connect to our Meraki MX. Authentication Type is SAML using our idP provider (OneLogin) for MFA.

Sometimes, after a user enters their credentials in CISCO Anyconnect, it goes to a white screen box after mfa authentication. The box will stay there about a minute and will error out. The error is "CSRF token verification failed"

This happens randomly to different users at different times, making it hard to replicate the issue. The rest of the times, we are able to connect without an issue. 

Has anyone see this issue before? 

[Ryan_Miles]

I've had one report of this, but the customer has since rolled back to 16.x so we can't troubleshoot any further. It would be great if you can open a Support case on this so they can gather some data to send up to Eng if needed.

[JRMcGee]

Hi Ryan,

We do have a case open with Meraki. We are still troubleshooting the issue. Just wanted to know if anyone else has seen this issue. Thank you. 

[Ryan_Miles]

👍

[coelmann]

This happens to us all the time. We had a case open with Meraki at one point, but I just gave up. The work around we have found is to re-upload the SAML Metadata File.

[JRMcGee]

Hi @coelmann,

Meraki support got back to me and instructed me to downgrade my MX to firmware 16.16.6 

My support agent told me that he has found related cases and this was the fix.

I am going to test it this weekend, hopefully all goes well.

[JRMcGee · Accepted solution]

@coelmann 

We downgraded our firmware to 16.16.6 on our MX. Since we have done that, we have not had any users experiencing the blank box after MFA on Anyconnect. 

[coelmann]

Thanks @JRMcGee! We'll roll our firmware back and let you know how it goes

[JRMcGee]

@coelmann 

Make sure it is 16.16.6, not 16.16

I had to get Meraki support to do it for me, as it was not an option for me. 

thank you

[ontheroadoflife]

Hey JRM, 

What firmware version did you downgrade from? Did you also have issues with users being prompted for username and password, but no MFA, it would just connect automatically. 

[JRMcGee]

Hi @ontheroadoflife 

It was 17.x - i cannot remember the exact version number. No, i did not have issues with users being prompted for UN and password but no MFA. What is your Authentication Type set to?

[ontheroadoflife]

Since we upgraded one of our MXs to 17.10.2, we've been having multiple VPN issues such as the CSRF error, users being prompted for credentials with no MFA at random times. 

We're using SAML against Azure.

[PaulStanley]

We upgraded to 17.10.2 two weeks ago and we have seen issue with AnyConnect and SAML. I've seen the white box and it hangs up for a minute and the user is able to login and also if a user selects to remain logged in and then it loops back to the email login again. We didn't have this issue on 16

[Dudleydogg]

I also have this blank box issue, but its rare the Bigger issue is Previously we could sign into the VPN disconnect and than reconnect and it would just connect again. NOW it not only prompts us each time the username field is blank. so y ou has to type in user and then select the MFA option. We noticed this a few weeks ago. I would prefer the browser cache and use that token but this feature seems to be lost, Any one else notice this behavior?

[WarrenG]

You will need to contact Meraki support and ask them to disable the "forceauthn" option (set it to false) for SAML authentication.

[Dudleydogg]

That was the Fix just a setting we can't see in dashboard

[Pathfinder89]

Any updates if a 17.x or 18.x firmware upgrade will address this CSRF issue? 

[WarrenG]

So we have a few clients with MX64 firewalls. Anyconnect functionality was not added until firmware version 17.10.2. So downgrading to 16.16.6 is not a solution for these firewalls. We also tried the beta 18.104 but the behavior is still the same (blank white windows as well as CSRF token validation errors). Seems like this feature is far from fully baked.

[BobL]

I upgraded AnyConnect from 4.10.00093 to 4.10.06079 and that solved the problem for me.

[Winter]

We found that even with the new AnyConnect client that we'd still have users get stuck after the saml login screens, typically the blank white page. To fix this create an OUTBOUND rule in Windows Defender Firewall for where ever you have \Cisco AnyConnect Secure Mobility Client\acwebhelper.exe and set it to ALLOW.

[WarrenG]

I tried this but still get the blank white screen for one of the users

[Winter]

WE had to add the vpnui.exe as well. However, we find that even adding both of these, sometimes it fails again after initially being successful and the rules have to be removed and re-added. Clearly something else going on here.

[mbadmin30]

A couple of workarounds I've used for this. Shift + F5 seems to reload the white screen and allow it to connect. Clearing this folder: "%UserProfile%\AppData\Local\Cisco\Cisco Secure Client\EBWebView\Default" clears the browser cache and allows it to connect as well.

[Winter]

We've found this as well, ctrl-F5 or F5 to refresh can sometimes get you through the problem. Thanks for the info on the user profile data. We tried having some people upgrade to the version 5 of the client per recommendation of the Meraki support and it didn't really help.

[JRMcGee]

We upgraded our MX to firmware 17.10.2 and we now use CISCO Secure Connect (Anyconnect) 5.0.01242 and have need seen this issue again.

[mbadmin30]

Darn! We're on 17.10.4 and 5.0.01242 and we've seen it quite a bit recently.

[mbadmin30]

Yeah, the user profile *fix* could work if we created a task to clear that folder periodically. Would just be a pain...

[WarrenG]

Thanks for this! I was banging my head trying to figure out which browser Anyconnect was using for that pop up window, thinking it was using one of the installed browsers. Great to know it's not, and which folder to clear to empty the cache. Will give this a spin here today to see if it works 👍

[PineTree]

Hello Gents,

Wanted to let you know we are having this exact issue with Azure SAML and have created a new case with wireshark captures, Anyconnect logs, and shipped it off to Meraki for association with the other bug reports. Happened on 17.10.2 firmware and on 18.106. Meraki is downgrading us to 16.16.9 tonight to test out whether that resolves this issue.

[WarrenG]

I haven't been able to test/prove that this is the issue yet, but I have noticed that any clients that do have connection issues are all configured to use a non-standard SSL port number for Anyconnect. For example, they might be using 4443 instead of 443. Not sure if that non-standard SSL port number might be something that the ISP of the remote user is interrogating a little more aggressively because it's non-standard?

[PineTree]

In our case we are using the standard 443 setting and we are seeing it regularly across every user trying to use anyconnect for a particular site. When we switch to AD authentication for testing instead of SAML we have a 100% success rate.

Based on the fact that clearing the AppData\Local\Cisco\Cisco Secure Client\EBWebView\Default fixes it every time, but only for one successful connection, I would have to assume its some sort of bug in whatever settings in that folder get created upon successful startup of the VPN. Also waiting 45 seconds on the blank screen and hitting F5 to refresh is also fixing the issue with a near 100% success rate. 

Maybe its a preferences/settings issue somewhere preventing it from properly resolving a hostname.

[PineTree]

16.16.9 worked

[Indubitably]

@PineTree Did they have any other long term fixes for this? We're seeing this for some users and we use Azure SAML for AnyConnect. I'm guessing we need to open a TAC case and have ourselves downgraded but that is super annoying so I figured I'd check in with you. Thanks!

[PineTree]

Supposedly the latest version of Secure fixes the issue but Im not quite at the point of testing that.

[dbsec]

I'm seeing this same problem with 17.x and 18.x, regardless of the VPN client version. Downgrading to 16.16.9 fixes the problem, but clearly this is a temporary solution as we can't remain on old firmware forever. 😅

I also setup a separate MX to try to better understand and troubleshoot this problem. It's configured with near-identical settings, as is the Azure registration and configuration. Frustratingly, I can't duplicate the problem on this test MX and all my tests against 17.x and 18.x firmware don't have any issues. I'm wondering if maybe there is something stuck in an 'old' VPN profile or configuration.

[PineTree]

Hello,

I can confirm i also have another MX that historically has not had VPN turned on and it doesnt have the same anyconnect issues and is on latest firmware. I'm pretty annoyed Meraki cant figure this one out (atleast publicly) given the large amount of info I know they have on it as well as detailed logs and captures.

[PineTree]

Looks like this was finally fixed, my trouble ticket was closed last week. I havent tested but Secure Client 5.0.05040 was listed as resolving it.

CSCwe45817 - Unencoded embedded URL in HTTP redirect prevents captive portal detection

[dbsec]

Thanks for the update! I'd be really interested in hearing about how well the fix works for you and whether you're able to move off of 16.16.9. 

[IT44208]

Hey PineTree, have you since tested 5.0.05040?

[PineTree]

I havent tested it yet because Meraki never updated their repository Version # and It took me forever to get cisco to rectify my software entitlements, but I am going to be rolling out the anyconnect update over the next week then updating meraki after that, I should know full well by December sometime.

[IT44208]

I deployed AnyConnect 5.0.05040 to everyone in my org and updated our MX to 18.107.2. I can confirm the "CSRF token verification failed" issue is no longer present in our environment. Looks like they finally fixed it!