Anyconnect VPN with Filter-ID via radius working partially

[GIdenJoe]

I have already opened a case for this issue but wanted to share a bug I seem to be having.

A customer of our company has MX95 HA pair at a site.  These are running MX18.211.01 (can't update much but the bug is not mentioned in newer versions).

We are in the process of setting up Radius so we can apply 802.1X group policies to VPN users so external partners can login and only access their servers.

We tried logging in with users from different AD groups and they seem to be getting the correct VPN filter (group policy).
However for some strange reason only ICMP traffic is blocked when it matches a deny rule.  TCP and UDP get through just fine which is NOT the intention.

The group-policies are simply configured as followed:
Allow any port to the specific server IP they are allowed to reach.
Allow UDP/53 to the internal DNS server
Deny Any to all three RFC1918 address blocks.

You can see in the eventlog that the VPN filter is being applied.
If you wait for a few hours you can also see the client finally popping up in the clients page with the 802.1X group policy correctly applied.

If I change the rules while connected or after reconnection the ICMP traffic is correctly allowed or blocked, however I can even run a full port scan towards servers I'm not supposed to reach with that user.

I have already referenced this post: https://community.meraki.com/t5/Security-SD-WAN/AnyConnect-MX250-Radius-authentication-and-filter-id... and I'm hoping I won't have to split this network up since that defeats a large part of the usability of the dashboard.

[thomasthomsen]

I am quite sure I have seen this problem recently.

Where everything actually worked fine with Group-Policy using Anyconnect, and then after a MX upgrade it failed (was not applied to the client).

Actually, it was applied to some clients, for unknown reasons, and not to others (hitting the same rule and results on the radius server).

I cant remember what happened, I will "check my notes" and get back to you, but Im quite sure I did not have to split the network.

[thomasthomsen]

From notes : Happened when customer updated from : 18.107.2 to 18.211.2

Customer opened a case, but since they downgraded again to 18.107.2 the problem is not there (only once in a while).

The only way for support to troubleshoot the problem was for the customer to upgrade again, and do "realtime" troubleshooting with support on the phone.

But from what I can see, they have not gotten around to that yet, and I think vacation set in, and the case was closed.

I guess they will re-open the case, and do some realtime troubleshooting after the vacation is over, and a service window can be found.

Sorry for not having any better news for you.

[GIdenJoe]

Lol, well I already had to to a downgrade and then upgrade again when I ran into that QoS bug where traffic got dropped through the AutoVPN tunnel due to QoS rules.

It's not a customer where I want to do many upgrades in a short period.

[thomasthomsen]

Uh ... I have never experienced that bug ... do tell 🙂

[GIdenJoe]

In MX18.211 there was a specific bug that when traffic was matching a QoS rule that had high or low as bandwidth it would simply drop inside the tunnel.  You could see the traffic leaving 1 MX inside the tunnel and not get to the other MX.  That was the whole reason they made MX18.211.01.  I fell into that bug with the same customer and spent half a day chasing this bug.  Had to retrace half of my work and put back the old firewalls from another vendor before trying again the week after when the fix release was there.  Didn't know about the QoS rules back then.

[thomasthomsen]

Interesting. Thanks for sharing.

[GIdenJoe]

I have an update on the case:

Apparently after a few days it starts to work.
The group policy is fully usable after a few days.

However the moment you connect with a user that is in a different group which gets a different group policy it is back tot he ICMP only and multicast too apparently.

I now started logging the MX when I'm testing and found the following:

ICMP's get firewall logging:

blocked
l7_firewall src=10.16.123.179 dst=172.20.2.221 protocol=icmp type=8 decision=blocked

allowed

ip_flow_start src=10.16.123.179 dst=1XX.1.2.40 protocol=icmp translated_dst_ip=1XX.1.2.40
firewall src=10.16.123.179 dst=1XX.1.2.40 mac=03:45:6C:XX:XX:XX protocol=icmp type=8 pattern: Group Policy Allow

But there is no firewall logging for TCP or UDP traffic.  It is however funny and shows actual flow_start logs for traffic that should be blocked 😜

Below should be blocked
ip_flow_start src=10.16.123.179 dst=172.20.2.XXX protocol=tcp sport=62559 dport=3389 translated_dst_ip=172.20.2.XXX translated_port=3389

So I am assuming it will start working in a few days...
Also the VPN clients don't appear in dashboard for a few hours.

[GIdenJoe]

Another update on the case.

Meraki helped me get back on the MX18.107 software train.  And from the first tests it seems to work immediately.

So I believe the MX18.2x train has this issue and it should be included in the issues list.