Cisco Umbrella and AnyConnect on MX-105 - DNS Issues

[AnthonyI]

Hi all,

We have a setup where our employees do user enrollment to register their devices with Meraki Systems Manager (SM). Once they enroll, they get profiles and packages pushed to them, including Cisco AnyConnect, Cisco SecureX (AMP), and Cisco Umbrella. Our testing prior to deployment of everything worked fine, so we started sending laptops to new employees and having them self-enroll in order to receive the required packages.

The users access our internal network using the AnyConnect Client VPN via an MX-105 HA pair in a split tunnel configuration. Once connected to the VPN, we assign custom internal nameservers to be able resolve our local hostnames. 

In one of our very first deployments on an out-of-the-box 14 inch MacBook Pro running MacOS Monterey 12.1, the user successfully enrolled into SM and received the three packages listed above. Once they connected to the VPN, they couldn't resolve any internal hostnames. Checking the DNS configuration in the network settings, it appeared that the nameservers weren't updating to our custom ones -- rather, it only had the localhost (127.0.0.1) showing in the list (matching what's in /etc/resolv.conf). 

After a few hours of troubleshooting, we had the user uninstall Cisco Umbrella and then everything started working. The custom nameservers were now showing up under the DNS settings (and resolv.conf) and user could resolve internal hostnames.

Some additional notes:

-  Our Umbrella system is not integrated into our Meraki system. We're using default Umbrella policies for DNS layer security.

- In Umbrella Dashboard -> Deployments -> Configuration -> Internal Networks, we have the two /24's (where our hostnames reside in) added.

- We're running Umbrella RC 3.0.5.

- This exact setup works on other similarly-spec'd MacBooks and on other Windows 10 machines. I'm unsure if the sequence of when each package is installed has something to do with this. 

I'm hesitant to integrate Umbrella into Meraki as I'm not sure if that could impact DNS resolution across the board over the VPN.

I also saw that someone else posted about this but didn't see a resolution: Umbrella Roaming Client and Cisco Meraki MX Firewalls - The Meraki Community. Any thoughts or help on this would be appreciated!

[OCTOMG]

Are you running the Umbrella Virtual Appliance on your LAN?  Also, have you set up separate Umbrella policies in the Umbrella dashboard for On-Net and Off-Net traffic?  Not sure that's the root cause of your issue, but it may be worth a look.

FWIW, we have Umbrella integrated with our Meraki stack and really like it.  Much better solution than what comes baked in with the Advanced Security license (which we also use).

[AnthonyI]

We are not running Umbrella Virtual Appliance on our LAN, or have any policies set up in the dashboard for the different types of traffic. I'll look into those to see if they'll change anything.

My understanding of integrating Umbrella into Meraki is that it would apply DNS layer security for devices connected to the MX (e.g., Client VPN) or behind it on the local LAN (e.g., Access point). Though this would cover a lot of our user traffic, we'd still like to have DNS layer security for devices that are remote and not connected to the VPN -- hence the need for the roaming client.