Auto-VPN, UDP pinhole in firewall and LTE Carrier NAT

[AndyGray]

Hi folks

We have a couple of MX250s behind Juniper firewalls acting as Auto-VPN hubs for 150+ sites. The Juniper has static NAT between an internet legal outside address and the MX250 DMZ address in each case, and the firewall policy allows all the appropriate ports and destinations for management and monitoring outbound, and the UDP range 32768-61000 to anywhere for Auto-VPNs. This 'pinhole' approach has worked perfectly for all our sites on various types of wired internet circuits, xDSL and Ethernet. There are no inbound permissions in the firewall towards the MX250s at all.

But now I'm evaluating an MX68CW, the intent being to provide the same access to internet resources over Auto-VPN when a site has either failed over to LTE for internet access or potentially as the only internet access where a wired circuit isn't available.

We've managed to confirm that there's some form of Carrier NAT in play on the LTE networks, slightly different if I swap between different SIMs, but always with the same effect - Auto-VPN into the DC MX250s fails.

We can see the MX250s attempting to create outbound UDP connections to the MX68CW LTE interface on ports below 32768.  Having widened the outbound UDP port list to effectively UDP port 'any', it still doesn't work.  It looks like we have to allow some inbound traffic through the Juniper for the Carrier NAT interference to be untangled and for Auto VPN to establish.

I know it works fine if we have an anything-in/anything-out policy in the Juniper, effectively making it just a NAT passthrough, but I'd like to define only the specific policy required for Auto-VPN over LTE to work, if possible.

Anyone seen this before, and can advise how they tackled it?

Thanks in advance...

Andy

[AndyGray]

Edit - in Paragraph 2, that should say ...provide the same access to internal resources over Auto-VPN.

[NolanHerring]

Is your MX in Routed mode or Passthrough/VPN Concetrator?

I have a similar setup but I'm using the VPN Concetrator mode and I've not had any issues with the MX67C cellular tunnels being established.

I have on the site-to-site settings a static public IP & port setup to our DC firewall

[AndyGray]

Hi Nolan, and thanks for the reply.

The DC MXs are in Routed mode, though they're not providing any local L3 routing, they just have an inside leg into our internal facing DMZ, and an internet connection in the external DMZ, static NATted at the Juniper. The MX itself has a 10.0.0.0/8 address on both interfaces.

I forget exactly why I chose Routed mode when they really are VPN Concentrators, but literal one-armed mode wouldn't have worked for us, Routed mode does - they work just great for all our wired sites.

So I imagine Carrier NAT is messing with your port negotiations for VPNs over LTE, so there're probably more packets exchanged before the tunnels come up, but in real terms you wouldn't really notice. VPN over LTE works great into our Azure vMX public IP, and I've tested with direct internet connected MX100s and they're fine.

Since we're putting MXs on direct public IPs all over the place, I think I could persuade our CAB to allow us to lift the Juniper policy, but was hoping I could find a more subtle solution 🙂

[PhilipDAth]

AutoVPN when running over a wired connected is exactly the same as when running over cellular.  So the issue is not on the MX250 end.

To get another potential issue out of the way - the MX68CW must be brought online once via a wired connection to get a firmware update before it can run over cellular.  This is an issue if it is only ever run over cellular.  Your use case sounds like it is for failover - but I mentioned this just in case I misunderstood.

The issue will be to do with the cellular carrier and the APN being used.  I have run into cellular providers that aggressively filter inbound UDP traffic preventing IPSec VPNs from working.

Some providers use pure IPv6 now and use NAT gateways to support IPv4.  You are highly unlikely to get AutoVPN to stand up on a carrier that does this.

Ask your cellular provider if they have a different APN you can use with less firewalling and filtering.

Failing that try getting a SIM from another provider to test.

[AndyGray]

Hi Philip, and thanks for the reply.

We might have cases where the MX68CW is used with only cellular, but the advice to onboard and configure it on a wired connection first is very helpful, we will be able to do that.

Quick update on the cellular testing - we found it works just fine with an 'EE' sim, with no change to our Juniper policy, but it fails in a similar way with 'Three' and our corporate standard Vodafone. 

I have a Change going through today to allow UDP in to the DC MX250s.  Where carrier NAT/PAT is translating ports I believe the arrival of a post PAT packet at the MX250 is the missing element that allows it all to be untangled by the MXs and registry.  I guess I'll find out one way or another later today. 

[AndyGray]

Just a quick update.

Letting UDP inbound to the DC MXs did fix it. The VPNs came up and stayed up for several hours, so we're going to proceed to a live trial with the MX68CW in some temporary office space.

The MX68CW complains about not having proper contact with the registry when on LTE but one way or another the VPN endpoints seem to figure it out.

We also did some comparative signal strength and throughput tests between the internal LTE and an external Mikrotik 4G router. The MX performed better, repeatedly and consistently. The Mikrotik can of course be more easily moved around to find a better signal so still handy, but overall I'm really impressed with the MX68CW.

Cheers

Andy

[NolanHerring]

You can also get the new Meraki MG21 and that will give you the mobility to move the device anywhere (even outdoors) for better signal. 😃