Auto VPN/SD-WAN over Private Networks

[Spivey]

I'm posting here because support hasn't found any documentation yet for a situation we had that I imagine is fairly common.

MX appliance WAN interfaces need Internet access and DNS resolution for their health checks and if doing Auto VPN, connectivity to the cloud hosted VPN registry service. It's a hosted solution, so it's assumed all WAN interfaces have Internet connectivity. The WAN interfaces don't care how they get Internet connectivity, so if they are using private IP addresses you just need a NAT and route out somewhere to get them Internet access.

We have an MPLS circuit using private IP addressing with no Internet access available. We needed this circuit connected as a WAN interface for SD-WAN. OSPF is also required on the LAN side, so using a LAN interface for the MPLS routing is not an option. The MXs are using Beta code to allow OSPF in NAT mode. Each site has direct Internet access and the MX appliances will be replacing the current firewalls and assuming the NAT responsibilities. The non-MPLS WAN interface is connected to a traditional Internet circuit.

The solution we have tested successfully uses the MX as it's own NAT device and Internet routing. We inserted a layer 3 switch between the MPLS circuit and the MX WAN interface. Then we provided a default route using the MX LAN IP as the next hop on this device with more specific routes for the MPLS networks. The site-to-site VPN tunnel traffic to other MPLS sites follow the MPLS routes while the WAN interface management traffic gets routed back to the LAN interface of the MX. The MX happily provides a NAT for the management traffic with no issue and the WAN interface with the private IP address now has Internet connectivity.

[PhilipDAth]

To form an AutoVPN over an MPLS network it is a requirement all the end points all go out the same end point and all be NATed to the same public IP address.

So when using AutoVPN over MPLS you must have a default route in the MPLS network pointing to one site, and that site must connect the MPLS network to the Internet.

Now you have the situation where "sitea" connects over the MPLS network to the Meraki cloud and is NATed to (say)1.2.3.4.

"siteb" connects over the MPLS network to the Meraki cloud and is NATed to 1.2.3.4

Because sitea and siteb are NATed to the same public IP address they can now form an AutoVPN connection over the MPLS cloud.

https://documentation.meraki.com/MX-Z/Site-to-site_VPN/Configuring_Site-to-site_VPN_over_MPLS

[Spivey]

Sounds like that might work well in an internet failover scenario from direct internet access.

[PhilipDAth]

Yes - I have used this config a lot with Internet Circuit + MPLS.  You can use it for pure failover - or both circuits at the same time.

Clients I have deployed it at tend to use cheap high speed Internet circuits - so tend to get me to configure it to send this bulk traffic over the Internet AutoVPN and use the MPLS AutoVPN for everything else.

[MerakiLover]

Hi! Question about this sentence: 

To form an AutoVPN over an MPLS network it is a requirement all the end points all go out the same end point and all be NATed to the same public IP address. 

What happens if the ip addresses are different? Let's say that the internet breakout is "local" and not centralized. 

Would the autovpn then get formed over the public internet addresses and so I lose the option to use the mpls backup for the AutoVPN? 

[Spivey]

If you are using the MPLS as a backup Internet egress point you need to send the Internet traffic to one egress point at a time, usually a hub. You should be able to have the primary Internet egress point for a spoke, for example, be local with a "backup" default route on the MPLS network. Just make sure you're not trying to use multiple Internet egress points at once. We usually add default routes into the VPN network over MPLS so the local static route takes preference until the interface is down in which case the VPN routes get installed. OSPF could be used as well, but Nat mode support was only available in Beta code last I heard.

[MerakiLover]

Hi Spivey, 

sorry but I think I have a different scenario that I try to describe: 

- customer with branches in different countries

- branches have one local internet connectivity (wan1) + one local mpls (wan2) that it has an internet breakout via mpls

what I need: 

- general internet traffic via wan1

- business traffic via mpls via wan2

I guess the requirements is that I need both Meraki mgmt traffic able to reach internet via both wan. Is that correct ? 

[PhilipDAth]

Under "Traffic Shaping" you can specify different flor preferences for Internet and AutoVPN (aka business) traffic.  You can tell one to use one WAN port and the other to use a different WAN port.

[MerakiLover]

my target is: 

having sd-wan and tunnels on both wans. 

What I don't understand is if Meraki mgmt traffic need to reach the Dashboard via both interfaces or not

[PhilipDAth]

To run AutoVPN over both circuits then yes, the management traffic must be able to get to try the Internet over both circuits.

[MerakiLover]

Good. Thanks. 

Now the question becomes: (  I add details to the previous scenario 😞 

previous scenario: 

- customer with branches in different countries

- branches have one local internet connectivity (wan1) + one local mpls (wan2) that it has an internet breakout via mpls

details: 

since the branches have an internet breakout via MPLS I expect the MX to inform the registry that it has on WAN2 a private ip address and a public ip address. This is on MPLS. 

I expect the MX being able to form two tunnels on this WAN2: one via private ip address on MPLS and one via the local internet breakout (via mpls backbone)

What would happen then? Would the MX prefer the tunnel on private ip addresses ? 

[Spivey]

Both interfaces need internet connectivity,  yes. That doesn't mean both circuits must have internet reachability. You could route internet traffic for one MX interface out another MX interface. That requires another L3 device of course.

The NAT point sounds like it will be in the provider network on the MPLS circuit. The MX will have no visibility of the public IP. The tunnel will terminate on the private IP that's assigned to the MX interface.

[MerakiLover]

ok, that's good. 

I understand I could route the internet reachability via a local L3 back to WAN1, but that's it not feasible in my case. 

I understand NAT is a matter of the ISP.. MX only see private ip. 

Are we sure that then MX would not create two tunnels via wan2 ? (one via mpls internet breakout and the other via private ip address?) 

I was thinking to filter out all the internet traffic and leave only the mgmt meraki traffic. Is that a good idea or useless? 

thanks

[Spivey]

Yes, you will only have one tunnel terminated on the WAN 2 interface. The tunnel will use the IP assigned to the physical interface.

Whether it's a good idea to filter internet traffic on the MPLS interface depends on your intent. Personally I would most likely utilize it as a backup to the local internet circuit. 

[MerakiLover]

Thanks a lot man.. You helped a lot!

[Spivey]

No worries, happy to help.