cancel
Showing results for 
Search instead for 
Did you mean: 

MX with MPLS and NO Internet

Conversationalist

MX with MPLS and NO Internet

Hi, I've battled with this issue in the past, and am about to do so again, so wanted to check if I'm doing it the "right" way.

 

I have a multi-site client with an MX at each site, and a single MPLS connection at each site, that will route all traffic back to HQ.

At HQ, the MX has an internet connection on WAN1 and the MPLS connection on LAN port with static routes.

At the branch, the MPLS connection connects to a cisco router, which then splits the /24 into a /25 and a /29. The /29 is connected to the MX WAN as a Internet connection, and the /25 is connected to the MX LAN with static routes.

 

All that means that we need additional cisco router, or layer 3 switch with VLAN, but at least it works.

 

I want to reduce that to simple connect the MX directly to the MPLS, preferably on the WAN, and allow it to get Internet access, and route private IP address space. I think the issues with this are:

1) The MX will NAT the private IP addresses, instead of just routing them and letting the HQ MX do NAT at the Internet gateway

2) The MX requires a working Internet connection on all WAN ports (ping to 8.8.8.8 at a minimum)

 

What do others do? Is this functionality better in firmware 13/14 ?

16 REPLIES 16
Kind of a big deal

Re: MX with MPLS and NO Internet

1. The MX will NAT any destination address not in its routing table, with the exception of the default route.  So you need to include a route for all private IP address in your network on the other side of the WAN port of the branch MX.  Traffic to the Internet will be NATed by the branch MX.  The only other except would be if you were running the MX in transparent mode ... but you are going to loose a lot of features.

 

2. The MX must have working Internet connection, so this can be provided via the HQ connection.

 

 

There are two common ways of doing this.  First use AutoVPN and run a VPN over the MPLS network.

https://documentation.meraki.com/MX-Z/Site-to-site_VPN/Configuring_Site-to-site_VPN_over_MPLS

 

The other is to simply use routes, or tracked routes if you want to be able to fail over to a backup VPN.

https://documentation.meraki.com/MX-Z/Deployment_Guides/MPLS_Failover_to_Meraki_Auto_VPN

 

ww
Head in the Cloud

Re: MX with MPLS and NO Internet

why not use a cheap internet provider or 3/4g dongle in addition.

Here to help

Re: MX with MPLS and NO Internet

Usually to provide SD WAN you would have an MX in one armd mode at HQ.  In your environment I would try connecting WAN 2 to MPLS and then it should be fine for all the MX appliances to NAT.  The autovpn will form an overlay on the MPLS network. You should not need additional routers.  In this setup your MPLS provider will not need any lknowledge of your internal networks, as they will be nat'd via the MXs.

Comes here often

Re: MX with MPLS and NO Internet

deleted

Here to help

Re: MX with MPLS and NO Internet

I have the same case, where I would like to run MX devices(hub and spoke), on both I will connect the internet to WAN1 and MPLS(without internet access) directly to WAN2, then form an SD-WAN setup over both links.

This what I will try! I hope it works!

Here to help

Re: MX with MPLS and NO Internet

Hi @hilehoffer 

This did not work for me, i am using a pair of MX68 devices where both are connected on WAN1 for Internet and on WAN2 for MPLS.

I could see the tunnel is up on the WAN1, but will never be up on the WAN2, is there a special configuration I need to add?

Kind of a big deal

Re: MX with MPLS and NO Internet

If the tunnel is not up on WAN2 (so assuming you are using AutoVPN over MPLS) make sure the that the public IP address that the spoke gets NATed to is the same as the public IP address that the HUB is nat'd to when the traffic goes over the MPLS, otherwise it wont work.

Here to help

Re: MX with MPLS and NO Internet

Hi @PhilipDAth Actually I am trying to achieve what we have Auto VPN over MPLS and SDWAN:

 

https://documentation.meraki.com/MX/Site-to-site_VPN/Configuring_Site-to-site_VPN_over_MPLS

https://documentation.meraki.com/Architectures_and_Best_Practices/Cisco_Meraki_Best_Practice_Design/...

 

 

With the following assumptions:

-Both MX Devices will run in NAT Mode (Not VPN concentrator)

-Both MX Devices will have one leg to MPLS (which has a router splitting traffic to the internet)

-Both MX Devices will be connected to WAN1 to the internet

 

 

I cannot get the tunnel on WAN2 up, I am not sure how to configure it.

Kind of a big deal

Re: MX with MPLS and NO Internet

The WAN2 interface of everything MX, including the DC/head office, needs to get NATed to the same public IP address.  If you look in the dashboard, does it report the same public IP address for WAN2 for each MX?

Here to help

Re: MX with MPLS and NO Internet

Thanks @PhilipDAth 

So we got everything set now, and both MX devices see there interfaces as active.

However I am facing an issue with VPN tunnels, when both lines (MPLS and Internet) are up and running, everything works great (eventhough I did not test SDWAN and path selection yet).

However when the Internet line is down at any of the devices, the tunnel would be down and it seems that the Internet1 of the working Internet line is trying to form an IPSEC tunnel to the other MX device over the MPLS link! and it not getting up (that is normal because the MPLS connectivity does not allow traffic from the outside except for Meraki Cloud).

I attached a diagram.MX Setup.jpg

Comes here often

Re: MX with MPLS and NO Internet

I used internet line into WAN 1, with MPLS into LAN port 1.

 

Created the Site to Site VPN and included the MPLS subnet into the VPN.

 

Created a static route pointing all traffic destined for any MPLS connection 172.16.0.0/12 (for example) via the MX MPLS subnet gateway IP address. 

 

MX will always favour the static route over the VPN route. So no NAT'ing to worry about on MPLS traffic, say voice or other business critical apps that dislike NAT. Local breakout too, which speeds up things like office 365 over going through a central HQ/Datacentre. 

 

We have the advanced security licence to help with security. In a nutshell, that's pretty much all I did. 

 

Everyone's a winner.

HTH. 

Here to help

Re: MX with MPLS and NO Internet

I depends on the type of applications/ traffic flow you have. In my case I do not mind traffic being Natted on MPLS network (no voice traffic), and I was able to achieve what I want from the setup

However I would always prefer to have WAN links to be connected to WAN1 and WAN2 then achieve an SD-WAN setup or load balancing

For your case, you are stuck with static routes which hinders the power of SD WAN policies. Meraki recently added to the beta firmware release 15.2 a feature for NO-NAT where the MX behaves like a L3 device, but does not do NATing on the uplink. This makes a perfect option for your case, if you would like to take the risk of having running a beta version

Conversationalist

Re: MX with MPLS and NO Internet

Some of the replies here seem to be correct, and I finally had the chance to do this on a live network, and can confirm it does work, with some pre-requisites.

 

1) One site must have Internet access, for us, this is the "Head Office" or "Main Site". It could be your DC or whatever. Connect this to WAN1 (or WAN2 as needed).

2) The MPLS connection will be on LAN side of your head office/main site. Add static routes as needed, so your head office knows to use the MPLS connection.

3) Your remote office will connect the MPLS to WAN1 (or WAN2) and if you have a backup Internet, connect it to WAN2.

4) Use site-to-site autoVPN

5) Set your traffic preferences to use the MPLS (or split however desired).

 

In my case, my MPLS doesn't have an Internet connection, so it took some discussion with Meraki Support to accept that this happens in the real world.

I also didn't want to buy another MX just for VPN, I already had a perfectly working MX acting as my Internet gateway/router.

 

Also, I wanted the Internet on WAN ports, and also the MPLS on remote sites on WAN ports.

Finally, I wanted automatic failover from MPLS to Internet without any manual intervention.

My understanding is that the remote site has "two internet connections", one is the backup Internet on WAN2, the other is via the MPLS to head office, and then out the Internet connection there. The reason this magically works is because meraki "see" the autovpn on from the remote office WAN1 and the head office as having the SAME IP address, so it shares the internal IP's with both of them, and they then setup the VPN directly, and thus it works.

 

If you want more details, let me know, and I can add in some more IP address examples, and try to expand on the description. I'm hoping to roll this out for another customer shortly. My plan is that all remote sites would have a second Internet on WAN2 (4G mobile), with MPLS on WAN1.

Comes here often

Re: MX with MPLS and NO Internet

Agree re: no NAT beta version. However we're not an organisation prone to taking 'risks' such as using beta software. Should it ever make it to a stable version, then we're likely to be all over it. 

Highlighted
Here to help

Re: MX with MPLS and NO Internet

We have our MPLS plugged into WAN1 at the branch, and landed the MPLS connection at our datacenter onto our Nexus 7k.  We created a VLAN that allows traffic out to the internet only, and ACL'd our local stuff out of it.  We then created a VRF and landed the MX into the nexus on the same VLAN.

 

The down side to this is that the headend MX will always show WAN1 as the internet traffic for it bleeds out the VLAN we created and up to our 3rd party firewall.  The branch sites will work correctly as if the MPLS is down then the meraki traffic can't bleed into the vlan because it can't get to the vlan.

 

We do have internet at all locations as well plugged into WAN2.

Here to help

Re: MX with MPLS and NO Internet

@AdamAus , would you please share your topology with IP addresses?

Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.