MX with MPLS primary LTE VPN failover

[brad1028]

Looking to impliment MX on a Hybrid MPLS / Internet network. The MPLS only sites will have a MX67C. Primary WAN will be MPLS with Internet access through hub site, looking to failover to integrated LTE in the event of a MPLS outage.  

See two methods:

1. MPLS on the WAN port,
   • VPN tunnel is built over MPLS link,
   • Meraki should detect a WAN outage and fail to LTE
2. MPLS over the LAN ports with static routes. 
   • No additional encryption and tunneling.
   • There is no primary WAN since LTE is failover only. 
   • This works better for my network from a design perspective but don't see it working?

[GreenMan]

If you don't provide a fixed WAN uplink to your MX (your scenario 2), the MX will utilise the LTE uplink, as a permanent backup.   This will work - provided you get an adequate mobile data uplink.   All of your MX management traffic, plus all your tunnel keep-alive traffic, will flow over the LTE link, so take the potential cost of this into account - you will really want to use a proper data plan for such a solution.   Bear in mind that your static routing towards the MPLS needs to be sufficiently broad to catch any user traffic you want to flow over the MPLS, otherwise it will NAT out over the LTE link (it will probably be a default route you use, TBH).   You cannot manipulate the static route to also encompass the MX management traffic.

On a wider note;  this isn't really the way most businesses are building their networks, moving forward: for most businesses, their workloads are increasingly moving to the public cloud, rather than private data centres.   Hairpinning all such traffic through a shared central DC Internet breakout, over the MPLS, is pretty inefficient, detrimental to user experience and usually expensive.   Have you looked at a fixed Internet link as an alternative (to be able to use as path 2, for a true SD-WAN solution similar to your scenario 1)?
As another alternative, why not deploy the mobile data link using an MG21, rather than within the MX, connected via WAN2, to make it, effectively a fixed uplink - then use SD-WAN over those two paths?

I'm sure you'll have found it, but your scenario 2 is documented here, albeit without the LTE providing the WAN uplink:   https://documentation.meraki.com/MX/Deployment_Guides/MPLS_Failover_to_Meraki_Auto_VPN   

Remember that you have to pre-provision your MX via a wired uplink - you can't bring it up for the first time solely over mobile data.

[PhilipDAth]

>MPLS on the WAN port

I prefer this approach when you want cellular data failover.  In this case (since you have an MX on every site) the MX is aware of all the routing in the environment.  Otherwise, with the VLAN approach, you have to add tracked routes.

This approach is also the more difficult of the two.

But as you note, both approaches are valid.

[RichG]

I have option 2 working.  Like you, I found it a better fit for the existing network design in situations where there is an existing robust MPLS WAN.  The key is corresponding conditional routes on the MX at both ends.

 - MX A has a route to site B VLANS through MX B if it can ping MX B

 - MX B has a route to site A VLANS (or 0.0.0.0) through MX A if it can ping MX A

 - If the ping fails, the VPN also has that route, but with a lower priority, so the VPN takes over

Most of my sites already had OSPF in place, so most of these are actually "While host responds" not "While next hop responds" and they are pinging the OSPF gateway at the remote site.  The hub site injects the default route into OSPF and the remote sites have a high metric default route to the MX, so the OSPF default takes priority.  If the MPLS circuit fails, OSPF loses that route and the default goes to the MX, where the VPN takes over.  At one site, the backup is via cable modem, not cell, so it will also advertise the default route, but with a worse metric than the hub site.  Our MPLS is over a point-to-multipoint, so any sites that can get to that site can share that cable modem.

My preference is to have a Cradlepoint or other LTE gateway device (like the Meraki MG if you want to stay on brand) connected to the WAN, rather than the built-in USB LTE backup.  It works much better and the MX doesn't complain.  If I don't have that, I have the WAN also connected back through a separate VLAN on the MPLS circuit so the MX can get to the dashboard, but I don't route any traffic that way. 

This let me add the MX for LTE/cable modem backup without disrupting my existing network.