Route Specific Traffic over VPN

[Phil_SCDS]

Hello, I have 2 sites connected to each other currently using the auto-vpn functionality. The Hub is running an MX84 and the Spoke an MX68. Due to the nature of the internet usage some traffic has to be routed to the hub site while the rest is normal internet usage. Currently I have it set up at the spoke site to use the hub as a default route as I cannot seem to route traffic destined for specific IP addresses only through the VPN. The issue with this is that all internet traffic is sent over the VPN and it has cut download speeds in half. Ideally I want to set up routes for traffic that is only trying to reach specific IP addresses to be sent to the hub site. I understand that this might not be possible with the auto-vpn and that I may have to set up a site to site vpn manually. I am happy to do this if it solves my issue but I need to know how to set up the static routes once the non-meraki site-to-site vpn is in place.

Thanks

[MarcP]

https://documentation.meraki.com/MX/Firewall_and_Traffic_Shaping/MX_Load_Balancing_and_Flow_Preferen...

should be possible through flow preferences

[Phil_SCDS]

Hi MarcP,

Thanks for your reply.

Flow preferences seem to only allow you to select wan 1 or wan 2 as the route for the traffic. I can't see a way to say specific traffic only uses the VPN. Unless I am missing something. The screen shot below shows that the preferred uplink is only wan1 or wan2. If there was an option there for vpn then I think it would work. 

[el_pajaro_bobo]

Hi Phil,

normaly you should be able to achive this, by adding the desired networks into the Security Assosiations (SA) of the MX84. So you need to have a route for these networks under Addressing and VLAN in the MX84. This way you can set them in you SAs on MX84 site so they are published to the MX68.

Steps:

- Add networks you want to reach on MX84 under Addressing and VLANs

- set the in VPN marker

Then you should be able to remove the "default GW" and be able to have the local internet breakout and reach your servers

[Phil_SCDS]

Hi el_pajaro_bobo,

This sounds promising, I shall give this a go and report back. Could you clarify what you mean by "VPN marker"?

Many thanks,

Phil

[Phil_SCDS]

Also, I assume I am adding the subnets from the spoke site (MX68) to the subnets section under addressing and vlans?

[Phil_SCDS]

I think I worked out what you mean by "VPN marker". When I try to add a vlan for the destination network it wants me to add an ip for the mx, the router doesn't have an ip on this range so I am unsure what I should add.

Thanks for your time.

[el_pajaro_bobo]

I mean the "in VPN" Checkbox which you can mark by adding a route.

So by adding the route you say the meraki over which router it can reach this specific network. For example you got a router A and router B, router A has a route to B and knows which subnets are behind this specific router. Let's suppose your Meraki is behind router A. You need to let the Meraki know which subnets can be reached through router A, otherwise it only knows about the directly connected ones. I've made a little drawing to hopefully clear things up a little 🙂

If you take my drawing your routing table would look like this (with static routing):

Meraki:

192.168.10.128/25 -> Router A

192.168.0.0/24 -> Router A

172.16.0.0/16 -> Router A

10.0.0.0/8 -> Router A

192.168.20.128/25 -> directly connected

Router A (the left one):

192.168.10.128/25 -> directly connected

192.168.0.0/24 -> Router B

172.16.0.0/16 -> Router B

10.0.0.0/8 -> Router B

192.168.20.128/25 ->Meraki

Router B (the right one):

192.168.10.128/25 -> Router A

192.168.0.0/24 -> directly connected

172.16.0.0/16 -> directly connected

10.0.0.0/8 -> directly connected

192.168.20.128/25 -> Router A

And then you should be able to reach the Server 🙂

Have you already tried to ping the Server from the Meraki connected to the network of it?
If the ping is possible you can add the Network to the Site-2-Site Firewall rules and this will create the route described above 🙂

Also your daily dose of kb and routing https://documentation.meraki.com/MX/Networks_and_Routing/MX_Routing_Behavior#Route_Priority

[Phil_SCDS]

Hi el_pajaro_bobo,

Thank you so much for all your help. This really helping to clear things up. In my case the addresses that the spoke site (mx68) is trying to reach are internet addresses not local networks. The hub site has an NHS broadband connection and the addresses are only available via this. On the mx84 there are traffic shaping rules to make sure that clients trying to access the NHS services use the NHS broadband (wan2 in this case). I would assume from your instructions that I could create a static route on the spoke (mx68) saying NHS traffic go to the mx84. However if I enter an internal IP address of the mx84 into the static route on the spoke router then it says the next hop address is invalid. Can I create static routes to the other MXs inside the auto vpn, or does this need to be done from the destination router so the mx84 in this case?

Thanks again,

Phil

[el_pajaro_bobo · Accepted solution]

Hey Phil,

yes, you need to create the routings on the destination Router (for LAN Addresses).

Since you want to reach WAN-Addresses you would need Policy Based routes, which the Meraki is not capable of 😞

So unfortunately I don't think that this is possible with Meraki atm 😕

Sorry I couldn't help more

Greetiungs

Sascha

[Phil_SCDS]

Hi Sascha,

Thanks for trying, I did see the same conclusion on a different forum but your explanation is much clearer, thank you for taking the time.

Phil

[Charlie]

@Phil_SCDS Ran across this post and was wondering if you ever came up with a solution.  Your example is exactly what I ran into and people have tried to direct me to the flow preferences.  I want it to give me the option to identify my internal traffic and then split-tunnel that network into my auto-vpn tunnel and anything that falls outside of that out to the local ISP to the internet.  Just wanted to see if you found a solution. 

[Phil_SCDS]

@Charlie Unfortunately the short answer is no. I believe it would be possible with another layer 3 device which you could use to essentially split the traffic into what you wanted to go over the auto VPN and what don't want to. I haven't tested this yet but it is something we are considering doing. I have also used the "make a wish" button to hopefully make Meraki aware that this would be a useful function. If you and others can do that too then maybe it will be available in a future firmware update. Currently we have left it as all traffic going over the auto VPN which of course slows down the remote site but in my case not enough for it to be a huge issue.

[Charlie]

@Phil_SCDS  Yeah I was toying with the idea of additional hardware as well.  I had previously used the make a wish as this need comes and goes in our organization (we end up not using Meraki in places that need a split the way we want it) but I will reinforce my previous "wish" with another today.  🙂    Thanks!

[MarcAEC]

As of March 2021, this method does not work.  See my 3/7/2021 post for a revised method.

The solution I had posted in November 2019 has been working.  The short of it is that you create a static route on the hub MX for the internet addresses you'd like to through the VPN tunnel.  For next hop, you put the hub MX IP.

This would create a routing loop.  So, you want to uncheck the Enabled box. 

This will cause the route to be advertised through the VPN tunnel by the hub MX.  The spoke MX will see the route advertised by the hub MX and send the traffic through the VPN tunnel.  Then when it arrives, the hub MX will send it out the WAN interface.  One would expect the hub MX to stop advertising routes that have been disabled, but that hasn't happened in the 3 months since I set this up.

I opened a support case to get clarification, since the setup seems a bit suspect.  The support rep communicated with the development team and relayed the following:

1. This is not intended behavior.  While the team doesn't have any plans to actively change how it is working, a future fix for an unrelated issue could have the side effect of closing up the loophole.  So, it's use at your own risk.

2. The team does plan to implement a supported way to accomplish selective routing of internet traffic through VPN tunnels.

My plan is to ride out the work-around until the supported solution is available.  The worst that can happen is that the work-around stops working before the official solution is available and I'm back to where I started. But right now things are working exactly as I wanted, and I didn't have to buy any extra equipment.

If you are doing pre-purchase research, don't base your decision on this work-around, because it is unsupported and can disappear.  But if you've already got Meraki gear running, you don't have much to lose by trying it out.

[Spruck-IT]

nice!
thanks alot
you get a beer in mannheim if you travel to. 😄

[ansan]

A late "thank you" as this appears to be working well for a specific use case in our environment even if it is unsupported and could evaporate at any time with a new firmware update.

I cannot state how great it will be to eventually have this as an official capability on the platform if and when it is implemented.

[MarcAEC]

It seems that day had finally come.  On Friday, I found the work-around not working and the desired traffic going direct to the internet from each spoke site.  I'm not sure how long it had been like this.  I had expected a firmware update would be the cause of this, but the one is installed is the 14.53 release from 6/2/2020.  It's possible Meraki released an update to the cloud controller, and that changes how the rules are downloaded to the devices.

[MarcAEC]

I've tweaked the rule and re-established the desired routing behavior.  As before, this is not a supported feature and Meraki can break it at any time with an update.  Of course, the hope is that they release an official method to accomplish this, as this is a basic feature that is very useful to have on an "SD-WAN" device.  The modified technique is to leave the rule enabled and then switch the condition to "While host responds to ping". Because the rule would create a routing loop, the host would never be able to respond to ping if the rule were active.  The MX still advertises the route to its peers, but then sends the traffic out the internet when it arrives.

[MarcAEC]

It looks like Source Based Default Routing (SBDR) in the 15.4 firmware (which is stable release candidate now) may work for some use case scenarios.  It looks like it supports routing all of a VLAN 's traffic across the VPN.  If you have VOIP phones on their own VLAN and want to send all of the traffic through to a central gateway before hitting the internet, SBDR could work.

[mspit]

Thank you for your post I've got this working in a test scenario.   My question is what happens when, "Host IP to ping" is publicly pingable?  Guessing we wind up in the same loop?    More importantly we can't Meraki just allow simple feature like this?

I'm interested to see if the source based routing somehow makes this supported method now.

The subnet from the static route shows in the list of *sources*, which sounds like a path to another unsupported hack.

[MarcAEC]

In the rules I have set up using this method, the next hop IP is (normally) pingable.  So, that's not a problem.

I haven't had a chance to try source-based routing because Meraki broke netflow in 15.x and didn't fix it until 16.x.  From the documentation, it looks like source-based routing will handle some use cases nicely.  The rule applies to an entire VLAN.  So, if you want VOIP traffic to egress via a specific MX, it should work for that.  But if you want all of your internal web site traffic to egress via a specific MX so that marketing can exclude it from the visitor stats, I don't think source-based routing will work because that traffic is going to be mixed with the traffic in the normal data VLAN.

[DerrickE]

Just checking in to say I've been searching for a way to do this for a while and this method is still working as of MX 17.6, thanks!

[MarcAEC]

I've been struggling with this for the past few days.  The company has VOIP telephone service.  Each branch has two WAN uplinks.  If an uplink goes down, then the phone calls will drop when the branch MX switches to the second uplink.  If I can route the VOIP traffic to the MX at the data center, then the branch uplinks can switch without the VOIP service seeing a change in the IP address.  I can also simplify IP whitelisting with 3rd party services if I can backhaul all of the traffic to those services through the hub WAN IP address.  But I can't send all of the branch internet traffic to the hub because that would kill the hub's uplink bandwidth.

I opened a support case and the rep responded it is not possible to set this up.  I called in and the second rep told me if I can get the hub MX to advertise the route, the branch MX will send the traffic over the VPN tunnel.  When I asked how to do that, he told me to put a static route into the hub MX with the IP of interest as a /32 subnet and the MX IP as the next hop.  I tried that, but the traffic got into a routing loop.  One would expect this to happen, but I have seen the Meraki dashboard have special support to "do what I mean" with some things like this. 

To resolve the routing loop, I disabled the static route.  Then something amazing happened.  Things started operating exactly as I wanted.  The hub MX continued to advertise the route to the branch MX, and the branch MX happily sent it the traffic, but when the traffic got to the hub, the hub MX sent it out the WAN link.  Perfect!

So, this is great.  But I am worried that this behavior would be considered a bug that will eventually be "fixed".  If I were a programmer of the MX firmware, I would not advertise static routes that are disabled.  The point of disabling something is to completely remove it from use without losing the information of what used be active. 

[MarcAEC]

This has been working great.  I used this technique to configure VOIP traffic to go through the VPN tunnel out a reliable data center fiber connection.  A colleague and I tested a phone call between a VOIP phone and a cell phone.  I unplugged the WAN1 link and the call continued.  I reconnected WAN1 and unplugged WAN2, and the call continued.  This is something I did not think was possible with Meraki.  We did notice brief interruptions due to lost packets.  But it was much better than expected.

I think the Meraki team is missing out on an opportunity here.  The Velo Cloud sales reps kept telling us how VOIP calls don't drop with Velo Cloud, but they do with Meraki. I was on a call with Meraki sales reps telling them I didn't think Meraki was "real SD-WAN", and they had no response.  And that's the sentiment I got from third party sales reps that sell both.  They told me if I wanted true SD-WAN, to get VeloCloud.  If I wanted a more reasonably-priced solution, get Meraki.  One VeloCloud sales rep used the phrase "you get what you pay for".

However, it is possible with Meraki to do a spoke failover without VOIP calls dropping.  If they wanted to, the Meraki team could implement packet duplication and jitter protection, and compete with the VeloCloud VOIP feature set.

[hianilz]

@MarcAEC  Hi, I am looking for same kind of solution where i have 2 WAN links at my Remote sites.., where WAN1 is for Internet access(through Zscaler tunnel) locally and WAN2 is MPLS where site Private subnet (Intranet traffic - example: 10.14.x.0/24 - which includes VOIP subnet)  will reach DC for Servers hosted.

Now i wanted a seemless failover(for Internet traffic) to WAN2 if site WAN1(Internet Link) goes down.,

how can i achieve will you be able to help ??

[MarcAEC]

I would think this would happen automatically.  If WAN1 goes down, the MX will automatically try to send the traffic over WAN2.  I've got a use case where I'd like to block that, and couldn't find a way to do it.

[hianilz]

@MarcAECthanks for your reply. Yes as per auto vpn it will failover but worry is how remote site subnet access internet ? As Soon as DIA link goes down entire traffic will be failover to MPLS link and it will go to HUB.., do I require me to add any routing on HUB or Remote sides to make sure seamless for internet access, especially VOIP as they are using internet link for SIP tunnel 

[MarcAEC]

With VOIP, if the internet circuit the call is using goes down, the call would probably drop.  If the internet circuit you have at the hub is more reliable, I'd recommend setting up the VOIP traffic to go to the hub even when the local DIA circuit is online (unless there is a large geographical distance that would cause the latency to be too high).  That way when the DIA circuit goes down, the VOIP traffic will not change how it is connected to the provider.  VOIP traffic typically has its own VLAN.  So, you'd use the source-based routing feature introduced in 15.x for that instead of the IP-based technique discussed in this thread.

[Frank-NL]

Hi,

Thanks@MarcAEC for sharing, I stumbled upon this and the solution still seems to work good.

And it looks like the sharing of the 'disabled' routes is expected behavior as also described in the knowledge base:

https://documentation.meraki.com/MX/Networks_and_Routing/MX_Routing_Behavior

Advanced Configurations->Static Route Tracking:

The route tracking mechanism only has local significance, which means, that the status of the route doesn't influence whether the route is advertised or not and it won't affect the routing on the remote end of the VPN.

Cheers