Multiple Default Routes in an SD WAN?

[mackem]

Hi

If you have the following scenario:

Company 1 HQ

Company 1 satellite office

Company 1 satellite office

Company 2 HQ

Company 2 satellite office

Company 2 satellite office

All companies are a part of a Group of companies so communicate with each other over the same SD WAN.

The satellite offices need to have an Internet connection which is fire-walled and filtered by an appliance (non meraki) at its respective HQ site. In other words, the internet for the satellite offices needs to be back-hauled to its respective HQ site Company 1 satellite office -> Company 1 HQ and Company 2 satellite office -> Company 2 HQ .

I have setup:

• Company 1 satellite office as SPOKE with default route to HUB Company 1 HQ
• A 0.0.0.0/0 route on Company 1 HQ MX pointing to the LAN core switch at Company 1 HQ
• The 0.0.0.0/0 route on the MX is not advertised into the SD WAN as i only want this to affect Company 1 satellite offices, not Company 2

Set up the same for Company 2 sites. What i've found is that this basically doesn't work, the internet bound traffic is routed into the HQ successfully, the traffic returns and hits the LAN side of the MX at which point the MX drops the traffic. I have had a ticket with Meraki over this (however this was raised while all sites were HUBS) but the result was the same, if the 0.0.0.0/0 is not advertised in the VPN, the return traffic is dropped by the MX. In my view I think this should work as the routing is good, however the inherent behaviour of the MX's is to drop the traffic.

The workaround currently is to have all internet bound traffic forwarded and filtered at 1 HQ, however for my business this is not desirable. I keep coming back to this to see how it could be achieved. I have even updated some sites to the Beta firmware which includes source based routing functionality however this is still not doing the job, since you can only select source networks that are local to the MX on which you are setting up the source based route!!

Any geniuses out there can tell me if this can be done? 😫

[ww]

You  can 't advertise on  both hq1 and hq2 the default route?

[mackem]

I was worried about the effect of doing this as there would be duplicate routes so how would sites know which to route to? I will try it and report back

[ww]

Sat2 spoke settings needs  to have hq2 as first  at the hub priority

And sat1 need hq1 listed as first

[mackem]

Yeah i did that already

[cmr]

Set all up as hubs and then you can pick a different exit hub for each site that used to be a spoke, i.e. spoke 1 is now a hub and has exit hub of company 1 HQ hub or whatever you need.

[PhilipDAth]

You shouldn't need to advertise the default route.  On the spoke you you can specify which hub to use (company 'A' or 'B') and then just tick the box to use that hub as your default route.

[AFamilyGuy]

I think PhilipDAth has a great answer for you. Having said that, I'm curious about your analysis that the return traffic gets dropped at the HQ hub. How have you confirmed this? Is autovpn advertising the subnet that's sourcing the traffic from the satellite office?

[mackem]

The default route that you specify on the spoke / having an exit hub isn't enough, in this case i want internet traffic to be forwarded onto the LAN not out onto the internet via the MX which is what happens. Therefore you need another manually created default route on the hub MX to point to the LAN, however this needs to be advertised in the autoVPN otherwise the return traffic is dropped. This was confirmed with packet captures as the traffic is seen leaving the LAN interface on the MX and also the return internet traffic is seen hitting the LAN interface on the MX - then the traffic is dropped, nothing is seen returning on the autoVPN. This is despite there being a valid return route for the return traffic in the routing table. Meraki have confirmed this is expected operation, if the default route is not advertised in the autoVPN.

[cmr]

Can you set up the HQ MXs in concentrator mode or do you need them on the edge?  We use MXs at the edge in routed mode for most sites, but the DCs have them in concentrator mode, then the exit hub / default hub feature works as you would expect.

[mackem]

That's interesting i wonder if that would work for us better, however it would mean a fairly drastic redesign. I can't see a reason why we couldn't have them in concentrator mode as we have existing FW's at our HQ sites. How does routing work between concentrators, can you set up tunnels between them to retain full connectivity throughout the SD WAN?

[PhilipDAth]

Are your HQ MX's running in VPN concentrator mode?

[mackem]

No, all of our MX's were configured as routed with full mesh when first installed

[Tungstenyiu]

Hi Mackem,

Have you solve the issue so far? 

Is advertise on both hq1 and hq2 the default route and set the hub priority at spoke site work? Thanks!