Guidance needed on how to best make this system work best with active-active datacenters

[Devaul]

Hello all,

Our HQ and primary DC is in NY.  The secondary DC which is partially a DR site but also has some active services is in NC.  We have branches in NY, NC, and SC connected to our DCs through metro-e connections.  Currently, with our ISR routers, NY branches connect to the NY DC, while NC and SC connect to the NC DC.  All branches also have cellular gateways as a backup connection.   

Below is how i currently have things hooked up in my lab phase.

The Metro-E is any to any so the 2 DCs can potentially communicate through it and branches can potentially communicate directly with both DCs.  I say potentially because in our current build pre-Meraki, that's not how it's set up.  You'll notice there's 2 more connections between DCs but my goal would be for everything branch related to stay on Metro-E until they reach the DC that whatever service is at.  Meaning, I would prefer a NC branch doesn't send traffic to the NC DC to then ride the p2p to NY and instead send traffic to the NY Hub, unless the NY Hub is down, then it would come into NC and traverse the datacenter p2p.

Note:  We currently plan for traffic to stay on Metro-E unless a branch needs to connect through their Cellular backup.  The MX105's are getting internet access through the default gateway on the L3 switch.

I believe my main questions at this point are:

Is it better to be using One-arm Concentrator or Routed mode for this?  I started with Routed mode and then just switched to VPNC after reading about dc-dc failovers.  Now i'm believing I don't actually want DC-DC failover because I don't need any local networks to failover between DCs.  I just need traffic to reroute.  Would either method have limitations in my scenario?

Note:  even if i were doing Routed mode, i still need the L3 switch in NY due to the HA pair of MX105s. 

Why would my cellular backups at the branches be able to form a VPN with the HUBs in routed mode but not VPNC?  Something to do w/ auto-NAT maybe?

Is it going to be possible to have branches form connections to both HUBs and send traffic to each?  Can the 2 HUBs send traffic to each other which essentially means there's a third connection between my datacenters?

[RaphaelL]

Why would my cellular backups at the branches be able to form a VPN with the HUBs in routed mode but not VPNC?  Something to do w/ auto-NAT maybe?   One Arm or Routed mode shouldn't matter. You should be able to build the AutoVPN on either.

Is it going to be possible to have branches form connections to both HUBs and send traffic to each?  Can the 2 HUBs send traffic to each other which essentially means there's a third connection between my datacenters?  No. Only one active HUB per Spoke. Failover will occur and the Spoke will converge to the next HUB in the list / priority order. Tunnels are always built but no data will go the other hubs.

One arm + warm spare is more than enough in your scenario.

[PhilipDAth]

I'll add this caveat to the above:

https://documentation.meraki.com/MX/Deployment_Guides/Datacenter_Redundancy_(DC-DC_Failover)_Deploym...

"In a DC-DC failover design, a remote site will form VPN tunnels to all configured VPN hubs for the network. For subnets that are unique to a particular hub, traffic will be routed directly to that hub. For subnets that are advertised from multiple hubs, spokes sites will send traffic to the highest priority hub that is reachable."

[PhilipDAth]

Is this a stretched layer 2 design, or does each DC have its own unique IP addressing?

[Devaul]

Each DC has unique IP addressing

[PhilipDAth]

Checkout this guide, "Datacenter Redundancy (DC-DC Failover) Deployment Guide":

https://documentation.meraki.com/MX/Deployment_Guides/Datacenter_Redundancy_(DC-DC_Failover)_Deploym...

[MartinLL · Accepted solution]

Cool setup!

So, to start with i would begin by configuring BGP unless its not already done. By the looks of it you will need the flexibility dynamic routing provides.

---

Is it better to be using One-arm Concentrator or Routed mode for this?  I started with Routed mode and then just switched to VPNC after reading about dc-dc failovers.  Now i'm believing I don't actually want DC-DC failover because I don't need any local networks to failover between DCs.  I just need traffic to reroute.  Would either method have limitations in my scenario?

Note:  even if i were doing Routed mode, i still need the L3 switch in NY due to the HA pair of MX105s. 

---

I would go for Concentrator mode unless you need the MX pairs in the DC to do some sort of firewalling.

Then i would set up BGP in the Meraki SD-WAN and do eBGP towards your L3 switches in each DC. This would allow you to advertise both datacenter ranges down to the MX pairs. If one DC or MX pair fails your meraki spokes can still reach that DC through the other VPNC across your datacenter interconnects.

As for your active active question. Just advertise the unique ranges to your SD-WAN and do AS-PATH prepend, that should force clients to take the more direct route as long as your DC is up.

---

Why would my cellular backups at the branches be able to form a VPN with the HUBs in routed mode but not VPNC?  Something to do w/ auto-NAT maybe?

---

Yes most likely a NAT issue. I would do a 1:1 NAT on your firewall towards the VPN HUBs Virtual IP. If you dont have IP addresses to spare, reserve a port and enter it manually in Meraki instead of using Auto-NAT.

---

Is it going to be possible to have branches form connections to both HUBs and send traffic to each?  Can the 2 HUBs send traffic to each other which essentially means there's a third connection between my datacenters?

---

Yes, you can add both VPNC as hubs to your spokes. If you want NC and SC to use the closest DC, simply create two network templates where NC and SC has the NC HUB as priority 1 and NY as 2. Then make one for NY spokes and do the opposite.

Yes, the hubs will build auto-vpn tunnels between them selves.