Meraki One Armed Mode HA

[Aamir]

Hi,

Is it possible to have MX in one-armed VPN concentrator mode HA across two different sites? I understand there should be a layer 2 link between the sites but according to the following link it seems they have to communicate to the dashboard via the same IP also:

https://documentation.meraki.com/MX/Deployment_Guides/MX_Warm_Spare_-_High_Availability_Pair

Thanks,

Aamir

[AjitKumar]

Hi @Aamir

The scenario is interesting however a little complex though. 

I do not see a great advantage of placing "MXes" at a distant sites.

Ideally you may place "MXes" at the same location and have the second site as a VPN Location.

Anyhow.

You rightly said "One Armed" concentrators in HA requires, the WAN Interface IPs  from the same Subnet / VLAN and reach ability among themselves.

I understand different internet connection shall not be a problem. As the fail over detection is via VRRP on the LAN.

I hope I am not wrong.

[GIdenJoe]

That is certainly not a recommended topology.

If you're doing Concentrator HA mode each MX must have it's own IP in the same subnet as the virtual IP and the upstream router.

You should also not have too much latency between both sites since VRRP hellos need to be succesful all the time.
If you only have 1 layer 2 link between the sites and that one goes down you'll effectively have a dual master scenario where routing towards your inside IP's will be indeterminate.

Using MX'es in VPN concentrator mode is usually intended for datacenter designs where you can have the same address space across multiple datacenters.  But then if you do HA both MX'es are on the same location.

So is there a reason outside of not wanting to pay another license why you'd want an HA pair over a longer distance link?

[PhilipDAth]

>it seems they have to communicate to the dashboard via the same IP also:

It will work with different public IP's but failover would be slower.

[PhilipDAth]

You could consider using active/active hubs.  You would connect spokes to both, marking one as the primary.

Each hub would need a route stub connection to the main DC network (because two MX's can not be directly connected to the same subnet).

[Aamir]

agreed to all the answer, I guess its better to do HA within the same site or put an individual MX in vpn concentrator mode in backup DC and have tunnels to both the DC. thanks a lot all.

Thanks,

Aamir

[Aamir]

Hi all,

Two more questions, does it still make sense to get adv. sec license for MX if its in one-armed concentrator mode in DC since DC would have its security devices? Also with once-armed concentrator mode, iBGP is enabled on the tunnels. If we use OSPF between MX  in one-armed concentrator mode  and the upstream device (switch) is there an automatic redistribution between iBGP and OSPF. I understand MX can advertised OSPF routes but cannot learn them.

Thanks,

Aamir

[Bruce]

If you have any MX devices in the organisation that need Adv. Sec. licenses then all the devices have to have it. If none of them need it, then you don’t need the Adv. Sec. license. It’s an organisation-wide setting.

If you’re using iBGP on the SD-WAN then you should be peering into your data centres with eBGP, not OSPF. If you’re using the ‘traditional’ SD-WAN (i.e. non- BGP) then the Auto-VPN functionality takes care of route distribution through the Meraki cloud, and you’re right, in this scenario the MX will advertise routes to the DC, but not learn from the DC.

[Aamir]

Hi Bruce,

The reason why I was mentioning to use OSPF on the iBGP on the SD-WAN scenario was in a situation where customer upstream device (switch) does not support BGP, hence cannot do eBGP peering between MX one-armed mode with upstream device (switch). In that case if we use OSPF between MX one-armed mode with upstream device (switch) is there an automatic re-distribution between iBGP and OSPF within MX?

[Bruce]

Hi Aamir,

If you can't do eBGP peering to the data-centre and use OSPF, then you will just be using the traditional AutoVPN route distribution, which doesn't use BGP (the BGP approach is an option). Have a look here, https://meraki.cisco.com/lib/pdf/meraki_whitepaper_autovpn.pdf, it doesn't go into depth about it, but gives a high-level outline.

[Aamir]

ok so using BGP on MX (iBGP over the tunnels and eBGP with upstream) is optional. So if our upstream device does not support eBGP we don't use iBGP also and can use OSPF or static routers? Is there any relationship of using BGP on MX for DC-DC failover scenario where each MX is advertising same subnet?

Thanks,

Aanmir 

[GIdenJoe · Accepted solution]

The power of using eBGP in the DC is the possible load balancing aspect.

So if you have multiple DCs where the same subnets exist then both MX'es will receive them via eBGP and distribute them using iBGP to the spokes.
In that case your spoke will send it's traffic towards the hub MX with the highest prio.

The reverse routing is also true due to using BGP.

The spoke subnets participating in AutoVPN will be distributed from the spokes to the MX hub via iBGP as being locally originated.  And in turn the MX primary hub for that spoke will give that route to the eBGP peer in the DC with it's own AS prepended 1 time.  The secondary hub with lower prio will also give the route it it's peer but prepend the AS twice.

So if both DC's have their own interconnect and routing is passed on DC-2 gear would send traffic towards DC-1 towards that hub because of the shorter AS path.  But it still can use DC-2 MX if that interconnect is down.

[Aamir]

so I understand the benefits of BGP but lets say customer has no DC interconnect, upstream switch does not support BGP and customer has same subnet across the two DC. Can we still use iBGP on overlay to choose the hub priority for the same subnet and MX will get the same subnet via OSPF from upstream switch? Or when we enable iBGP we must use eBGP also?

[PhilipDAth]

>lets say customer has no DC interconnect, upstream switch does not support BGP and customer has same subnet across the two DC.

You wouldn't use BGP.  You could connect each hub via a unique stub network to the network core.  You would then add a static route to the subnet that existed at both sites to both MX via the core switch at each site.

[Aamir]

thanks, then how would the spoke know which hub is primary, can I still use the hub priority without iBGP on overlay?

[PhilipDAth]

When you configure a spoke you have to specify the hub priority order.

https://documentation.meraki.com/MX/Site-to-site_VPN/Site-to-site_VPN_Settings#Configuring_multiple_...

[GIdenJoe]

Yep, the spokes choose their own priority by configuration of yourself.
And the upstream DC devices just need a route back to your spokes pointing to your MX in the DC.

The BGP solution is more if you want to vMotion between DC's and have the ability to go through the interconnect if needed or directly down to the spokes if needed.  Or if those routes go beyond the datacenter into the ISP where traffic can be routed through either DC depending on the spoke to reach.

[Aamir]

is there a way on MX on spoke to see BGP path similar to show ip bgp on a router?

[PhilipDAth]

Security & SD-WAN/Route Table

[GIdenJoe]

The routing table view will only show the RIB.
So only the selected path will be in there, not the alternate paths.
It will however display internal BGP as route source.

So there is no show ip bgp/ show bgp ipv4 unicast equivalent.

And there will be some weird IP address as hop between hub and spoke beginning with 6.something.  I believe this subnet is created inside the AutoVPN tunnels to allow BGP messages to flow between all BGP speakers.

[PhilipDAth]

>The routing table view will only show the RIB.
>So only the selected path will be in there, not the alternate paths.

This is an example from a dashboard that has two routes to the same network.  It displays the one it is using as "Active".  It then just lists the others.

[GIdenJoe]

I got it mixed up:

It's when your route comes from another source than BGP and is preferred over the BGP route it only displays the preferred route source (like VPN peer).

[Aamir]

I thought Meraki has now introduced per device licensing?