HA MX on a campus site

[Dunky]

I am installing a new Meraki LAN (MX, MS, MR, MT, MV) at a site that has multiple buildings.

The design calls for a pair of MX configured for HA.

The question is, if I have 2 switches connected by fibre (different building other side of the road) and an MX in each and split the 2 ISP connections 1 into each building - what would happen should the fibre link between the 2 be lost.  Would each MX become a primary and effectively create 2 networks?   And then what would happen when the fibre connection between the buildings is restored (I assume the Primary one would become primary again and the one over the road revert back to the spare - however whilst it was the primary it will have issued out IPs from its DHCP poll which could potentially cause issues?

And to complicate matters further, the MX participates in AutoVPN so which MX would be participating in the VPN as they would have the same subnets.

Or is is just a matter of should the fibre link do down then we manually have to shutdown the switchports to the spare MX to take that MX and switch(es) offline until the fibre link is restored.

All advise and guidance greatly received.

[alemabrahao]

If communication is completely lost, yes the two have taken over as primary.

And yes, if communication is reestablished, the primary will take over again.

I don't know how you planned your design, but having only one fiber between buildings is not a very good idea.

Remember that you will need at least 2 WAN IPs (one for each MX) 3 if you consider using a VIP.

[Dunky]

Thanks for the reply.

There will be 2 fibres LACP and each fibre will terminate into a separate switch the the stack at each end.

The ISP connection and the MX WAN ports will patch to local switches using a L2 transit VLAN.

Yes, will be using a VIP.

My concern is really around what would happen if say a digger was to dig up the fibre under the road or some other catastrophic event that took out all the fibres between the 2 buildings.

Would we end up with both MX's being a primary.  Or is Meraki clever enough to know that even though VRRP has been lost on the LAN, that the Primary MX is still up in the dashboard and hence the Spare MX will remain as spare ?

[alemabrahao]

The VRRP exchange is done via the LAN interfaces, so if communication is completely lost between both MXes, both will assume it as primary.

[Dunky]

Yeah that is my fear.  We will end up with both MX's thinking they are primary which is going to totally screw the AutoVPN and wreak havoc when the LAN connection is restored between them.

[rhbirkelund]

You could consider, in stead of running the two fibres between A and B as a pair, you could do fibre runs in different directions. A customer I have has an East-West link between two organizations, with a dark fiber, using LACP. On fiber i 1 km, the other is 10 km long in a different direction.

That way, if you have one fiber that's dug over, chances are the other isn't impacted, as it is, physically, somewhere else.

Other than that, you might want to review the Campus LAN design guide, https://documentation.meraki.com/MS/Meraki_Campus_LAN%3B_Planning%2C_Design_Guidelines_and_Best_Prac....

[Dunky]

Thanks, unfortunately we have to live with what is there as we cannot dig up the highway to get a 2nd diverse route. 

[cmr]

Could you install a wireless / microwave link between the buildings as a backup?

[Dunky]

Good shout, I will have to investigate.