Does Meraki MX Supported LAN VRRP?

[OPASNET]

Hello everybody,

Does Meraki MX Supported LAN VRRP?
Will it be configured with Warm Spare and LAN VRRP if supported?
We want to build the following network configuration.
If you can, please let me know how to set it up.

[AjitKumar]

Hi,

Certainly.

MX uses the VRRP, for sharing uplink health and connectivity status information between appliances.

VRRP heartbeats are sent across the LAN interfaces on each VLAN every second.

If no VRRP keepalives are heard by the secondary MX on any VLAN after three seconds, the dead timer will expire triggering a failover event.

The concept is explained neatly at the following Urls.

https://documentation.meraki.com/MX/Networks_and_Routing/NAT_HA_Failover_Behavior

[OPASNET]

Thank you for reply.

So, do both MXs use only one IP for the VLAN Interface?

[ww · Accepted solution]

yes. on the lan side with warm spare you can configure/use 1 IP only.

and do not connect both mx directly . let vrrp make use of your lan switches 

[OPASNET]

Thank you so much 🙂

[MerakiDDSDWAN]

Hi, 

I read your case and I have the same question but in another design. If I have two or More physical LAN interface and one of them fail. The vrrp of this vlan will stop to work but I had a issue that Warm Spare MX not change the operation of the Passive to Active and the communication of this vlan stop. Anyone had this situation?

[Bruce]

The MX failover is for an entire MX, not individual VLAN interfaces. But the MX will send VRRP messages on all configured VLANs (it does not send them on WAN interfaces though - failure of a WAN is detected through other means). Thus, say you have 3 VLANs configured, and the VRRP messages fail on one (e.g. cable fault, intermediate switch fault, or a port fault) there will be no failover since the VRRP messages a still being received via other VLANs.

In this regard you need to be careful when planning your failover scenario to ensure you don’t end up being ‘half failed’, and understand the failure scenarios. If you’ve multiple ports from an MX to physical infrastructure you also need to ensure you fully understand the spanning-tree implications.

[RW2]

In risk of Necro posting on this thread, I need clarification of something.

What I am unclear of is if it is possible to use a configuration like below with a singe connection to each MX without using STP

This is a very common HA carrier solution with cisco and other vendors which I am told is not supported by Meraki

To Clarify, Meraki rep and other "Meraki specialists" are claiming that for this design to work there needs to be a full mesh and STP enabled between R1,R2,S1,S2 (The Meraki MX would be in place of the R1 and R2 in this diagram)
So instead of the diagram above with single link between each R1 and R2 there would need to be links from R1 to S1 and S2 and from R2 to S1 and S2 .

I don't understand why in Meraki's VRRP implementation there needs to be a full mesh on the lan side?

In Normal VRRP/HSRP design the heartbeat takes place on the configured subnet, if VRRP heartbeat fails then the backup will become master.

In testing we found this happened for VRRP (the gateway floated over correctly) but the Primary MX still stays active on the WAN side and return traffic from WAN side destined for LAN side arrived on the original primary which no longer "owned" the VRRP gateway.

This immediately seemed like a mis-configured implementation to me but after many meetings and being linked to some articles, it seems that unless the MX is completely dead, the Primary and Warm spare must always be able to heartbeat on VRRP to function correctly.

This goes against my core understanding of VRRP, as I said above I was expecting the MX to be smart enough to fail over the entire MX if a VRRP heartbeat is missed.

Sources

https://documentation.meraki.com/MX/Networks_and_Routing/Routed_HA_Failover_Behavior

https://documentation.meraki.com/MX/Deployment_Guides/MX_Warm_Spare_-_High_Availability_Pair

I would really really really like to talk to someone who understands this mechanic properly as it seems very difficult to find someone who does and it seems like previously meraki used to advise to link the MXs by a dedicated heartbeat cable but that is now been retracted from supported configuration.

[Bruce]

@RW2, you’re correct in that the design you show will work fine if you have a complete MX failure. What that design doesn’t cover for is the failure of a link between the MX and the corresponding switch, or the failure of an entire switch. In any of these cases, as you correctly state, you lose the heartbeat between the switches, which means both MXs believe the other MX is dead and claim the ‘active’ state (and the associated IP addresses). Depending on the WAN side configuration, and whether you are using a VIP and SD-WAN, you may also see impacts with routing to the site itself.

This is the reason for the recommendation to mesh connect the MX and switches and use STP - it protects against the failure scenarios of a switch failing, or a link to a switch failing, and both MXs becoming active.

Its worth noting that the WAN side on the MX does not run VRRP, each MX has its own IP address, and if the MXs share a broadcast domain then they can also have a shared VIP address which is owned by the active MX. However, they work just as well without the shared VIP (and sometimes better). Remember that the standby MX never passes traffic, it only connects to the cloud for telemetry (hence why it still needs an IP address on the WAN port).

[cmr]

@RW2 if you have Cisco IOS switches in a stack then you should cross connect as has been suggested to you and this will mitigate switch or cable failure causing the MXs to go into dual master mode. 

If you have Meraki switches in a stack then you should single connect as otherwise when you reboot the stack, both connections to each MX will come up unblocking leading to failure.  You could set STP loop protection on those ports to avoid this, but will need your management to use a different uplink in order to do that.

I'm not sure with separate non stacked switches, but I'd bet the same is true.

And if you want to know how I know, the answer is being bitten more than once..!

[RW2]

Thanks very much Bruce for your explanation

I came to realise this myself upon thinking it over in my head. It is not exactly a typical VRRP implementation and it was confusing to see Meraki used to advise upon a dedicated heartbeat connection between MX to negate the requirement for a STP mesh. Seems for whatever reason this is no longer supported.

[Bruce]

@RW2, I actually believe that Meraki stating that the dedicated heartbeat link was no longer supported was a poor decision. I believe there are definitely situations where that direct MX to direct MX heartbeat link actually makes sense - the exact scenario you have being one of them (i.e. two switches, two MXs, one link from one MS to one MX, and then the same between the other MS and MX, and a link between the MS).

I don't know the exact reasoning for the recommendation but I'm guessing it stems from people implementing a dedicated heartbeat link with a dedicated VLAN - which doesn't work. The MX sends VRRP heartbeats on all VLAN interfaces and so long as at least one VLAN manages to pass the heartbeat from the active to the passive MX then the passive will stay passive (there is no per-VLAN failover). If you have a dedicated VLAN on the heartbeat link then this will keep the passive MX as the passive even if a switch downstream of the active MX has failed and its carrying all the other VLANs.

From what I can see, so long as all the links between the MS and MX, the two MS, and the two MXs are all trunks and carry the same VLANs then it should work - obviously consideration needs to be given to what @cmr said re. STP/BPDUs and convergence (although I'm not sure that ports configured as trunks immediately go into forwarding state, but never investigated this so not entirely sure). You'd also need to check how the STP is converging and which path traffic is taking (you really don't want the traffic between the switches going via the MXs), there's no real way to influence this other than switching physical ports around to get the convergence you want.

Anyway.... this is all my opinion, it still stands that Meraki don't actually support a direct MX to MX 'dedicated' heartbeat link.

[Peter-Loyen]

I use a dedicated heartbeat/VRRP connection between MX nodes when the LAN network is not managed by us but by a 3rd party (i.e. the customer).

The LAN is then an untrusted network with no visibility or management capabilities.

The dedicated connection is using a separate vlan (access port).

Whatever happens on the unmanaged LAN is not taken into consideration to trigger a failover but the WAN interfaces will ALWAYS trigger a failover.

If the MAIN MX looses a WAN interface, there will be a failover to the SPARE MX.

Tested and working