In MX HA Warm Spare design, does anyone know why Meraki doesn't recommend directly connect two MXs?

[Oucean]

Hello Everyone

Just got some mental block about MX HA Warm Spare design.

In Meraki Document, we all know that the recommanded diagram from Meraki for two MX HA and two switch is as below.

And below one seems not recommended by Meraki. The reason in this document says "there is an increased potential for a spanning-tree loop". Although this is for one switch design, just still it is not recommended by Meraki.

But I'm confused... In my opinion, the less cables, the easier to manage. If something goes wrong, there will be less "road blocks" while troubleshooting. So...Does anyone know if there is any reason Meraki doesn't suggest this way?

I know we must connect both MXs. If not, the two MXs will become both active if the cable between switches is down or if either switch is done. And I also know MX will not run STP but they will forward BPDU, so I believe STP can work properly to avoid layer 2 loop.

So I'm thinking, maybe connecting both MXs with the same VLAN configuration and ultilize STP on the two switches to avoid layer loop will be a better choice? May I know your opinion? Thank you very much.

[PhilipDAth]

The MX series is not spanning tree aware.  It can not speak spanning tree protocol.  All it will do is repeat any received spanning tree packet out of the other ports.

As a result, you can get some non-optimal traffic flows at layer 2, and [if using multiple VLANs] ports getting incorrectly blocked on switches resulting in an outage.

[tellis002]

Yup.  I'm in talk with Meraki now due to using a trunk, controlling vlan's and magically you get loops sometimes/outage.  As a network engineer, i cringe at any design based on STP working correctly.  

[Oucean]

Hi Tellis002

I used to work with multiple layer-2 networks so I'm always thinking STP should work fine...

Could you please update in this thread when you get a feedback from Meraki? I'm also very curious about what will cause this kind of issue. Thanks a lot.

[PhilipDAth]

ps.  I mostly use the second picture without the cable between the MXs, so it is 100% loop-free.  This is super reliable.

[Oucean]

Hello PhilipDAth

Thanks for your reply.

And yes, MX series is not spanning tree aware. It will just forward spanning tree packet (BPDU) within the broadcast domain that the packet was received. So I believe I can say from STP perspective, the diagram is like below.

The STP will still work to block one port to avoid layer 2 loop. I think this is good. If there is some non-optimal traffic flows at layer 2, we can manually configure STP attributes to manually select which port to block, right?

And if using multiple VLANs, we can see the link between two MXs to allow the same VLANs (Also the same Native VLAN). Seems...I don't think this will result in an outage?

But let's see your "ps". I think this will cause issue. Just image if one of the switch is down, or the link between the switch is down. Then both MX can not receive VRRP packets from each other, right? So both MX will be the active one. I think this will high possiblely result in an outage.

[PhilipDAth]

>we can manually configure STP attributes to manually select which port to block

Usually no.  It's difficult.

>So both MX will be the active one

Correct.  As long as you are not using virtual IP, outbound traffic will continue to work, unaffected.  AutoVPN will break.  Some inbound NAT cases will break.

My experience is you will experience more outages from spanning tree failures than you will from switch failures.

[GIdenJoe · Accepted solution]

You cannot manually select which ports STP blocks.
You can only influence STP.
If your left switch is the STP root and you want traffic to flow from the left switch to the right switch directly you have to make sure the link between the switches has a lower cost (using aggregated link which ultimately has higher bw (lower cost) than the link going up to the MX'es.  Or if the bw is equal than you need to make sure the link between the switches is a lower portnumber since sending port ID is the tiebreaker.
On MS switches you cannot set the port priority.

[Oucean]

Hello Glden Joe

Thanks for your reply.

Oh...I'm very sorry. I forgot to highlight this part. Yes, I know Meraki MS switch can't manually adjust some attributes. So actually in this design, the two switches are not Meraki switches. I can't find other switch's icon so I used MS switches to draw this diagram. Sorry, It's my fault.

So in this design, I'm planning to use port priority on these two switches to manually select preferred port. And I believe this work fine.

But still, what I'm keeping thinking is, why Meraki doesn't suggest this design (With one cable directly connect two MXs with proper VLAN configuration). May I ask if you know other potential issues for this design? Exclude the issue may caused by Meraki MS switches. Thanks a lot.

[TCAP]

what would happen if you lost switch 1 and the HA did not fail over to mx2?  you would be dead on both switches then?  

[PhilipDAth]

If you lost switch 1 both MX's become become master (they would assume the other has died).  They would NAT outbound traffic to their WAN interface IP address, and outbound web browsing would continue without issue.