MX high availability with breakout switch

[janic]

Hi all

I’ve got a topology with an ISP uplink, 2 MXs and one breakout switch. The setup works perfectly fine concerning the WAN side. But as soon as I hook up the breakout switch to the secondary MX which is in standby everything goes offline, it seems like it’s causing a loop even though it's connected to two different MXs. The following topology is exactly like the one I’m having issues with. My question is how are the MS management ports supposed to be configured? Is the STP guard needed in a topology like this or am I missing something else?

The MS management ports are currently configured like this:

The MX LAN port which connects to the MS management port is configured like this:

[alemabrahao]

This is not an ideal topology, as the MS also depends on a connection to the Meraki cloud, so you may experience an outage during failover.

In your place, I would connect the WAN directly to the MX or use another switch that was not an MS.

[janic]

Hi Alessandro, that doesn't work in our case. Our ISP is providing a failover from fiber to copper in case one line goes down. This results in 2 RJ45 ports which have to be connected to both MX's, as a result both MX can use the MS as a breakout to each ISP Device.We can't hook up MX1 and MX2 with failover if we don't use the MS.

The MS continues to work just fine even if it has no cloud connectivity, but in this topology it should always have since it can connect ot the dashboard using VLAN 1000 which is connected to both MX. 

[PhilipDAth]

>This results in 2 RJ45 ports which have to be connected to both MX's

If it is for failover; can you connect the primary circuit only to the primary MX, and the secondary circuit only to the standby MX?

Or are they using HSRP or something?

[ww]

This should work.

This is exactly what is connected currently? Or is there more connected to the mx/switch?

[janic]

On the MS Port 1-4 are only used as WAN Ports using VLAN 900. The ISP is using two devices to provide failover from fiber to copper, but thats working fine.

Port 1 -> ISP Device Fiber

Port 2 -> ISP Device Copper

Port 3 -> MX1 - WAN 1

Port 4 -> MX2 - WAN 1

And the MS is connected to each MX on Port 7, every port is configured as "Access" with VLAN1000.

MS Port 7 - > MX1 - Port 7

MS Port 8 -> MX2 - Port 8

The MS resides in a separate meraki network, otherwise it would influence the topology or cause issues. As soon as MS Port 8 -> MX2 - Port 8 is plugged in, it all goes offline.

[Ryan_Miles]

Ensure all ports are configured the same way. Meaning if trunks have all the ports be trunks with the same native and allowed VLANs. Also, on the MX don't have it configured to drop untagged traffic on the LAN port(s).

[janic]

Hi Ryan, all MS LAN and MX LAN Ports are configured as "access" ports. To further explain the setup this might help:

On the MS Port 1-4 are only used as WAN Ports using VLAN 900. The ISP is using two devices to provide failover from fiber to copper, but thats working fine.

Port 1 -> ISP Device Fiber

Port 2 -> ISP Device Copper

Port 3 -> MX1 - WAN 1

Port 4 -> MX2 - WAN 1

And the MS is connected to each MX on Port 7, every port is configured as "Access" with VLAN1000.

MS Port 7 - > MX1 - Port 7

MS Port 8 -> MX2 - Port 7

The MS resides in a separate meraki network, otherwise it would influence the topology or cause issues. As soon as MS Port 8 -> MX2 - Port 7 is plugged in, it all goes offline.

I'm seeing RSTP changes and MAC Flapping when plugging it in, it even says loop detected. But it didn't seem like STP loop guard helped.

*slight mixup with ports, obviously the connection MS Port 7&8 are connected to MX1/2 on Port 7

[Ryan_Miles]

This topology alone doesn't have a physical loop. What else is downstream of your MXs and have you removed VLAN 1000 from those links? I have this exact topology running here and no issues.

* I see you have UDLD events in your MX/MS network. I would track that issue down and figure out what's going on.

[janic]

Downstream on the MX only the MS in the separate network is using VLAN 1000 and only on port 7 on each MX:

I'm a bit puzzled why this configuration is causing a loop:

[rhbirkelund]

There's something with the screenshot of MX LAN interfaces, and the topology drawing that doesn't quite add up. The MX105 only has 10 interfaces, but your sketch indicates 12, and that you're using port 12 for the link to the breakout switch. Also the LAN interface for port 7 is access, whereas the rest are trunk.

Make sure you use the same LAN port on both MX'es and connect them to the breakout switch. And also, configure said LAN port on the MX to Access in vlan 1000.

If I read things correctly, it looks like VLAN 1000 on the breakout switch is being jumped to the native vlan (id 1), and I suppose you have some downstream switching aswell, which is probably also with vlan 1 as untagged.

Make sure that all management to the breakout switch is only connected with acces ports. No trunk.

[janic]

Please note that the drawing is only a best practice topology, our setup is slightly different but functions in the same way as the topology drawing. I could make an exact one, but since it's rather simple:

MS Port 1 -> ISP Device Fiber

MS Port 2 -> ISP Device Copper

MS Port 3 -> MX1 - WAN 1

MS Port 4 -> MX2 - WAN 1

And the MS is connected to each MX on Port 7, every port is configured as "Access" with VLAN1000.

MS Port 7 - > MX1 - LAN Port 7

MS Port 8 -> MX2 - LAN Port 7