Dual ISPs, ASR non Meraki Peer, MX HA Warm Spare

[Netwow]

We have been working on a deployment of 2 MS210-8 as ISP passthrough switches to a pair of MXs . See Picture. The issue we face is that the MXs sometimes (not all the time only when there is a VPN tunnel change on the ASRs) go into Active Active mode. We are load balancing across both ISPs. The fix offered to us  is to connect the 2 MXs together. By my understanding this already achieve via ports 3 and 4 of the MXs (see the port descriptions). The ASRs see traffic across the IPsec tunnel from both ISPs but due to the ASR tunnels being in active/passive WAN1 being primary, I believe traffic being load balanced onto WAN2 is being ignored as that tunnel is idle. I have seen this design a lot lately due to the limitations of the carriers to provide more than 1 port. 

Anyone see issues with this topology without the physical connection that was suggested?

[jdsilva]

I have never tried to do what you're doing here, so I can't offer any words of wisdom based on experience. But I wonder, are you configuring the tunnel end point on the ASR to be the VIPs? Do you get better results if you configure the tunnel end points to be the IPs of the WAN ports? I.e. configure two tunnels (or four with both ISPs) on the ASR that go to each MX. 

[Netwow]

Using VIPs on both sides. The tunnels are stable for a couple of weeks and then Then one of the tunnels will fail. When this happens the MXs go into Active/Active mode and we see asymmetrical routing. Have to bounce the tunnels on the ASR side to stabilize. 

[NolanHerring]

Shouldn't there be a connection between the switches (or is it just not being displayed in the image?)

[PhilipDAth]

You have layer 2 loops, especially with the trunk ports.  I bet you are experiencing spanning tree blocking ports.

Are you able to move the LAN ports onto different switches and make the design loop free at layer 2?  I bet the problem will go away if you do this.

[ByronC]

The dual master state occurs when the firewalls are unable to communicate effectively with each other via VRRP. This communication is done on the LAN side of the firewalls. Based on the recommended topologies and best practices provided in Meraki documentation compared to your diagram, you may be running into issues with STP convergence and getting some blocked ports.

https://documentation.meraki.com/MX/Deployment_Guides/MX_Warm_Spare_-_High_Availability_Pair#Require...

[Netwow]

We have a non-meraki vpn peer (ASR) that we have IPSEC tunnels to . Because we are using the uplink addresses for VRRP heartbeats the ASR sees each MX WAN1 uplink (primary and spare) as a tunnel and keeps them both active. The MX is configured to only use the secondary tunnel if the primary fails. We have a TAC case with Cisco now. I know they will say "Use the VIP" however I do not feel this is a viable solution. We did install a cable between the 2 MXs and we do not have an issue with ACtive/Active anymore but the ASR does. Very interesting.