Nexus - VPC with Meraki MX100 One Armed Concentrators

[PatrickBB]

I found another thread with a similar topic about MX-84s connecting to Nexus VPCs. 

I am looking to deploy 2x MX-100s in One Armed Concentrator HA mode for VPN terminations.  I did this in one of our data centers where our core is not using VPC and was able to get it up and running rather quickly.  However, in our other data center, the entire core is Nexus VPC.  Below is a basic diagram.

My understanding is that in One Armed Concentrator mode, the Internet 1 interface is the only one used.  It is used for Ingress and Egress traffic as well as the HA heartbeat.

Nexus switches have a Peer Link between them.  That is used for control plane messages and does not forward traffic.

My concern here is that the HA heartbeat will not be seen.  The other switches connected to this VPC core are also Nexus and also in VPC mode.  

The HA heartbeat from the secondary MX100 would have to traverse the switch that it is connected to, down to another switch that has a VPC port channel with both of these cores, then come back up the other nexus switch in order to see the primary HA MX 100.  

I have asked Meraki Support and my Sales Team if this is even possible.  The reply is that this may not be a supported topology.

As a work around, I have thought about not deploying both MX-100s in HA, but rather as 2 separate VPN hubs.  I would not use the VIP.  On the spokes I would define both hubs.  I am sure that this would work, but it is not what I consider ideal give the time to transition between hubs if there is a failure.

Has anyone deployed One Armed Concentrators in HA mode connected to Nexus switches running VPC?

[Tadpole86]

As long as there is layer 2 connectivity between the MX100 devices this will work. 

However, as you mention the peer link does not forward user traffic. So how are you expecting the heartbeat to reach the other MX? From your description, I don't believe this diagram includes all devices which doesn't help.

I have never worked with VPC technology, however, I have seen this work with VSS which is similar technology. 

This looks like a question better answered by Cisco TAC who work with Nexus, as opposed to the Meraki support team.  

[KRobert · Accepted solution]

Hello @PatrickBB,

We have this exact same setup with two Nexus 3Ks connected with a Peer-Link and we do not have any issues with HA between the two MX appliances. I have done this with both MX100s and MX250s. We utilize the HSRP setup rather than the VPC

We were having some network issues between two sites so we segmented our One-Arm concentrates in their own VLAN and that helped, but it isn't needed. I setup each MX as a one-armed concetrator and they share a /29 network with the Nexus switches. The /29 provides enough IP addresses for this to work.  

Nexus 3K Setup

1. Create a new VLAN for your HA Pair on each Nexus Switch.
2. Create a VLAN interface on each Nexus Switch.

   1. Nexus Switch 1

      1. interface VlanXXX
         description Meraki_Auto_VPN
         no shutdown
         ip address 172.16.100.5/29
         hsrp 200
         authentication md5 key-chain HSRP
         name VLANXXX
         preempt
         priority 101
         ip 172.16.100.1

   2. Nexus Switch 2
      1. interface VlanXXX
         description Meraki_Auto_VPN
         no shutdown
         ip address 172.16.100.6/29
         hsrp 200
         authentication md5 key-chain HSRP
         name VLANXXX
         priority 50
         ip 172.16.100.1
3. Setup your interface on each Nexus switches:
   1. interface Ethernet1/X
      description Meraki_HUB
      switchport access vlan XXX
      spanning-tree port type edge
4. Setup your MX HA Pair using the remaining IP addresses.
   1. Primary: 172.16.100.2
   2. Spare: 172.16.100.3
   3. VIP 172.16.100.4

This should allow your HA Pair to communicate with each other without interruption and maintain the Heartbeat you would miss using the VPC.

[PatrickBB]

I was able to implement this and it works.  I did not need to create a separate VLAN.  We had an existing VLAN that had an SVI on the upstream VPC Nexus switches.  That VLAN also has the interfaces used by the routers that I am eBGP peering with along with the interfaces to the firewall where Internet traffic flows.

While I was not able to get a validated design from either Cisco or Meraki support, I do have it working in our environment.

[KRobert]

Glad to hear it is working for you!

[KRobert]

Hi @PatrickBB,

Did this help you? 

[PatrickBB]

I will find out tomorrow after we get it racked and powered on in our data center.

[EBarratt]

I am running into a similar issue. In the end we want to have a HA pair, but this is currently a POC, so we just need one up and running. Anytime the port is turned on that's connected to the Meraki, it goes into a Local VLAN Suspended State. And when I check on the 9k1 the vpc peer info doesnt populate. I just got off the phone with Meraki they don't have any info or documentation for a supported architecture that we are trying to run. 

[PatrickBB]

That was the issue I ran into when contacting Meraki support and my Meraki account rep.  There was no documentation for connecting Meraki to Nexus running vPC.  

Since this thread has been solved, I would recommend starting a different thread as I believe the solution your your issue will be different.

[Aaron_Wilson]

I'm curious, why did you just not just directly connect the two MXs for the heartbeat using its own vlan? That is an acceptable design from Meraki.

[PatrickBB]

Aaron,

Below is a link to Meraki's documentation for MX Warm Spare - High Availability.

https://documentation.meraki.com/MX/Deployment_Guides/MX_Warm_Spare_-_High_Availability_Pair

The document contains 2 different diagrams, neither of which show a direct link between the MXs.  Also, my configuration uses the MXs in One Armed Concentrator mode for VPN termination.  That only uses the uplinks for connectivity, also noted in that document.

Also, in the following community post the accepted solution notes that when having a connection between the MXs, it can force traffic through the spare as spanning tree my block the port to the primary MX.

https://community.meraki.com/t5/Security-SD-WAN/How-to-cable-MX-amp-MS-for-HA/td-p/22765

[Aaron_Wilson]

Since the Meraki's are going to separate switches I'd be worried about the trunk being lost and the Meraki's going dual active.

While I cannot find my Meraki document I referenced years ago, Aaron's article still exists: https://www.willette.works/mx-warm-spare/

However, it does look like Meraki treats warm spare slightly different between NAT and concentrator modes:

"In order to receive heartbeats in a one-armed concentrator configuration, both VPN concentrator MXs should have uplinks on the same subnet within the datacenter.

For Routed mode configurations, both concentrators must be able to communicate using the LAN ports."

If you make the directly connected ports a dedicated /30 subnet you shouldn't have to worry about loops or anything. I have mine setup this way....but I also run NAT mode 🙂

[GIdenJoe]

In one armed concentrator mode you only have one uplink, so you can never experience a dual active issue if a VPC peer switch would fail because both up and downlinks would be gone on a single appliance.

However VPC is in itself cannot be used in this case because the MX does not support port-channels.

Individual links to both MX'es is okay though.