Nexus 9Ks - VPC with Meraki MX250 One Armed Concentrators

[EBarratt]

I recently just had this project handed to because I have deployed Meraki networks at past organizations. Above is the current architecture and I have been fighting some issues all day today. Anytime the port is turned on that's connected to the Meraki, it goes into a Local VLAN Suspended State on the 9k's. And when I check on 9k1 the vpc peer info doesnt populate. I just got off the phone with Meraki they don't have any info or documentation for a supported architecture that we are trying to run. I have also read in other posts/ across the web people have had similar issues with Meraki and vPCs.

[PhilipDAth]

Is the port on FEX108 an access port?  If you plug a notebook into the same port does it suspend?

Are you sure the port on the FEX108 is not part of a port channel by accident?

[EBarratt]

Yes the port on FEX108 is an access port. I have a UCS living on that same FEX utilizing the same data vlan, and it has no issues when plugged into the port. 

[PatrickBB]

Is the Uplink port on the MX250 configured with the same VLAN as the access port?

If you have a VLAN specified on the Uplink port of the MX250, have you tried with leaving the VLAN field blank?

Edit:

I double checked my configuration which is working.  The switch port on the Nexus side is an access port.

interface Ethernet1/XX
switchport
switchport access vlan XX
no shutdown

The uplink port on my MX100 is not configured with a VLAN tag.

I have ran into issues with this before whether to VLAN tag or not.  

[PhilipDAth]

What kind of 10Gbe connection is it?  TwinAx?  10GBaseSR?

[EBarratt]

It's actually not 10Gb now. We didnt have an open SFP port on the FEX so I have a 1Gb GBIC in the Meraki MX, connected via ethernet to the FEX.

[Bruce]

What VLAN(s) are on the port from the FEX towards the MX250, and is any of them a newly configured VLAN for the MX connectivity? What is the exact error message on the Nexus 9k switches - normally that gives a good indication of why the port/VLAN has been suspended.

[GIdenJoe]

I think Cisco TAC will be the correct place to troubleshoot since the Meraki cannot have any influence on the access port on the FEX.

Do tell us what the solution was 🙂

[EBarratt · Accepted solution]

Update:

Thank you all for the valuable input.

The issue was fixed by adding the 'access port' to both 9k's. I was doing the initial config on only 9k1 and that's why the VLAN was going into a suspended state. I sometimes forget we dual homing our 9k's.

Once that was complete and the necessary rules were added to our upstream NGFW's. We were able to see the outbound traffic and the return traffic, but the MX250 would never connect to the cloud. Since we are hoping to have this solution live pasts its POC. We created a new VLAN and subnet for it, and once we got our NAT squared away everything from the MX250 perspective is up and running. 

[GIdenJoe]

Oh I see, because your FEX is controlled by two parent switches you needed to match the config on both parent switches to get it running?

[PatrickBB]

The thing to remember about a FEX is that it works just like a module in a chassis switch.  When connecting a FEX, the configuration is done on the parent switch.  In your case, you have a FEX connected to 2x 9ks so both 9ks are acting as the parent.  Both 9ks have to have the same identical configuration for the FEX.

The below link discusses configuration sync and Active/Active FEX topologies.  It is specifically for the N5k, but the theory and design should be similar for N9k.

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/operations/n5k_config_sync_op...

Config-sync is how you can synchronize the FEX configuration between the 2 9ks as long as they are running as a vPC pair.