VPN over a direct Metro-E connection possible?

[Devaul]

Is it possible to establish a vpn over a direct connected Metro-E connection between two MX devices without adding routers or L3 switches into the mix?

We bought Meraki devices for all of our branches because we were told it'd work and now that I'm in the thick of it.  It's looking like a no.

If we did use a L3 switch at the hub and put all of the MX interfaces on the same network with that switch as the gateway, then we could share the default route over the Metro-E to give the branches internet access but wouldn't that basically make traffic able to flow over those Metro-E connections outside of the VPN?  We cannot have that.

We were also looking at trying non-meraki Peer VPNs as a cutover strategy from our ISR routers at the branches but it's looking like that's not possible either since the WAN interface on the Meraki still needs a gateway and there isn't one without that L3 switch.

[ww]

Every wan interface needs to be able to talk to the meraki services/ internet 

You basically need something like this design

https://documentation.meraki.com/MX/Site-to-site_VPN/Configuring_Site-to-site_VPN_over_MPLS

[Devaul]

We currently only have one internet connection.  We figured that would be the backup connection to the Metro-E.  Would that internet going down also cause the Metro-E to drop it's VPN since it can't talk to the dashboard?  Effectively making our internet a single point of failure.

[cmr]

You use full tunnel between the MXs at the edges to a VPN concentrator MX in the core to the L3 switch on it's own subnet.  That way the only traffic traversing the direct link to the internet is the MX management traffic.

[Devaul]

The way I got it working in my lab environment is through a VRF on our Cat9500 core switch.  I'll eventually add a connection from the firewall into that VRF as the default gateway but right now I'm route leaking it to the default gateway on the 9500's main routing table.  Then the LAN connects back to the 9500 on a different VLAN.

It's working great until I cut the internet access.  Even though I have my 2 test MX's set to test connectivity to the gateway of the VLAN they are on, the VPN still drops after a couple minutes.

I don't get why it would drop the VPN if it's passing the connectivity test.  So even though I have Internet and Metro-E, it seems that the internet going down will cause both to go down even though metro-e connectivity is good.

I don't really understand the one arm concentrator mode.  Maybe it wouldn't be the case with that?

[cmr]

@Devaul that shouldn't happen.  Losing the control plane is supposed to leave existing VPN connections up.  I must admit, I don't think I tested it for more than a couple of minutes as we had three separate internet connections at the primary hub, so at least one was always up.

The main difference with using the MX in single ended mode for this setup is that it doesn't directly terminate any of the connections and allows you to use as many internet connections as you like, depending on what device you have at the edge.

The design we used at the hub was:

Metro-E 1 -> VLAN A on L3 switch

Metro-E 2 -> VLAN B on L3 switch

MX hub -> VLAN C on L3 switch

Internet firewalls -> VLAN D on L3 switch

The L3 switch had an interface on each VLAN and the default route was set to the devices on VLAN D.

The default route on the MX was set to the L3 switch.

[Devaul]

How are you routing the different VLANs?  Static or Dynamic?  I planned on having everything on the L3 switch on the same VLAN so there's no routing needed between the interfaces.  I'm doing Routed mode though not concentrator.

[cmr]

Static routing as the only routes needed are how to get to the internet and what is behind the VPN concentrator MX.  The remote sites are all in routed mode with 1-2 Metro-E connections and 0-1 internet connections.  Therefore everything is dual connected.

[Devaul]

Did you use a L3 switch dedicated to the Meraki traffic and nothing else on it?

[cmr]

Actually we just used the L3 core and had a VLAN for each Metro-E network along with the one for the MX (and of course all the normal data centre VLANs).