Network reconfig with Comcast EDI/ENS circuits

[MikeSBS]

Hey all, looking for some guidance on configuring our updated network setup. Here’s what we’re working with:

Current Site Layout
Site A -  MX95 with 2 WAN connections, network 10.40.1.0/24 
Site B (new site) - MX95 with 0 WAN connections, network 10.50.1.0/24 
Site C - MX75 with 1 WAN connection, network 10.30.1.0/24 
Site D - MX75 with 1 WAN connection, network 10.20.1.0/24 

All sites have Meraki Layer 2 switches. Sites A, C, and D currently use AutoVPN to share resources hosted at Site A. Site A will be dropping one WAN connection in lieu of the Comcast EDI.

Upcoming Changes
Comcast EDI circuits are being installed at Site A and Site B. All sites will be connected via Comcast ENS circuits. WAN connections at Site C and D will be removed — ENS will be the only uplink. Goal is to configure Site A and Site B in a failover setup to provide internet to all sites in case of an outage at either location.

Questions & Considerations
I found Meraki’s doc on MPLS with AutoVPN as backup, but unsure if it applies since we’re not using MPLS. I’ve read that the easiest method is to remove the MX75s and plug ENS directly into the L2 switches to “extend” the network — but that’s not an option for us, as MGMT want to use the MX hardware they purchased for the branch offices...

I’ve browsed Reddit, Spiceworks, and Meraki forums, but haven’t found a setup quite like ours. I did see a few threads that spoke of using Static Routes and such, but I'm just not sure. 

I'll admit, I am new to Meraki and Comcast ENS, as my team usually deploys Watchguards and standalone internet to each site.

Any advice or configuration tips would be greatly appreciated. Thanks!

[Mloraditch]

The meraki doc on auto vpn with MPLS would work here. MPLS is more used as a generic term for any private WAN. Comcast ENS can function (as far as Meraki is concerned) in the exact same way as MPLS would.

I'm quite curious on the reasoning on dumping Internet for ENS. That is the opposite direction most companies I deal with are going. ENS is usually more expensive than edi at the same speeds because it can require taxes that internet circuits do not. You also are forced to use comcast even if the area you are in isn't a native comcast territory.

However if I was forced to use your setup, I would not be leaving the MXs in the site with no internet.  I would do as you thought. Regardless you are going to have difficulty with failover, at least in an automatic way.

I wouldn't really want to do it, but I can imagine a few hacky ways to leave the MXs via creating two vlans on your ENS, one terminating at Site A and one at B and then presenting those as ISP 1/2 to the MXs. There's a bit more work to it, but it would function and allow failover.

Regardless, outside of crazy security or very specific bandwidth needs I'd see if you can reconsider using ENS. Comcast generally doesn't care how they get their money so you can likely change the products w/o incurring pricing changes.

[MikeSBS]

Thank you for the advice!

This is for an Architectural firm and they are sending large files over the WAN via VPN right now, so the "higher powers" that do not understand technology were in charge of finding a solution to improve traffic speeds between our sites. My suggestion was to get each site Gig fiber and call it a day. The talked to Comcast and was sold ENS.

I didnt buy it -- I'm just the guy left to make it all work now.

If we didnt care about failover would something like this work:

Site A (EDI) > Site C (with no MX) (extension of Site A network)

Site B (EDI) > Site D (with no MX) (extension of Site B network)

Site A and Site B connected via AutoVPN.

Once this is configured, I will follow the MPLS with AutoVPN doc -- or would this work better (ignoring the fact that our Site C/D wont have internet) Configuring Site-to-site VPN over MPLS - Cisco Meraki Documentation

[Mloraditch]

If you didn't care about failover what you have listed would definitely work. If you did that the AutoVPN over MPLS would probably be irrelevant. Or are you trying to also have A and B failover to each other?

[MikeSBS]

That would be a great option to have A/B failover to each other, as we're discussing adding redundant virtual hosts at Site B.

[Mloraditch]

It's theoretically possible, but a bit of a complex setup. I'm spitballing this in my head, but I think it could work with the ENS being VLAN'd similarly to my earlier idea as you still want direct connectivity C/D behind the firewall so you'd have the ENS into a switch port, and use one vlan for internal routing and use the  other to create an internet handoff that you could use on WAN 2 at A and B. 

This is a somewhat high level thought to help guide you down a possible path. Others may have different ideas and you may want to engage your partner for help.

[PhilipDAth]

I am not familiar with Comcast EDI/ENS.  Is this a layer 2 or 3 service?

In either case, I will refer to it as a WAN.

For this kind of configuration, you connect the WAN to the Internet.  You could do this by doing that at one location, or by buying such a service from Comcast on the EDI directly.

You'll use this topology, except you don't need an Internet connection for the branch.

https://documentation.meraki.com/MX/Site-to-site_VPN/Configuring_Site-to-site_VPN_over_MPLS

> Goal is to configure Site A and Site B in a failover setup to provide internet to all sites in case of an outage at either location.

This would only be possible if sites A and B were adjacent in layer 2.

[Mloraditch]

Comcast ENS is a mesh layer 2 service. Comcast EDI is their regular internet service.

[PhilipDAth]

In that case, if you want HA Internet, I would use an additional pair of MX at site A and B to provide the Internet access.  They would be a warm spare pair.  The ENS would plug into those.

You would then plug your current branch MXs in behind those.