VPLS from an End-User Network Admin Perspective

[Twitch]

Hey gang - my company ordered a VPLS circuit a while back and two of our site's circuits are completed and ready for us to connect to our network. The challenge is that I can't find any information about how to configure VPLS from a customer perspective - everything I find is related to service provider edge and within the service provider network configuration. I can't find anything from our (end-user, if you will) perspective. 

I called tech support for our VPLS provider and that was zero help - they basically told us to hire a consultant. I myself do not have any experience configuring MPLS or VPLS, both technologies have not crossed paths with me during my career, but I don't want to look like a total incompetent who is unable to get this circuit working. 

So, can anyone point me to a Meraki configuration guide or document that could help me bring this circuit to an up/up state?

It is my understanding that VPLS is a layer 2 technology, so, from the demarc device I would assume that I would connect from the user (hand-off) port to a switchport on one of my MDF switches.

Is this assumption correct?

If so, is there any configuration required on the switchport, considering that VPLS is MAC based and learns the network paths via MAC addresses, if I am understanding that correctly. (I do know that if I plug a Cat 6 cable into the demarc device and one of my switches the connection comes up, but what's required behind the scenes remains a mystery...)

Are IP addresses required? If so, I'm going to assume I need to tie the MX into this configuration. (But, if it's a layer 2 technology, why are IP addresses required?)

All this new-fangled technology... I'm hoping that someone will have the Magic VPLS Configuration Guide out there, or some great advice to help an old guy out. I sure would appreciate it. 

Thanks!!

Twitch

[cmr]

@Twitch VPLS is a cable 

We have two VPLS WANs and they are so much better and easier to manage than MPLS as you control everything and their latency is lower.

What we do (as we don't want a broadcast domain covering the whole country) is put an MX pair at each site and connect the VPLS circuits to the WAN ports.

However at our main DC the VPLS circuits are terminated in routed switch ports (Cisco 3850s) and the MX pair is in single arm concentrator mode.  This allows a third WAN (MPLS) to also be part of the MX SD-WAN solution.

You choose the IP addresses you want to use, you choose the subnet size, you choose where you want to use MX WAN ports and where you want to use switch ports.

If you do want you can span a single VLAN or even multiple VLANs in a trunk across multiple sites but be very careful doing this, only do it with low latency and high bandwidth links.

[Bruce]

Remember that the MX devices need to be able to reach the internet from their WAN ports to register to the Dashboard. @cmr’s approach works well, where you can use private IP addressing on the VPLS with the Dashboard traffic coming back to a headed data centre and then being NATed out to the internet by a firewall (or similar).

[Twitch]

Thanks @Bruce @cmr.

Would it be correct to say, then, at least from a high-level view, that I can assign each site a new private IP range assigned to the MX as the VPLS interface, place that interface in its own VLAN, route the private range with OSPF, and as long as each site knows a route to the internal networks located at each site, the VPLS will simply pass the traffic inter-site, much like frame relay did "back in the day?"

We don't want to change the IP range assigned to the internal networks, hence my desire to add another subnet just for the VPLS and rely on INTER-VLAN routing to pass the traffic. 

The remote sites will come through our office for Internet access, which is a 1 Gbps link. The VPLS links right now will be 100 Mbps to start with. 

At our office, I can either extend the demarc directly to our MX100, using one of the LAN ports (port 8, for example), bypassing the switches, or I can connect to a switch first, then connect from the switch to the MX. Are there advantages for one over the other? I'd prefer to go straight to the MX.

The WAN port on our MX will stay as it is because it's our (and the remote sites once the VPLS is in running) Internet connection, hence my desire to use an MX LAN port. 

At the remote sites, based on what you guys have said, I can use the WAN port to connect the VPLS since all of their traffic will come back through our office anyway.

Does that setup make sense?

I have a clearer picture in my head now. At least I think I do... 

Thanks again! 

Twitch

[cmr]

@Twitch pretty much, except the WAN interfaces on the non DC MXs will all be in the same private VLAN and subnet. i.e. 192.168.234.1/24 for site 1, 192.167.234.11 for site 2 etc. the main site VLAN interface on the MS or MX would be 192.168.234.254 for example and would be the default gateway on the other site MX ports.

Therefore all traffic from remote sites will head to the DC if it doesnt know what else to do, then out to the internet if you let it.

The reason I'd separate the IPs by more than one is that if you have an HA pair you'll want to use VIP and have 3 IPs per site out of the 192.168.234.0/24 subnet used on the VPLS WAN.

[Twitch]

@cmr Aha! I see. So basically, I can conceptualize VPLS as a bunch of routers plugged in to a single central switch, with all of the routers having an IP in the same subnet, all sharing the same VLAN ID? That ain't so bad, then, is it? 

Great call on spacing the IPs out. We currently have one MX at each site, and I am pushing hard to get rid of that single point of failure, especially here at the DC. The network is a work in progress. 

I really, really appreciate the help and advice. Thanks!! 

Twitch

[cmr]

@Twitch yes, exactly 😎

[Twitch]

Hey @Bruce - so is it safe to say that I can use a private IP address, say 172.16.1.x /24, on the WAN ports of the remote site MXs and they will still be able to reach the Dashboard via our data center MX Internet connection, and that NAT will then translate that private IP to a public IP which gives the remote MXs Dashboard connectivity?

Reason I ask is that our MXs at all sites currently have a public IP assigned to their WAN ports with direct internet access. Once the VPLS is live, the remote MX's traffic will travel through two designated primary sites to access the Internet, and will be using private IPs instead of public IPs.

We're hoping to drop the internet service at all remote sites and only need to maintain active internet connections at our two primary sites.

Thanks for your help!

Twitch

[cmr]

@Twitch as far as I'm aware, you either need internet on the VPLS or, as we do, the DCs have switches on the VPLS directly with the MXs in single arm mode so both the internet and the VPLS are on the WAN ports of them.

[Bruce]

@Twitch yep, what you’ve said is correct, you should be able to use private IPs on the VPLS and then NAT them to public in your data centre. You just need to give some though as to how you are going to do that. Are you going to have VPN concentrators that terminate tunnels from the remote sites, and the hand-off to other firewalls (possibly MXs) that then do the NAT; or are you going to go without VPNs which means you’ll lose some visibility into the site as all traffic gets NATed at the remote MX. You could go with a no-NAT solution at the remote sites, but then that would mean a potentially higher overhead in managing your route table - and you’ll likely need support to enable a few things.