How to turn MS120 into WAN breakout for 2 ISPs and 2 MX250s in HA?

[CaseyFowler]

I'm far from a network engineer but have been tasked with transitioning our SPOF config into a HA.

Our MSP recommended we get a unmanaged L2 switch to sit between each ISP's router at our edge. They recommended we order a Cisco Cat 1000, but they are on backorder for 6 months

I've seen many state that an MS120 can be configured to support this(which we have many spares of), but I can't find specifically what the config looks like. I asked the question elsewhere and got this recommendation:

Set a vlan ID for each ISP(999/998) and just tag 3 ports per ISP. This will create the path the broadcasts can talk over and you don't need to buy anything new.

• ISP1>Switch(VLAN999)>MX(01&02) internet 1 ports.
• ISPb>Switch(VLAN998)>MX(01&02) internet 2 ports

I'm working my way through what each port should look like and I'm realizing I'm a bit out of my depth. Wold the MS120 not doing VRRP cause an issue here?

Here's an image of the layout, and what I'm assuming port 1 will look like for each breakout.

If this is correct, would the same port config be used for ports 2&3? What about port 4? I'm assuming if 1-3 are correct, then the only difference with port 4 is adding 999, 998 to the VLAN.

[alemabrahao]

Check the recommended topologies:  https://documentation.meraki.com/MX/Deployment_Guides/MX_Warm_Spare_-_High_Availability_Pair#Recomme...

[rhbirkelund]

I've done this a couple of times, and while it's not really the most optimal way of doing it, it's certainly a possibility.

There are some caveats with using Meraki Switches as WAN breakout switches.

One aspect that you need to have in mind is that Meraki needs to have an IP address on it's Management. And that's whether you're using it on the LAN or in a WAN breakout setup. So you need a connecting from the WAN switch and back to the LAN side of MX, for it to obtain an address for it's Management.

In terms of when you configure the switches, just remember, that it is imperative that the LAN side VLANs are not mixed with the WAN side i.e. do not trunk the WAN side VLANs into your LAN side.

Another issue is that fact that during maintenance windows where you perform firmware upgrades, you'll experience that your entire Meraki organization will go offline at least twice; once during MX upgrade, and second during MS upgrades. The WAN breakout switches will upgrade at the same time as the rest for your switching infrastructure, unless you define upgrade groups for staged upgrades.

And in the end, you can never guarantee that something doesn't go wrong during Firmware upgrades, that end up causing a Loop between you LAN side and WAN side.

The third caveat is that, your Network Clients page will also contain publicly routed addresses, and you'll end up seeing some odd statistics in the Applications details.

All this being said, as long as you are aware of what you are doing, this is certainly a possible setup. Like I said, I've done it a couple times, but mainly out of necessity, rather than opportunity.

But these are just my two cents.. 🙂

[CaseyFowler]

The third caveat is that, your Network Clients page will also contain publicly routed addresses, and you'll end up seeing some odd statistics in the Applications details. Meraki Switches as WAN breakout switches.

This is a great point that I did not consider. Thanks for all the info! Would placing the switches into a different dashboard network help with this? 

I spoke with a tech with Meraki and they said it's a fairly common config. He did recommend to not to connect the 120's together via port 4, but to instead link them to a LAN port of the MX so that they can communicate with the cloud.

[rhbirkelund]

You can definitely break them out into a network of their own, but I'm not sure if that will make you clear of the public addresses in the Clients page etc. Bit I'd love to here the result.

But still the other point stand, although you can better schedule the WAN switch firmware upgrade, in the sense that it won't happen at the same time as your downstream LAN.

[Ryan_Miles · Accepted solution]

@CaseyFowler I put together a deck on this topic awhile back. Might be of help to you.

https://docs.google.com/presentation/d/1xsb8imtUFjN13so86kIZ04IR9f6WEKdbpUrYVON64Zg/edit?usp=sharing 

[rhbirkelund]

@Ryan_Milesout of curiousity, what are your thoughts on models as WAN Breakout switch? I usually see customers opting for the small 8-ports MS120 switch, but seeing as it will be handling "raw" internet traffic? A customer I have, which has just been through this maneuver used an MS410.

[Ryan_Miles]

Same. I often see MS1xx & 2xx series used for breakout. But, depending on physical handoffs, performance, or redundant PS and fans I could see what someone might go higher in the product family.

[leewalhovd]

The routing for the ISP and handoff can play a role as well. There are some ISPs that will do a transit IP and a block of public IPs behind that require at a minimum a static route in front of a warm spare MX setup. 

[CaseyFowler]

This is awesome and a huge help.

The only other bit that I'm not 100% on, has to do with the uplinks on the MX. Support basically said I just need to make sure to change from the default of VLAN tagging off and change it to use VLAN tagging with the corresponding VLAN from the MS120 it's connecting to.

Basically, does this make sense based on my original image?

[Ryan_Miles]

The MX doesn't need to have any special VLAN configs. The switches hand off a L2 VLAN. No tagging comes into play.

[bulkjunooni]

Hi Ryan, this is awesome.

Curious if this will cover any redundancy as well??? in case one meraki fails or one ISP goes down?

[cmr]

We use small 5 port Cisco unmanaged Gigabit switches for the ISP splitting.  I prefer not to have managed (and therefore more hackable) switches outside of the perimeter.  They are cheap as chips, perform perfectly well and have been very reliable.  They are also pretty much always in stock!

[CaseyFowler]

---

@cmr wrote:

I prefer not to have managed (and therefore more hackable) switches outside of the perimeter.  

---

Great point that I wasn't really considering.

Do you have a particular model you've had good luck with? MSP had us looking at a catalyst 1000, which struck me as overkill but who am I to say...

I'm seeing a few like the CBS110 for $100. Would this be good? 

[cmr]

We use the SG110D-05 and I think the CBS110-5T-D is the direct replacement.  We've had very good service out of them and as they are so cheap, we always have a spare or two hanging around.

[Ryan_Miles]

In the Meraki breakout switch design the management tunnel between the switch and dashboard is going through the MX appliance. I'd argue that's better than a simple switch sitting outside a firewall that might have telnet, SSH, or a web page enabled with less than adequate security config in place.