How to turn MS120 into WAN breakout for 2 ISPs and 2 MX250s in HA?

Solved
CaseyFowler
Conversationalist

How to turn MS120 into WAN breakout for 2 ISPs and 2 MX250s in HA?

I'm far from a network engineer but have been tasked with transitioning our SPOF config into a HA.

 

Our MSP recommended we get a unmanaged L2 switch to sit between each ISP's router at our edge. They recommended we order a Cisco Cat 1000, but they are on backorder for 6 months

 

 

I've seen many state that an MS120 can be configured to support this(which we have many spares of), but I can't find specifically what the config looks like. I asked the question elsewhere and got this recommendation:

Set a vlan ID for each ISP(999/998) and just tag 3 ports per ISP. This will create the path the broadcasts can talk over and you don't need to buy anything new.

  • ISP1>Switch(VLAN999)>MX(01&02) internet 1 ports.
  • ISPb>Switch(VLAN998)>MX(01&02) internet 2 ports

 

 

I'm working my way through what each port should look like and I'm realizing I'm a bit out of my depth. Wold the MS120 not doing VRRP cause an issue here?

 

Here's an image of the layout, and what I'm assuming port 1 will look like for each breakout.

If this is correct, would the same port config be used for ports 2&3? What about port 4? I'm assuming if 1-3 are correct, then the only difference with port 4 is adding 999, 998 to the VLAN.

 

 

CaseyFowler_3-1677514496985.png

 

 

 

1 Accepted Solution
Ryan_Miles
Meraki Employee
Meraki Employee

@CaseyFowler I put together a deck on this topic awhile back. Might be of help to you.

 

https://docs.google.com/presentation/d/1xsb8imtUFjN13so86kIZ04IR9f6WEKdbpUrYVON64Zg/edit?usp=sharing 

Ryan / Meraki Solutions Engineer

If you found this post helpful, please give it Kudos. If my answer solves your problem please click Accept as Solution so others can benefit from it.

View solution in original post

15 Replies 15
alemabrahao
Kind of a big deal
Kind of a big deal

Check the recommended topologies:  https://documentation.meraki.com/MX/Deployment_Guides/MX_Warm_Spare_-_High_Availability_Pair#Recomme...

I am not a Cisco Meraki employee. My suggestions are based on documentation of Meraki best practices and day-to-day experience.

Please, if this post was useful, leave your kudos and mark it as solved.
rhbirkelund
Kind of a big deal

I've done this a couple of times, and while it's not really the most optimal way of doing it, it's certainly a possibility.

There are some caveats with using Meraki Switches as WAN breakout switches.

 

One aspect that you need to have in mind is that Meraki needs to have an IP address on it's Management. And that's whether you're using it on the LAN or in a WAN breakout setup. So you need a connecting from the WAN switch and back to the LAN side of MX, for it to obtain an address for it's Management.

In terms of when you configure the switches, just remember, that it is imperative that the LAN side VLANs are not mixed with the WAN side i.e. do not trunk the WAN side VLANs into your LAN side.

 

Another issue is that fact that during maintenance windows where you perform firmware upgrades, you'll experience that your entire Meraki organization will go offline at least twice; once during MX upgrade, and second during MS upgrades. The WAN breakout switches will upgrade at the same time as the rest for your switching infrastructure, unless you define upgrade groups for staged upgrades.

And in the end, you can never guarantee that something doesn't go wrong during Firmware upgrades, that end up causing a Loop between you LAN side and WAN side.

 

The third caveat is that, your Network Clients page will also contain publicly routed addresses, and you'll end up seeing some odd statistics in the Applications details.

 

All this being said, as long as you are aware of what you are doing, this is certainly a possible setup. Like I said, I've done it a couple times, but mainly out of necessity, rather than opportunity.

 

But these are just my two cents.. 🙂

LinkedIn ::: https://blog.rhbirkelund.dk/

Like what you see? - Give a Kudo ## Did it answer your question? - Mark it as a Solution 🙂

All code examples are provided as is. Responsibility for Code execution lies solely your own.

The third caveat is that, your Network Clients page will also contain publicly routed addresses, and you'll end up seeing some odd statistics in the Applications details. Meraki Switches as WAN breakout switches.

This is a great point that I did not consider. Thanks for all the info! Would placing the switches into a different dashboard network help with this? 

CaseyFowler_0-1677523117947.png

 

 

I spoke with a tech with Meraki and they said it's a fairly common config. He did recommend to not to connect the 120's together via port 4, but to instead link them to a LAN port of the MX so that they can communicate with the cloud.

 

 

You can definitely break them out into a network of their own, but I'm not sure if that will make you clear of the public addresses in the Clients page etc. Bit I'd love to here the result.

 

But still the other point stand, although you can better schedule the WAN switch firmware upgrade, in the sense that it won't happen at the same time as your downstream LAN.

LinkedIn ::: https://blog.rhbirkelund.dk/

Like what you see? - Give a Kudo ## Did it answer your question? - Mark it as a Solution 🙂

All code examples are provided as is. Responsibility for Code execution lies solely your own.
Ryan_Miles
Meraki Employee
Meraki Employee

@CaseyFowler I put together a deck on this topic awhile back. Might be of help to you.

 

https://docs.google.com/presentation/d/1xsb8imtUFjN13so86kIZ04IR9f6WEKdbpUrYVON64Zg/edit?usp=sharing 

Ryan / Meraki Solutions Engineer

If you found this post helpful, please give it Kudos. If my answer solves your problem please click Accept as Solution so others can benefit from it.

@Ryan_Milesout of curiousity, what are your thoughts on models as WAN Breakout switch? I usually see customers opting for the small 8-ports MS120 switch, but seeing as it will be handling "raw" internet traffic? A customer I have, which has just been through this maneuver used an MS410.

LinkedIn ::: https://blog.rhbirkelund.dk/

Like what you see? - Give a Kudo ## Did it answer your question? - Mark it as a Solution 🙂

All code examples are provided as is. Responsibility for Code execution lies solely your own.

Same. I often see MS1xx & 2xx series used for breakout. But, depending on physical handoffs, performance, or redundant PS and fans I could see what someone might go higher in the product family.

Ryan / Meraki Solutions Engineer

If you found this post helpful, please give it Kudos. If my answer solves your problem please click Accept as Solution so others can benefit from it.

The routing for the ISP and handoff can play a role as well. There are some ISPs that will do a transit IP and a block of public IPs behind that require at a minimum a static route in front of a warm spare MX setup. 

This is awesome and a huge help.

The only other bit that I'm not 100% on, has to do with the uplinks on the MX. Support basically said I just need to make sure to change from the default of VLAN tagging off and change it to use VLAN tagging with the corresponding VLAN from the MS120 it's connecting to.

 

Basically, does this make sense based on my original image?

CaseyFowler_0-1677541462354.png

 

The MX doesn't need to have any special VLAN configs. The switches hand off a L2 VLAN. No tagging comes into play.

Ryan / Meraki Solutions Engineer

If you found this post helpful, please give it Kudos. If my answer solves your problem please click Accept as Solution so others can benefit from it.

Hi Ryan, this is awesome.

Curious if this will cover any redundancy as well??? in case one meraki fails or one ISP goes down?

cmr
Kind of a big deal
Kind of a big deal

We use small 5 port Cisco unmanaged Gigabit switches for the ISP splitting.  I prefer not to have managed (and therefore more hackable) switches outside of the perimeter.  They are cheap as chips, perform perfectly well and have been very reliable.  They are also pretty much always in stock!

CaseyFowler
Conversationalist


@cmr wrote:

I prefer not to have managed (and therefore more hackable) switches outside of the perimeter.  


Great point that I wasn't really considering.


Do you have a particular model you've had good luck with? MSP had us looking at a catalyst 1000, which struck me as overkill but who am I to say...

I'm seeing a few like the CBS110 for $100. Would this be good? 

cmr
Kind of a big deal
Kind of a big deal

We use the SG110D-05 and I think the CBS110-5T-D is the direct replacement.  We've had very good service out of them and as they are so cheap, we always have a spare or two hanging around.

In the Meraki breakout switch design the management tunnel between the switch and dashboard is going through the MX appliance. I'd argue that's better than a simple switch sitting outside a firewall that might have telnet, SSH, or a web page enabled with less than adequate security config in place.

Ryan / Meraki Solutions Engineer

If you found this post helpful, please give it Kudos. If my answer solves your problem please click Accept as Solution so others can benefit from it.
Get notified when there are additional replies to this discussion.