Adding redundant connections - best practices

[Raccus]

Hi Everyone - I am fairly new to networking so please bear with me.  I am working on adding redundant connections to our topology at a remote site.  We are introducing two 3rd party core switches and demoting our meraki switches to distribution/access.  These switches are not stacked.   We plan to physically interconnect all switches.

Below is a small snapshot of what I am after. Red lines indicate FIBER that is ran to neighboring buildings (Switches 5 & 6).  Switches 1-4 are all located in our sever room.

After reading some documentation, I am having difficulty understanding how to best configure each MS port to support the below topology (ex: STP Guards). Any help or guidance will be appreciated.

[PhilipDAth · Accepted solution]

They kind of design looks great on paper but usually ends up in spanning tree hell with the occasional unplanned outage due to spanning tree issues.  If you do go down this path make sure you adjust the spanning-tree root bridge priorities and make something like switch 1 the primary and switch 2 the backup.

To get a more stable design you need to use stacking (I know you already said you aren't).  You stack switch 1 and switch 2.

I don't know if switch 3 and 4 are located next to each other, but let's pretend they are in different buildings.  You run two cables in an ether channel from switch 1 and 2 to switch 3.  Ditto for switch 1 and 2 to switch 4.  You do not connect switch 3 and 4 to each other directly.

If switches 3 and 4 are next to each other then you stack them and run a simple 2 cable Etherchannel from switch 1 and 2 to switch 3 and 4.

If switches 3 and 4 are stacked then you run dual cables from them in a channel to switch 5.  Repeat for switch 6.

You want to end up with a situation where there is no layer 2 loops, and then it will be rock-solid reliable.

[Raccus]

Thank you Phil.  I plan to setup the two core switches exactly how you explained it.

Switch 3 & 4 are located next to each other but I don't think I can physically stack them due to hardware limitations.  Are you referring to a virtual stack?

EDIT: 

I gather this topology is possible without stacking, assuming the root priority is set correctly on all switches. Outside of that, would I benefit from enabling STP guards on certain ports to mitigate the occasional spanning tree issues?

[CavemanDan]

A virtual stack of your MS210's would surely help reduce the STP footprint for you. 

I know it's documented about promoting the STP root. It'll definitely make things much more stable, but I have still run into trouble in mixed environments with that when things go pear shaped. But that happens with any environment with STP. 

[CavemanDan]

As it was mentioned before, it works great on paper, but I have had issues myself getting Meraki and 3rd party (Even Cisco Catalysts) to play nicely in a mixed environment. Spanning Tree is going to be your enemy here unless you make sure your Non Meraki switches are set up to play nice in such an environment. Even then you'll run into situations where STP just freaks out and your network will have unexpected complications. 

I personally have adopted stacking technologies as my standard best practice for enhanced redundancy to avoid STP issues. Stack your switches. Use LACP EtherChannels whenever possible. Use Layer 3 Segmentation as much as possible (Don't let your client VLANs on your Meraki's pass through your core - use a transit route between the MS210's and the Core). Try to limit your Layer 2 domains as much as you can -- don't let them span past switch stacks. This is especially important in a mixed vendor environment. Try to minimize your Spanning Tree as best as you can because that's where you're going to find the most problems. Unfortunately, STP is not an easy concept for a network newbie to grasp (I've been doing this 15+ years now and I still have a hard figuring out where the STP problems are). 

If it was me, I'd virtually stack all of your Meraki switches together. All those MS210's will do virtual stacking so you don't have to have them adjacent to one another physically. If your "core" supports some form of stacking I'd enable it. Then use those cross-connected interfaces between your two core's (1 and 2) and MS210 (switch 3 and 4) as a 4x LACP channel. This will give you combined throughput (in theory) and sub 1ms failover redundancy. 

Only thing you may find problematic is if you lose one of your MS210's. STP will try to reconverge again as the switch topology changes. Since they are Virtually stacked, they are still independent switches and have to learn the topology before they can finish the stack creation. You might find that an uplink port on your core goes into Error Disable during the reconvergence. 

From your diagram though, using the ASA 5510 (might want to upgrade that EoL device!) is probably going to save you some of the headache I ran into with my Meraki MX firewalls... they don't do STP. 

[Raccus]

Thank you for the detailed information. I will certainly look into stacking modules when the budget allows.

[CavemanDan]

What make/model are they? I recall for the 3560X series the stacking modules for those were horribly expensive. But aftermarket you might find "refurbished" units for very reasonable prices. There are aftermarket resellers of refurbished equipment that would offer such things for good prices. 

[Raccus]

They are Aruba 3810s