Need help with RSTP or OSPF configuration issue

[GaelGiraud]

Hello everyone,

I have a problem with my OSPF configuration. I am attaching a diagram below to better understand our network.

We have three datacenters connected together by dark fiber.
We have created a loop, DC 1 is connected to DC 2, DC 2 to DC 3 and finally DC 3 to DC 1.
We have servers on DC 1, DC 2 and DC 3.

The servers on DC 1 are backed up on DC 2, those on DC 2 are backed up on DC 3 and those on DC 3 are backed up on DC 1. We use OSPF to announce networks to other sites.

On the diagram, in green, I noted the interconnection networks between the DCs. All ports connected to the dark fiber are in access mode placed in the dedicated vlan.

In the white square placed on the switches, I noted the networks announced by the OSPF interfaces.
My problem is that RSTP blocks a port on the DC 3 switch to avoid the loop. The dark fiber between DC 2 and DC 3 is useless.

The backup traffic of the DC 2 servers (the red line) goes through DC 1 before going back down to DC 3.

In my mind, OSPF optimizes traffic by taking the shortest path between the source route and the destination route, the whole point being to recalculate the routing table when a link goes down.

When I look at blogs on OSPF, I find diagrams identical to mine and none of them talk about the problems of layer 2 loops.

My problem is also that this design implemented for one region, we want to implement it also on other regions where we have several data centers.

Has anyone in the community already set up this type of diagram or am I wrong with this design?
Thanks in advance to everyone, if my explanation is not detailed enough, I will answer all the additional questions.

Best regards,

Gael

[ww]

These are switches sending bpdu on native vlan, and meraki only support stp, so just a single topology (not per vlan). 

Most designs you looking at are probably using routers or switches using layer3 ports. (Like a catalyst with  # no swithport)

I supose you could disable stp on the ports using this design, buttttt a configuration/mistake in the future could lead to a loop. I think i wouldnt use meraki switches for this design

[GaelGiraud]

I disabled RSTP on the ports on each end and now my backup traffic goes through the shortest path, thanks for the advice.

If I use "MAC Allow List" on each end to limit communication to only the two switches, can it secure the disabling of RSTP?

For the rest of our deployments we will use C9300-48P.

With IOS, the "no switchport" command is possible so the hardware supports it.

I hope this will be possible with the same hardware in the Meraki dashboard, I will ask support.

Thanks again for your help.

[Ryan_Miles]

I would expect routed ports to come to C9300-Ms, but no timeline today.

[cmr]

Presumably with the Meraki native IOS version, hopefully next year 🤞

[MichaelChapman]

Hi, I have the same conundrum.  I have a college with multiple campuses.  Each campus has a core stack (some are Meraki and some are C9300 all managed via Meraki).  Each core has been set as the root bridge with a priority of 4096.  Each campus has multiple edge stacks and Wifi AP's (all Meraki).  The P2P links between all campuses are set to Access and use a SVI to mimic a L3 connection.  OSPF is run to share routes.  I too see some of the P2P links set to a discarding state to avoid a loop.  I cannot find anything definitive on Meraki website on how to configure a port that will be used as a P2P link (L3 only).  All of P2P ports have the RSTP Enabled and STP Guard Disabled.  I am always weary of disabling RSTP on a port.  But from what I have read from this thread, it is ok to do so?  All P2P links are on the core stacks.

Thanks

[ElwinNetworking]

Hi Michael,

The issue isn’t really OSPF but the fact that the inter-DC links are still running at Layer 2. With RSTP enabled, one leg will always be blocked to prevent a loop, which is why the DC2–DC3 link stays unused. Disabling RSTP removes that limitation but also removes loop protection. If you need to stay in Layer 2, it’s better to keep RSTP and enable BPDU Guard or Root Guard on those uplinks. Loop Guard adds a safety layer if a blocked port stops receiving BPDUs and accidentally transitions to forwarding. In the long run, using routed ports (no switchport) is cleaner since OSPF handles convergence and STP won’t interfere.

Docs worth checking:
https://www.cisco.com/c/en/us/support/docs/lan-switching/spanning-tree-protocol-stp-8021d/218321-con...
https://pingmynetwork.com/network/ccna-200-301/loop-guard

[GIdenJoe]

If you have multiple L3 switches directly peering to each other and you want to use a /30 p2p links between them you will need to do the following.

• Place each of those ports in access mode.
• Provision each of those ports on an unique VLAN per link (this VLAN must not be used anywhere outside of that link).
• Disable STP on the port.

[MichaelChapman]

Hi Elwin,

Thank you for the response.  I would like all of my P2P links to be L3, therefore should I set RSTP to disabled on the ports at each end of the P2P link?  I am always hesitant of disabling STP!

Thanks

[ElwinNetworking]

Sorry for the late reply, Michael.

If your P2P links are meant to run as pure Layer 3, the problem is that Meraki still treats them as switchports. Since they stay in Layer 2, RSTP will naturally block one side of the triangle. On Catalyst, no switchport fixes this, but Meraki doesn’t support routed ports yet.

f the P2P VLAN is fully isolated and only used between cores, disabling RSTP on both ends is usually safe. The only real risk is if that VLAN is ever extended somewhere it shouldn’t be.

[GIdenJoe]

Did you know that Catalyst switches that use 'routed ports' are in fact using hidden internal VLANs on each of the L3 ports individually?

If you look at the running config, somewhere it will say "vlan internal policy descending" or ascending 😉
For this example assume the switch is set to policy descending.  That means the first port you put on "no switchport" will in the backend become the only access port on that switch using vlan 4093.  The next one will then use vlan 4092 and so on.

So yes you can perfectly set this up.

[MichaelChapman]

One thing that does worry me, the Meraki's are running RSTP and the Catlyst C9300 (which are the core for some of the sites) are running MSTP (This was confirmed by Meraki Support).  Therefore even though my VLAN is only used on the P2P link, will it still be calculated in the STP algorithm?

I did hear that RSTP (and potentially) RPVST are coming out soon for the Catalysts (I am managing these via the Meraki dashboard).  

Thanks

[MichaelChapman]

Hi Elwin

The P2P vlan is exclusively used for the P2P link - I will try this on my next set up which should be in a month or two.

Thanks for the advice.