Using uplink ports to connect switch

[Peter-Loyen]

Hi,

We have a MS225-24P on which we use port 25 to connect a Cisco catalyst switch.

This switch is part of a stack of 3 switches.

Only 1 has the port 25 connected.

The actual uplink to the router is port 24 (the one used to connect to the internet and cloud controller)

Issue: When we reboot the switch with the uplink port 25, the whole stack goes down.

Is this because the switch has a dedicated uplink port (port 25-26) are SFP ports and considered as uplink ports?

Has someone encountered the same behaviour?

We are on release 12.2

Thanks

Peter

[Bruce]

I don’t have an answer for you, but here’s some things to check:

• If you’re on MS12.2 I’d upgrade MS12 code is up to about 12.33 now.
• Not that I think it’s an issue, but make sure the Catalyst switch is configured to run a compatible version of STP. By default Catalyst switches run PVST/Rapid-PVST which can cause issues on a Meraki network - switch the Catalyst to MSTP (MST).
• Are the other switches in the stack actually going down, rebooting, or failing to pass traffic, or just being reported as offline/down? The latter would be expected if the switch with the link to the Meraki Cloud goes down.

There’s no ‘designation’ of SFP ports as uplink ports or the such - they’re just ports, like an RJ45 port, but they’re SFP-based.

[Peter-Loyen]

Hi Bruce,

First of all, I made an error on my initial statement: The switch stack are all MS410-16 switches.

port 17 on the switch A has the following SFP:

SFP module

CISCO-AVAGO - 1000BASE-T: 1.3 Gbps

Yes all switches in the stack go down.

Switch A and B in the stack have port 16 connected to the router in an aggregation (LACP)

Only switch A has a port 17 connected.

Switch C is more of a backup with very limited connections.

Performing a soft reboot on the switch A via the cloud portal kills the whole stack and we get somewhere a loop of traffic because there is about 340Gb/s passing through these switches.

Only after hard rebooting (power down/up) all 3 switches, things got back up.

To be more clear and make things complicated, the catalyst switches are in between two Meraki MS switches.

And a dot1q tunnel is created between the ports of the catalyst switches on each end where the Meraki switches connect to.

So you get this:

MS220-8P (port 8 -----> Cisco catalyst --> Cisco catalyst --> fiber port to MS410-16 (port 17)

                    trunk    <----------------------- dot1q tunnel vlan 104 -------------------------> trunk 

                (vlans 602, 605, 10,20)                                                                                (vlans 602,605,10,20)

So there is no real STP negotiation with the Catalysts for the vlans in the trunk.

The reason for such an ugly setup (let;s face it, it is) is the catalyst switches belong to the customer network and there are used for other purposes. Additional fiber connections cannot be provided or created due to the location of the sites. 

[cmr]

@Peter-Loyen what firmware are the 410s running?  If it is 12.2 as you initially mentioned then it seriously needs upgrading, I suspect that it might be different as 12.8 is the earliest released version as far as I am aware.

[Bruce]

Maybe this has nothing to do with the Catalyst switches. Maybe when you reboot Switch A the link aggregation is breaking and you’re creating a loop through the router. Rebooting all the switches may be getting the link aggregation working again.

Have you tried disconnecting one of the links to the router when you have the issue? Or trying and reboot Switch A with only the link to the router from Switch B being connected?

What model is the router, and how is the LACP configured on it?

[PhilipDAth]

This smells like it could be a spanning tree issue.  I'd follow @Bruce 's suggestions first.

Do any of the switch logs report anything interesting?

Make sure to use MST on the Catalyst switches.  Which switch have you made the root of the spanning tree?