C9300X-48HX Stacking Issues

[Robthesoundguy]

Has anyone experienced issues with C9300X-48HX-M switches running CS17.2.1.1 in an 8 switch stack losing connectivity to the cloud? 

I restarted a stack that I have pre-staged right now and it won't reconnect to the cloud. While working with support, they said that it's a known issue with 8 switch stacks and that the best option is to split the stack into two, smaller stacks. 

I'm curious if anyone else has experienced this behavior and if they've been told the same thing by support. 

[cmr]

@Robthesoundguy I would definitely split the stack.  With the CS firmware anything over 4 switches is dicey.  The IOS-XE beta should be better but I'd still stay with no more than four for the moment.  What are you using as the uplink connection, or were you planning on the stack of 8 being the core?

[Robthesoundguy]

@cmr The stack of 8 is the core for the particular building. There are three other switches (two switch stack and a single) that mLAG off of this stack and this stack routes upstream to the DC for global vlans and internet. 

Splitting the stacks is less than ideal, but I supposed I could aggregate ports on the network modules between the two smaller stacks for linking. 

My gut is telling me that IOS-XE would fix it, but I'm not willing to put a beta firmware into production. 

Thanks for the insight!

[cmr]

@Robthesoundguy if it is routing L3 traffic then 100% split the stack!  In fact I would usually only have a stack of 2 for routing.  I have had L3 issues with stacks of 4 MS switches, let alone CS ones.  3 is usually okay, but 2 is the sweet spot.

[PhilipDAth]

I can't find it now, but I have read several documents on the Cisco Enterprise site for the C9300 that recommend not using a stack of more than 4 switches.

My recollection is that the switches can actually only form a stack of 4 switches, and when you physically have more than that, they self-organise into sets of stacks of 4.

[GIdenJoe]

I find this strange.  I haven't had any C9300 occasions with more than 4 switches however I would recommend using show switch command to verify this.

I'm not a fan of the CS sofware and exclusively run these in IOS-XE native mode.  I found 17.15.3.1 to be quite stable.  Now I can see what is going on in the switches themselves.

Officially you can use datastacks up to 8 where you have 2 power stacks.

[PhilipDAth]

2 power stacks are not supported within one data stack. You are allowed to use a maximum of one power stack.

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9300/hardware/install/b_c9300_hig/Install...

[GIdenJoe]

I have read the stacking and power stacking 3x now and I cannot find any definition that says you can't have 2 power stacks within 1 data stack.

Reddit and Cisco community boards also give mixed results where some give it as a solution, even one person that wanted to implement it over a year ago and there were no followups on that thread so I assume it either worked or the person gave up on it.

[GIdenJoe]

Aha I found official documentation from Cisco that says you CAN have 2 powerstacks within 1 datastack:

https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9300-series-switches/white-paper...

[PhilipDAth]

I found one of the documents.  If you are also using stack power, a maximum of 4 switches is permitted.

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9300/hardware/install/b_c9300_hig/Install...

[Robthesoundguy]

What burns me about this is that the official datasheet says that it can stack up to 8. 

https://documentation.meraki.com/MS/MS_Overview_and_Specifications/Catalyst_9300X-M_Datasheet

[cmr]

The data stacking *can* cope with 8 switches, but it really isn't recommended, the power stacking can only cope with 4 switches, but it is still one of the best C9300 features in my opinion!  Power and data stacks don't have to match, but I think it is better to do so from a human brain point of view!

[PhilipDAth]

>Power and data stacks don't have to match

I think there are a lot of caveats around this.  I don't "think" you have two power stacks in a single data stack.  Sure, you can plug the cables in, but I don't think the power redundancy works as you would expect it to.

As soon as you start using power stacking, limit the data stack to four switches.  Or at least, do not use power stacking for more than 4 switches.

Once you know that the data stacks internally form into groups of 4 switches, it all becomes clearer.