9300 - Out of date (reverted to safe config)

[NetEngJH]

Has anyone seen this error before and know how to resolve it?  I've got some 9300 switches that are offline and unable to get back online, even after a factory reset.  Dashboard connectivity is there as the switches are fetching the config, but not applying it and reverting to some sort of safe config.

[alemabrahao]

Do you have any firewalls in the way? Have you tried accessing the Local Status page of the switches to check the error?

I also suggest you open a support case.

[NetEngJH]

The local status page isn't responding on http.  It is however accessible on ssh which is very suspicious. Although I can't login with any combination of serial numbers and admin username, or the local network credentials.

Wireless access points connected to the switch on the same management subnet can connect to the dashboard absolutely fine.  So the path from the network to the internet is working OK i.e. no firewall issues.

[alemabrahao]

If you are accessing via SSH then your C9300 is not running the Meraki image.

If it was, then you would not be able to access via SSH. Have you tried the default Cisco user?

[alemabrahao]

Are you managing via the Dashboard or just monitoring?

[NetEngJH]

Managing via Dashboard on v17.1.4.  Yeah tried Cisco/Cisco and combinations of upper and lower case, as well as serial numbers.

[alemabrahao]

So I highly recommend you open a support case with Meraki.

[NetEngJH]

yeah that is already in progress.  I wanted to see if the community had any suggestions in the meantime. 

[rhbirkelund]

One of the ways Meraki manages the 9300 in meraki persona is also using SSH. If you have a 9300 online, and view the logging from the in-browser terminal, you'll see many logins using the meraki-user account. So eventhough it's in Meraki Person, it'll probably still be open to SSH, but only the meraki dashboard backend user will be able to login. You as an administrator will not.

My guess is that there may be some conflicting config that is resulting the 9300 to lose its uplink. Did you recently do any changes to the 9300? Did you change vlans on the uplinks or modify stp? 

The other day I had put Root Guard on the 9300 uplinks, and that resulted in all uplink ports going in to a blocked state. Fortunately I could change the stp guard again, and the switch came online again.

I suggest that you verify the dashboard configuration and then factory reset the 9300 by holding the Reset (mode) button for more than 21 seconds. https://documentation.meraki.com/MS/MS_Installation_Guides/Catalyst_9300-M_Series_Installation_Guide...

[rhbirkelund]

Here is a port scan of a 9300 in meraki persona i a lab environment.

RHB@wopr ~ % nmap 10.10.1.56
Starting Nmap 7.95 ( https://nmap.org ) at 2025-03-25 11:23 CET
Nmap scan report for 10.10.1.56
Host is up (0.042s latency).
Not shown: 994 filtered tcp ports (no-response)
PORT     STATE SERVICE
22/tcp   open  ssh
53/tcp   open  domain
80/tcp   open  http
81/tcp   open  hosts2-ns
443/tcp  open  https
2068/tcp open  avocentkvm

Nmap done: 1 IP address (1 host up) scanned in 59.49 seconds
RHB@wopr ~ % 

[NetEngJH]

nmap of the the switch stack in this case only has ssh open.  If only I knew what the login credentials were.

Starting Nmap 7.93 ( https://nmap.org ) at 2025-03-25 12:13 UTC
Host is up (0.088s latency).
Not shown: 999 filtered tcp ports (no-response)
PORT STATE SERVICE
22/tcp open ssh

[NetEngJH]

I've removed the switches from the network and back to inventory, and back into the network.  This removed all the switchport configurations and moved them all back to trunks. I've also changed from using static IP for management IP back to DHCP.  Still to same problem.  This is actually a pair of switches and the last thing that changed was creating them into a stack.  I have also tried remove the stack config, physically removing the stack cables and factory resetting each switch individually, then try to form the stack again.   Still the same problem.

Support are in the process of issuing an RMA.