MS220-8P switches randomly go offline

Cheif
Comes here often

MS220-8P switches randomly go offline

System: MX80 (DHCP)/ 5-MS220-8P/ 33-MR12 and MR62 mixed APs. / DNS is 8.8.8.8

 

Switches and APs go down several times a day, (all devices). MX gateway remains connected and up. This has been an ongoing problem, so I recently replaced a non-Meraki gateway/router with an MX80 device hoping to resolve the issue, but it did not. So, every time I get an alert I remotely reboot the gateway and all come back online. Configuration is straightforward and set to DHCP. I have checked every setting on the dashboard I could think of including running a cable check for each device. Anyone that has had a similar situation with a solution would be greatly appreciated.

15 Replies 15
Ryan_Miles
Meraki Employee
Meraki Employee

DM and your dashboard url and I can take a look

Ryan

If you found this post helpful, please give it Kudos. If my answer solves your problem please click Accept as Solution so others can benefit from it.
Cheif
Comes here often

Mssg sent, and thank you!

PhilipDAth
Kind of a big deal
Kind of a big deal

When you have a device showing as offline, which does the local status page say?  This will give you a big hint to the root cause of the issue.

 

Guesses of mine include:

  • IP address conflict (you mention you are using DHCP, but perhaps there are statically configured devices, or maybe even a second DHCP server).
  • Because you say rebooting the gateway resolves the problem, especially check for an IP address conflict with the default gateway address.
  • Perhaps DNS servers have become unreachable, or are no longer responding.
  • Do you have a switching loop, or do you have the MX dual connected to a switch?  If so you may have a spanning tree loop.  If the MX is dual connected for redundancy, try removing the redundancy.
  • Examine any redundant aspects in the network, in case the redundancy is incorrectly kicking in and causing the outages.
Cheif
Comes here often

The modem is DHCP and the MX is as well. I see a few IP conflicts in the event log for 1.1.1.1 ? and also see DHCP problems but those timestamps don't match outage times. I will call ISP and ask for a static IP.

 

I changed the DNS from ISP to 8.8.8.8 yesterday but no change..ie. went down twice since then.

 

MX to 5 MS220s. No dual connection. This problem existed long before adding the MX. I hoped the MX would solve the issue.

 

The routing is simple...Modem to MX to 5 switches, and those power 33 MR APs

 

 

Event Log.png

 

 

Jeizzen
Getting noticed

Basic question but : electrical issue ?

 

Are device really going down, or just losing connection to the cloud

 

As @PhilipDAth  said, ip conflict seems a good start to look at also

Cheif
Comes here often

Whole system goes down including internet connectivity. I put everything on a UPS and it seems to have become much more stable. I suspect ISP issues might be causing some of the problems but right now I am just monitoring to see if my changes have helped but it will take time.

Make_IT_Simple
Meraki Alumni (Retired)
Meraki Alumni (Retired)

If you are using an MX80, your ISP should be set to bridge mode, it shouldn't be doing DHCP of traffic inspection. Let the MX handle all the services including DHCP. Also, it would be a good idea to statically assign all your Meraki devices and exclude their IP for the DHCP pool.

Cheif
Comes here often

After talking to the ISP tech I clarified that the modem is open with no firewall. It is not a DHCP server and will only assign a WAN address to connected devices.

 

I have assigned static IPs to all 5 switches with no effect/ benefit. 

 

I can't logically figure out why the whole system goes down multiple times a day. I get the alert, and then remotely reboot the gateway and system comes back up. The gateway does not go down. This is the same issue we were having before installing the MX gateway. Yesterday I changed MX port 2 from LAN to WAN in order to add a secondary uplink to a different modem. I have already been told this won't help my situation but I am running out of ideas.

BlakeRichardson
Kind of a big deal
Kind of a big deal

Did I read correctly that you are running DHCP services on more than one device i.e. multiple devices giving DHCP? 

 

I would set your MX and one MS on static IP's and see if the problem goes away. 

Cheif
Comes here often

Currently the MX is the only DHCP server. I assigned all 5 switches static IPs. I was thinking about isolating 1 switch by directly connecting it to the MX to see if it continues to go down with the whole system. Running out of ideas here.

redsector
Head in the Cloud

Newest firmware in use? Use MS14.33.

Cheif
Comes here often

Running current firmware but no joy. The problem has existed for the past 2 years even before the recent addition of the MX. Wire tests on all devices give 8 green/ all good. System is a mixture of MR12, MR62, and a few MR66

Jeizzen
Getting noticed

Do you run RSTP on the switchports

BPDU guard on access ports

 

How many switches are directly connected to the MX (best practice should be only 1)

MX doesn't do Spanning tree

 

Cheif
Comes here often

MX > SW1 > SW2, SW3, SW4, SW5

RSTP is enabled, but BPDU was not so I enabled for all ports that feed an AP. 

IronBones
Here to help

So did making sure RSTP was enabled and enabling BPDU fix the random switch/system drops?

 

Similar issues on my network where switches randomly show offline. STP looks solid with NO Switchloops I can find. Topology describes a straight forward link from firewall SPF port to Switch, but still have switches dropping soon after reboot.

 

Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.
Labels