Spontaneous Factory Reset?

MarcAEC
Building a reputation

Spontaneous Factory Reset?

I have a site that is experiencing a long term outage.  The users keep reporting loss of access to the network drives that are hosted on a server at the site.  When we looked into things, we found the Meraki MX had stopped issuing the correct IP addresses via DHCP and started issuing the default 192.16.0.x IP addresses (which no longer match the static IP the server is using). And when I went to the local status page, the password was back to the default (device serial number) and the static WAN configuration had been erased.  It appeared as if the MX had factory reset. 

 

At first I suspected a power disruption could have caused this.  I've seen cases when multiple power disruptions in quick succession did something like this.  But it has been happening every day now (3 times so far) and there haven't been any symptoms of a power problem.  Is the MX programmed to go back to defaults when it is unable to connect to the cloud controller for an extended period?

10 REPLIES 10
alemabrahao
Kind of a big deal
Kind of a big deal

I suggest you open a support case.

I am not a Cisco Meraki employee. My suggestions are based on documentation of Meraki best practices and day-to-day experience.

Please, if this post was useful, leave your kudos and mark it as solved.
MarcAEC
Building a reputation

I did.  Support's response was to state that the MX will not reset by itself.  But that's what it has been doing.

alemabrahao
Kind of a big deal
Kind of a big deal

What version are you running?

I am not a Cisco Meraki employee. My suggestions are based on documentation of Meraki best practices and day-to-day experience.

Please, if this post was useful, leave your kudos and mark it as solved.
MarcAEC
Building a reputation

17.10.2

alemabrahao
Kind of a big deal
Kind of a big deal

Have you noticed this issue with other versions? If not try downgrading It to version 16.16.x

I am not a Cisco Meraki employee. My suggestions are based on documentation of Meraki best practices and day-to-day experience.

Please, if this post was useful, leave your kudos and mark it as solved.
MarcAEC
Building a reputation

What a technician has been doing each day is taking the MX to a location that has internet and then allowing it to download the configuration.  Then we brings it back to the site without connectivity and we confirm it is issuing the correct IP addresses.  So, it's not a case of power being required in order to sustain the storage.

ww
Kind of a big deal
Kind of a big deal

Sound like some kind defect of the unit.. however 192.16.0.x is not a default.

192.168.128.x is.

Maybe hard factory reset it again and try run 16.16.x as suggested above

MarcAEC
Building a reputation

That's interesting.  The service provider resolved the issue, but we found the MX still couldn't connect to the cloud controller.  So, we did factory reset it and it came back online with the correct configuration. 

MarcAEC
Building a reputation

I asked support to have engineering grep the source code for "192.16.0" to find out under what conditions the MX will fall into serving the IP range, but they won't do it.  They're asking me to put the MX into the error state (e.g. take it offline for an extended period) and then get logs.  I don't have physical access to be able to do that.  And that shouldn't be a requirement to grep the source code anyway.

MarcAEC
Building a reputation

I set up a spare MX67 and replicated the conditions.  First, it was attached to the network and downloaded a configuration.  Then it was disconnected from the WAN and left offline for a few days.  Then a computer was connected to one of the network ports.  The MX67 was found giving out the 192.16.0.x IP addresses and the local status page had reverted to the default username and password. 

 

To me, it looks like a fail-safe mode that has the unfortunate side effect of making a major outage worse by removing access to local resources.  I could see some reasoning behind designing this type of fail-safe, but it seems the developers didn't take into account the fact that local servers tend to have static IPs.

Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.
Labels