Upgrading firmware Bricked MX64 Cold Copy Config to backup unit and then Bricked backup MX64

[CharlieCrackle]

I have had a bad day today.  I had MX64 with   FW15.34  Insight was not working (not uploading data) Case 05530983  I was told that the next firmware update would fix this.  MX 15.36 was released  I upgraded  unit went into an end less reboot loop.   

Red light….  Then rainbow lights   then solid  white   then flashing white.   And then reboots back to red

              If I disconnect all cables  and  reboot and ONLY plug in the LAN cable  I get the normal  Meraki local status page  with internet problem  (rainbow lights)

              If I then plug in the internet  the light goes solid white  and I get the Green tick connected to meraki on local status page

              The light then goes  flashing white..  ( 2-3 min)  and then reboots and goes red.

              I tried to get a diag log from the device when connected to cloud but the unit reboots before it completes.

              I rebooted  the device  with no internet   and can connect via the LAN  but the device still reboots even with no internet  and I am unable to get it to download local support data before a reboot.

I rang support and they could see it connecting but did not know what was happening  and RMA the unit   CASE 05593909.

Network was down  (and MX is the DHCP server) BUT I was prepared I had another MX64.

I removed the old MX from the network and  and cold swapped  and the new unit connected to the cloud  (it was running  MX15.34)    I then got the error  "Unable to fetch configuration"  contact support  {I have been getting this alot lately with various customers, very annoying}    Support worked their magic  CASE 05594042  and fixed this connection issue.   

The unit then connected to the cloud loaded config  and then started working.

This was short lived as this unit then went into the reboot loop after about 10 minutes of working.

I went into the console and rolled back the firmware to  MX15.34    this did not help  still stuck in the reboot loop  support could see the device connecting to the cloud  and then it started to do a download.  they even blocked the download but still it did the download.

I factory reset both units and still they connected to the cloud  again with the   "Unable to fetch configuration"  contact support  support then fixed this again and again  they both went back into the reboot firmware down loop

Support tried to force a stable 14 version download  but this did not work either just kept rebooting.

I have the RMA device coming tomorrow but I am scared this unit is just going to have the same issue  (it might not now as the firmware has been downgraded  finger crossed)

It was a real pain as the network was down most of the day until I configured and OLD Cisco ASA to replace the meraki.

Any way  this is the reason I usually test firmware on my network before loading on customer network  but this got me worried.  I have so many customers on meraki now  if something like this happened to them I would not be very popular !!

Does any one have any idea what is going on here ????

[cmr]

@CharlieCrackle not that it helps but we have some MX64s on 15.36, they were on 15.33 and upgraded the day 15.36 came out.  Also MX65s, 67s, 84s and 100s.  Have support any idea why yours are bricking?

[CharlieCrackle]

Thanks for the info at least I know others are using with out issue. may be is it something in my config. not that my config is anything special other than a lot of DHCP scopes and insight.

[CharlieCrackle · Accepted solution]

Just and update for any other that get this issue

Meraki RMA unit. after looking that this logs this was the reply from support.

"it had an issue where it was not able to keep its configuration properly. Due to that, it also could not keep the firmware that it tried to download from the cloud and properly install. It became a loop where it needed the configuration to fix the issue to keep its firmware. As you could see, it visually reflected that as it kept going from red to rainbow to white and back to red. Usually we would help it get online by pushing that configuration and then applying the fix. It did not hold it well even after that so it is fine that we RMA'd this device. "