Meraki MX75 Random reboots

[SeanW]

Just an open topic to owners of the MX75 router. I noticed that my MX75 would randomly stop passing traffic and at times would randomly reboot on its own. There is no real constant with this issue as its happened over multiple firmware releases, cable replacements, and UPSs updates. I was on the phone with support today about this issue and he stated there have been reports of MS75 owners having random issues the unit have two or three causes they are looking into. I'm ok with this (to a point), but I hadn't seen any buzz on the forums about this so I am just opening this topic to see if anyone else is having this issue as well, and if it was fixed what fixed it for you?

[BlakeRichardson]

Hmmm I don't recall seeing this being a reoccurring issue. Have you tried an RMA to see if that resolves the problem for you. Usually with large scale issues a replacement program is launched and affected devices replaced.

[SeanW]

My Eng wants to wait and see before setting up the RMA.

[RWelch]

I’d have a tough time with the wait and see approach.  If there is an impact to business the RMA would be the route I’d lean on/toward.  Just my 2 cents.

[Brash]

Not sure about the not passing traffic. 

There is however  a hardware defect that impacts some MX75's causing it to crash and reboot due to a kernel panic.

Support should be able to confirm whether that's occurring or not.

For reference: Re: I'm seeing some very strange and unlikely high defect rates with MX75 (... - The Meraki Communit...

[cmr]

I had an MX75 replaced as the power socket was dodgy and touching the connector would reboot it.  The replacement was apparently one of the dodgy ones as it rebooted a few times, especially when both PoE ports were used.  However upgrading to MX19 fixed it and it has been stable since.

[SeanW]

Funny enough 19.1.5 was my less stable FW.

[TyShawn]

Hey @Brash,

What was the final result of your issue, did the replacement fix your issue or are you still fighting the good fight?

[Brash]

Support confirmed the issue on 2 MX75's and sent RMA's. Aside from the replacement device, they also came with smaller power transformers.

I haven't had any further random reboots on them since then.

[cmr]

I had one of those smaller (more powerful) transformers with the RMA replacement I had for a loose power connection.  Are your new PSUs more powerful?

[Brash]

Hmmm can't recall. I just checked they had the same voltage and polarity and swapped them in along with the replacement MX

[cmr]

The replacement one I received was an 120W FSP power supply.

[TyShawn]

My PSU looks to be the same for both units, but give me a couple hours and I can post some pics of both.

[Inderdeep]

I would suggest reach out to meraki support first and then go with RMA if required @SeanW 

[PhilipDAth]

I *think* I have had my MX75 reboot once.  It's hard to tell apart from jut a short Internet disruption.

[jimmyt234]

Yes - we have seen lots of MX75 devices plagued by 'panic reboots' which can have several underlying causes according to support. Get it RMA'd.

[SeanW]

After rolling back to 18.211.4 things have stabilized again. Next steps is to try 19.1.4 again and see if the issues return. 

Guess it would help if I gave some additional information on the setup a well.

Topology ISP 1 (1G) - ISP 2 (1G) --> MX75 --> L3 VLAN --> Unifi 10GB switch L3 VLAN --> Multiple VLANs over multiple ports --> MR 44 1 - MR 44 2 --> Clients wired - Wireless.

[SeanW]

Small update on this issue. FW 18.211.4 was stable and did not have the reboot issue. I've updated the MX to 19.1.6 and so far the issue hasn't returned. 

[TyShawn]

Christmas Eve update: My MX75 had another kernel panic event yesterday with 19.1.6, Meraki support is sending out a replacement unit.

[TyShawn]

Update: The replacement unit is installed and updated to 19.1.6, now we wait and see if the issue returns.

[TyShawn]

Welp that was the longest couple hours in history :).

OLD PSU

New PSU

[cmr]

I think my replacement PSU was the one you had...  The old one I had was smaller and I think the make started with Chang...  I can't tell if your new is that, of a third variant?

[cmr]

This is the replacement I got, I think it is your old one as it doesn't have the EAC mark on it:

 

[cmr]

Actually mine doesn't even have a UKCA mark... 🤔

[TyShawn]

Maybe a pre-production model...?

[cmr]

😬

[TyShawn]

So the PN is the same over all three pictures, and you're right EAC is different.

[JGTS]

Hi all,

Also experiencing this at one site, and have RMA'd 3 times now. The most recent replacement lasted 32 days online over the Christmas break but then rebooted today randomly again. I've reopened the ticket again.

Support had previously advised that engineering were aware of the issue but no solution available yet. The client is ready to pull the plug entirely. 

Has anyone had some success on this yet?

Thanks. 

[TyShawn]

So far things are holding steady for me but still soaking.

[JGTS]

Our last replacement took a month to fail again, just when we thought we were home free. 😓

We're in Australia, where RMA stock is supplied separately from new stock. I.e., if I order a fresh MX75 it comes out of Ontario and takes a week to arrive, but if I RMA a device, it comes locally the next day. If it is indeed a PSU problem, I wonder if the 'new' stock coming out of Ontario has the fixed PSU. 

[cmr]

I had one restart yesterday morning that (as far as I can tell) has been stable for months, but initially crashed a few times.  Unfortunately the logs only show ethernet carrier changes for all ports followed by X dropped events...  Therefore unless it impacts you, it's difficult to trace when it happens!

[dhub]

Same issue here, I'm running MX 18.211.4.  Support suggested I factory reset, I did that on January 1 - the MX75 has crashed 3 times since factory reset.  Event log shows Ethernet port carrier change.  I'm upgrading to MX 18.211.5 tonight.

Support offered to RMA if it crashes again but I'm not too optimistic reading through all of these responses.

[JGTS]

I'm now onto my 4th RMA for the same site. The customer is ready to walk away. 

[cmr]

What kind of configuration and ports in use do they have?

[jOMeraki2]

This issue happened to me with the same model, and the solution was that they processed an RMA for me.

[GiacomoS]

Hey folks,

Apologies for the radio silence on this issue here. We have been seeing this issue with a few customers and have been hard at work with all your support (thank you everyone who troubleshot with us!) to get to the root cause of it. 

This investigation is being treated as a priority and we have dedicated all suitable resources to help resolve it; for units identified hitting this issue we have been leveraging RMAs as the quickest path to recovery, but we appreciate that this is not an ideal step in all circumstances and deployments. 

For the time being, I'd continue to encourage everyone to raise a case (if you haven't already), make sure you troubleshoot with Support and discuss the best option based on your individual circumstances (RMA or wait for a software fix if possible). 

Thank you for all your patience here, we completely understand how impactful this is. 

Giac

[CedricMX]

Hello,

Are you 100% sure that the MX75s returned as replacements will not restart randomly?
I saw on the Meraki community that some users had obtained replacement MX75s that were also affected by this same problem.
Can you send 100% functional MX75s? Is it possible to do the stress tests at Cisco rather than asking your customers to do them?
We are Meraki customers and have no time to lose.

[JGTS]

Thank you for replying. I'm on my 4th replacement at one site, and have two more exhibiting the same behavior. Is there a way we can get an RMA from a different batch?

As you can imagine, it's extremely disruptive to the client and puts a lot of stress on our technical team to respond each time. As a business, we're also having to cover the labor costs each time it needs troubleshooting and replacement, as well as wearing the reputational damage for continuing to stand behind the product. 

Is there anything else more tangible that you can provide around an ETA, so that we can pass this on to the client? I'm sure everyone's working hard on a solution, but at this late stage, we need to communicate more than it is still a priority. 

Thank you. 

[CedricMX]

Did the Meraki RMA solve your reboot problem?
Has a replacement product ever been affected by the reboot problem?

[JGTS]

Yes, I've had 3 faulty replacements and currently onto a 4th now (stable for four days so far). The 3rd replacement lasted 31 days before it started rebooting randomly again. 

[CedricMX]

ohhh ! 

I understand why you are talking about time and costs about this issue with MX75.

If spare products are not usable, has Meraki proposed another solution? For example, make somes tests on replacement products before sending them to customers or offering another mx model in order to compensate for customers extra cost due to this issue ?

[JGTS]

Unfortunately, not yet. I've asked the question, but haven't been given a solution apart from continuously RMAing at the moment.

I do have a number of other MX75s in the field that aren't affected by the same issue though, so it's very much a game of roulette at the moment.