GX20 reboots every morning.

[awawa]

There's no particular update announcement, but is everyone's GX20 OK?

I've been restarting it every morning since Sunday. Is it broken?

[ConnorL]

Drop me a DM with a link to the GX, or the serial number and I can take a look. 

[awawa]

A lot of people seem to be having issues with their GX20. Any solutions?

[ConnorL]

As mentioned, drop me a message and I can take a look. Reboots can occur for a number of different reasons.

[SMAT_jp]

I'm having the same problem. I sent you a DM. Thanks in advance.

[scottysmoov]

Mine is doing the same thing. About 6:15am GX20 going offline.

Any news on what the problem is?

[hmja]

Since November 16th, I have been receiving an email alert every morning with the subject "Hardware offline alert (site name: ****)" for my GX20.

Then, about 30 minutes later, I receive an email alert with the subject "Hardware online alert."

It's inconvenient when the hardware is offline for those 30 minutes, so I'd like to know what to do.

[trgeuy]

I have been seeing the same issue. The uptime between outages appears to be 24 hours with a ~40 minute outage each time. Just opened a ticket with my ISP right before I found this.

[scottysmoov]

Make sure you've all raised a support case through the Meraki Go app. That why their service centre can see the extent of the problem and will up it's priority accordingly.

I have also reached out to my Contacts in Cisco. When I get an update I'll post it here.

[1R191]

Glad to see im not the only with this issue.  I raised a ticket and will be monitoring this thread.  Hopefully there is an update on this soon.

[SMAT_jp]

Since November 18th (Japan time), my Meraki Go GX20 has been repeatedly going offline, 
coming back online after 40 minutes, 
and then going offline again after 24 hours. Even after rebooting, 
it goes offline again after 24 hours and comes back online after 40 minutes. 

[scottysmoov]

Seems this problem is on going. My GX20 was down again this morning, it actually seems to happen every 25 hours, my outage is happening later each day.

I also checked the hardware info for my GX20. In the product information settings, next to Firmware it says 'Not Running Configured Version'. This suggests to me that maybe a faulty update has gone out.

Does anyone else have this?

[1R191]

I have the exact same issue.

[Boom]

I have this exact issue, also. 

[kozu_hiro]

私も同じ現象が続いています。
世界全体で起きている事象でしょうか。

[SMAT_jp]

私も同じ現象が11/18（日本時間）から毎日発生しています。どうもアップデートにバグがあったような話もありますがCISCOからはサポートには連絡しているものの音沙汰なしです。

[kozu_hiro]

ハード故障だと思い、GX20をAmazonで購入してしまいました。。。そのあとに、このサイトで同現象を発見しました。

[toru]

私のGX20でも同様の事象が発生しています。
丁度24時間なら時間がずれないのでいいのですが、オンライン（30～40分）から24時間後なので、もうそろそろ業務時間中にさしかかるため、シャレにならない。

[kozu_hiro]

ハード故障だと思い、GX20をAmazonで購入してしまいました。。。そのあとに、このサイトで同現象を発見しました。

[SMAT_jp]

私もカートにまでは入れてしまいました。買わなかったけど

[kozu_hiro]

新古品ですがAmazonで残り1個だったので、他に買われるとまずいと思い、私の方が先に買ってしまったかもしれませんね。会社で使っているものだから、毎日、オフラインになると仕事になりません。。。

[SMAT_jp]

私もテレワークで使っているのでGX20壊れたか？と思い再購入をまずは考えてしまいました。

ひとまず手動で再起動したら（まあ、家のネットワークをすべて再起動しましたけど）繋がったので様子をみていて『一応コミュニティ見てみるか？』でここにたどり着きました。

今のところ寝る前にGX20を手動で再起動することにしています。そうすれば24時間は使えるので....（笑）

[AmandaRD]

Same issue. Filed a ticket. Does anyone know if we can avoid this by doing manual restarts at a more convenient time? It’s happening during our morning coffee shop rush now and is really affecting business. 

[Wisekrak]

This began for me about a week ago. My theory is that a firmware update is trying to get pushed to our gx20’s and it is locking them up. Mine first locked up a week ago a little after 4:00AM. This is the time that I had set in the app to perform updates as to avoid interruptions. After power cycling everything is back up but under the GX20 details it states under firmware”Not running configured version”. That tells me that the upgrade failed and the device is running on the previous firmware.

then the next day it locks up nearly 24 hours after I power cycled the device. My theory here is that the cloud management realizes the firmware upgrade has failed and waits 24 hours to attempt again, resulting in a locked up router. The time the lockup occurs has been shifting forward each day as I have been getting the device back online sometimes 30 minutes after the lockup. Then another 24 hours passes, and it locks up again. Thiis morning after nearly a week if this issue the lockup occurred about 8:15 AM as I expected because that was about the time I got my device back online yesterday.

I have a lot of experience with Meraki’s enterprise product line and with those’d devices like the MX64 or MX67, you have a lot more control over the firmware updates. You can select beta versions vs stable, and you can schedule firmware updates with very granular control. You can even force the update immediately in order to get it to complete successfully while everything is already offline and I really wish I had that control on this product line.

i have been hesitant to perform a factory hardware reset if the device because of scars I have had in the past with enterprise MX64’s refusing to connect to the Meraki cloud and pull the config or firmware after doing so. I may just have to gamble on that soon because since I cannot control the deployment of firmware, the only other setting I have not the time to perform the upgrade. And I think the 24 hour retries are now in effect and changing the update time would have no effect.

[SMAT_jp]

For the time being, I don't use the network much late at night, so I manually restart it before going to bed. That way I can use it 24 hours a day.

[SMAT_jp]

手動でrebootのテストをしたところ　タイミングによってはLEDがレインボーと赤を定期的に繰り返して立ち上がらなくなりました。(10分以上放置) ご注意ください。電源ケーブルの抜き差し(10秒以上放置)で復旧しました。

When I manually tested the reboot, depending on the timing, the LED repeated the rainbow and red regularly and stopped standing up. ( Leave it for more than 10 minutes) Please be careful. It was restored by unplugging and inserting the power cable (left for more than 10 seconds).

[kozu_hiro]

私もその経験があります。再起動しても立ち上がらないことがあり、ハード的に故障したのかなと思ってしまいました。

[kozu_hiro]

ルータを別の製品（他社製品）に変えて解決しました。

[ConnorL]

Hi folks,

Thank you for sending over some example units, spot checking these I can see they're failing to upgrade to MX 19.2.4 and therefore rebooting every 24hrs. We've got a bug pending review whilst we gather some additional data points before sending this over to Engineering. Thank you for flagging these here, no need to send me any additional units. 

If you haven't already, please ensure you raise a case so this can be properly tracked.

[JamesO3212]

Connor,

I have several of these in the field.  Can you confirm I can reboot manually very late at night so they stop rebooting during work hours?  I will open tickets for each of these too.

Thanks,

James

[Ben-Wilson]

Hi,

Thanks for this thread as I was going crazy trying to fix this issue and was pointing the finger at my ISP.

Do we have a time frame for when this will be resolved just so I can manage client expectations? 

Thanks. 

[scottysmoov]

At the moment we have just been informed that Engineering are working on a fix, but there is no timescale. Updates will be sent when they have them.

The only work around seems to be to reboot your router at a convenient time, you then get 24 hours before it'll reboot itself again.

[JamesO3212]

I missed this.  Thanks for the work around, that is good enough for now if it prevents downtime during work day.

Happy TG!

[DarrenThorley]

Thanks for starting this post, I too thought I was going nuts as there is nothing I could see from the app to suggest what the issue may be. Hope it isn't long before a fix gets pushed out.

[JSKN]

Same issues here.  Started 11/16 around 4:30 AM.  Already opened a ticket with support. 

[JamesO3212]

disregard question answered tyty

[Ben-Wilson]

Can't login to my account as Meraki Go 2 factor email codes are just utterly broken so I spoke with Meraki support and they helped me pin the previous working update to the GX20 so it won't reboot every morning. 

I'm not sure why this wasn't done to begin with while we wait for the engineering team to fix this across all the devices.

It's clear this product is pretty much dead from a support point of view and will be looking to just bin them off and use something else in the near future. 

[JamesO3212]

I hear ya, I have been meaning to get the newest MGo router or move ecosystems.

[DaveTrompler]

Yes, having the same issue.  Also, the firmware is stating "Not running configured version".  

So, a manual reboot every night is the workaround huh?

[Dwig-HTLC]

Yeah, really frustrated about this - I've had a ticket open all week with no response, and clearly there is both an acknowledged bug in the firmware update and a known mitigation they could take (locking the firmware to a lower version) and Meraki is not prioritizing solving the problem.

I've always had great experiences with the enterprise Meraki gear we use for my day job, so I recommended the Go gear for the small installation at our church. Starting to regret that decision.

[JamesO3212]

Like all this consumer-ish stuff is eh, as soon as i was done with this convo yesterday I had Eero 6 Pro just default hard on me for no reason, could have also been firmware failure to be honest.  I was like what first I have 3 misbehaving GX20s then a E6Pro pops too. lol sunspots

[MP318]

I've got the exact same issue going on with one of my GX20's.  Opened a support case 
THREE DAYS AGO and haven't heard a word.  I've been a diehard Cisco & Meraki fan for years, but this is ridiculous.  I'm seriously looking at competitors products, as I can't have my customers going down daily and all I can tell them is "It's a known issue that they're aware of and working on" or "I opened a support ticket, but haven't heard back yet".  

Come on, Meraki!  You're better than this.  Or at least you used to be.....

[SMAT_jp]

Hi

今日も再起動しました。
昨日CISCOのサポートから１週間経ってから連絡がやっと入って、CISCO側の問題であることが確認できました。ファームアップグレードの失敗＝ファームのバグみたいです。
気の利いた会社ならば、まずは１つ前のファームウェアに戻すとかしてくれると思いますが、CISCOさんはイケてないです。

以下CSICOからのメールの抜粋です
ーーーーーーーーーー
ーーーーーーーーーー
現在、GX20がファームウェアアップグレードに繰り返し失敗し、1日に40分程度オフライン状態になるという問題が確認されております。
この度は弊社製品によりお客様に多大なるご不便、ご迷惑をおかけしており、誠に申し訳ございません。

この問題は現在弊社製品スペシャリストチームまで報告があげられ、問題解決のために調査が行われている状況でございます。
何かご報告可能な進展がございましたらケース上にてご連絡をさせていただきますので、
ご迷惑をおかけいたしまして誠に申し訳ございませんが今しばらくお時間をいただけますようお願い申し上げます。
ーーーーーーーーーー

Hi

I rebooted it again today.
Finally got a response from Cisco support a week later, and they confirmed it was their issue. The failed firmware upgrade seems to be a firmware bug.
A decent company would at least offer to revert to the previous firmware version, but Cisco is not cool.

Below is an excerpt from the email I got from Cisco:

——————————
We have confirmed an issue where the GX20 repeatedly fails to upgrade its firmware, resulting in approximately 40 minutes of offline status per day.
We sincerely apologize for the significant inconvenience and trouble our product has caused you.

This issue has been reported to our product specialist team, and investigations are currently underway to resolve it.
We will contact you via your case number as soon as we have any reportable progress.
We deeply apologize for the inconvenience and kindly ask for your patience and understanding as we work to resolve this matter.
——————————

[kinokino]

こんにちは
私もGX20にて11/16から毎日皆さんと同じ症状で悩んでおりましたが、このフォーラムを確認し、GX20のファームのせいだと確認できました、（間違えて中古のGX20を購入してしまいました^^;）
Ciscoが早く修正してくれるように皆様と共に祈っております

[SMAT_jp]

本日Ciscoサポートから連絡があって、

『本事象は現在修正に向けた対応を進めるために弊社エンジニアリングチームまで報告があげられている状況でございますが、
弊社内で検証が進められる中で、バックエンドから別のファームウェアバージョンにアップグレードを行うことで、
この問題を回避できる可能性があることが判明いたしました。

つきましては、先ほど私にて11月28日の日本時間午前3時に上記のアップデートが走るよう、バックエンドの設定変更を行わせていただきました。
これにより、明日以降はこの問題が発生しないことが期待されますが、もし問題が継続してしまう場合はその旨ご連絡いただけますと幸いでございます。』

とのことです。

やっと解放されそうです。（笑）

ーーーーーーーーーーー

Today we received notification from Cisco Support that: 'This issue has been escalated to our engineering team for resolution. During our internal verification process, we discovered that upgrading to a different firmware version from the backend may potentially resolve this problem. Therefore, I have just made the necessary backend configuration changes to trigger the above update at 3:00 AM Japan Standard Time on November 28th. We expect this to resolve the issue starting tomorrow. However, if the problem persists, we would appreciate it if you could kindly inform us." It seems we're finally getting some relief. (lol)