MR's intermittent loosing connection to Meraki cloud

[Jayt]

Started 2 weeks ago. Across 3 networks & different locations random MR36 & MR46's loose connection to the Meraki cloud. Last 2 minutes then they reconnect. I can see they say offline & then come back. Running 28.5 on them all.  And it's waves. I'll have 2 from 1 network then a 3rd from another all at the same time. This week is worse then last week.

1 site may have 4 AP's but only 1 or 2 goes down. I never lose the network connections or internet.

Any suggestions? I was on hold with support for 35 minutes & had to hang up.

Thanks

[cmr]

Are they all going through the same firewall or same model of firewall? 

If not, are they all going through the same ISP?

I'd suggest it is something in either of those places.

[Ryan_Miles]

Hard to say without looking at your APs. Did you get a case opened with Support?

[redsector]

I have got the same issues. A lot of networks, and sometimes one of many accesspoints isn´t reachable for a few minutes. Different networks, different firewalls. Only random accesspoints out of many. MR 44, MR42, MR46, MR4E, MR46E.

[maxwell]

We are having the same issue after upgrading to 28.5.

[cmr]

It sounds to me like a shard issue.  We ran 28.5 from the day of release on 120+ APs without any of this and we are now on 28.6 without issues.  We use shards 184 and 199, what are you on?

[maxwell]

Cmr, 

We are using Cisco Firepower for our firewall - though we have exceptions for all the necessary Meraki traffic. 

[cmr]

@maxwell what dashboard shard are you on (i.e. n123.meraki.com), we didn't see this at all with 28.5 and haven't seen it with 28.6 o far either.

[maxwell]

Shard 179. I rolled back to stable firmware 26.8.2 due to the issue progressing, unfortunately. 

[cmr]

@maxwell why didn't you try 28.6?

[redsector]

stupid question: what is shard?

[cmr]

@redsector no questions are stupid!  It is the number in the dashboard URL, i.e. n123.meraki.com

[redsector]

Thank you.

[chadnick18]

We have been having the exact same issue since upgrading to 28.5. We get down alerts for random AP's across all of our networks. Most of our networks share the same ISP connection and are behind a Palo Alto firewall.  One of our networks is a remote site that has a completely different ISP, is behind an MX250 and connected to an MS390, and we receive random down alerts for the AP's at that location too but not for the MX or MS. We've had 2 separate tickets open with no resolution, they want us to call as soon as it happens so they can collect more information while it's happening.

[MrRoboto2338]

Had this issue for the last few days. Have had no changes to our firewall configuration or network. The only change was upgrading to firmware 28.5 and our MR36 network, the MR33 network on 28.5 has no issues. I suppose it could be our provider, TDS.

[cmr]

This sounds to me like the firewall is interfering with the management traffic, now that it is using port 443.  Can each person who is suffering this issue check to see if their management traffic is using 443 or the old ports.  Can you also please report what firewall vendor you use, if you are allowed to.

[WB]

Had this issue after bumping to 28.5 and the change to use 443 as the main comms port. On our FW we had to both allow HTTPS outbound for the MRs and exempt the external Meraki destinations from the captive portal as it captures any unauthenticated traffic

[Jayt]

OP here.  It just stopped. Don't know why.  It wasn't a firewall problem with ports. The same firewall at some locations serve the same MR 36 & 46's. Yet only some would go offline & come back.

[EJN]

Same issue. Haven't been able to fix with support. 43 APs into 21 MS switches into a core MS390 into a Juniper SRX1500 Comcast Managed Router. It's not power or ISP down. Random APs at random times. Usually under 5 minutes. Some days none, some days a few. Running 28.6, but now new. This has been happening since Fall 2021, but for sure more often since December.

Could this be port 443 related? If yes, I'm assuming this requires a call to Comcast to open that port on the Juniper SRX1500?

[WB]

I'd say at least opening all the required ports Meraki states to all their external IP ranges would be the minimum that support will check with you first during troubleshooting. Hopefully support can drill down in the backend to check logs of the APs that go down/up

[MrRoboto2338]

Had this issue for the last few days. Have had no changes to our firewall configuration or network. The only change was upgrading to firmware 28.5 and our MR36 network, the MR33 network on 28.5 has no issues. I suppose it could be our provider, TDS.

[letskungfu]

Also experiencing this issue for the last few weeks, seems to line up with the switch to 443.  Mostly MR46 and MR42 in my enviornment.  The 46's are experiencing random disconnects from the console as described here and also higher latency when pinging these devices from the dashboard.  Interestingly, the 42's are uneffected by both the disconnects and the latency. 

[RLEONARD-SJUSD]

We have begun seeing the same thing since upgrading to 28.6 on our MR 46. Devices will go offline for a couple/few minutes in batches throughout our domain.  

[ZedKay]

Noting the same issue. MR on 28.5 and we are seeing multiple disconnects across on various networks - but all at the same time.

MS MX and MV all remain up.

ticket has been opened with support. - but no update.

[scytales]

Yet another reporting of the issue here. 

Currently on 28.6. MR APs randomly lose cloud connectivity. confirmed accessibility to 209.206.48.0/20