502s and 503s to the Meraki API

[CH_EDJ]

Has anyone else been experiencing 502 and 503 errors when talking to the Meraki API too? It seems to have started in our systems around ~1pm CT. I am also getting them to the spec page too: https://api.meraki.com/api/v1/openapiSpec.

I am talking with my reps and it seems like this is a growing issue

[CH_EDJ]

It seems to be on all major endpoints (Clients, networks, devices, orgs, etc) and it is present on all of our Organizations.

[GiacomoS]

Hey CH,

We are doing some digging internally. Please keep an eye on the status page (status.meraki.net) as we might update everyone there if we track down anything major.

Bear with us as we scope this please!

Giac

[CesarCruz]

I had the same issue, but only with paginated requests using 'api.meraki', from 17 UTC to 21 UTC.

We usually use 'server.meraki.com', due to some constant 502 responses when we are using 'api.meraki'.

[RaphaelL]

Yup using the mega proxy seems to trigger way more http 5XX than using the shard directly.

[CesarCruz]

Unfortunately, when using paginated endpoints, the response headers refers to 'api.meraki'.

[RaphaelL]

Yes , but you can replace "api.meraki.com" to your shard. This is what I'm doing to avoid 502 and the slowness related to the mega proxy.

[CH_EDJ]

The issues seem to be resolved on all of the endpoints I was monitoring.

[PhilipDAth]

I actually got a 429 error today when using the dashboard.

[Ryan_Miles]

Using what endpoint(s)? The claim endpoints now do rate limiting and will throw 429s if you exceed 10 calls in a 5 min window.

[Alisdair85]

Hi Ryan, when did this rate limit on device claims get implemented, guessing last few weeks. We fully automate ZTP and have projects that are provisioning c.500 Meraki devices in single batches, this rate limit will impact that process if we can on average claim one serial number per 30 seconds? 

[Ryan_Miles]

On or around Feb 27th from what I see.

You can claim up to 200 (order numbers, license keys, and/or device serials) per request.

[Alisdair85]

Thanks Ryan, I was doing testing last night and the API bombs out due to the claiming rate limit after 6/7 serial numbers being claimed, going to investigate further.

[Ryan_Miles]

I would also recommend opening a case on this as that isn't even meeting the rate limit it sounds like.

[Ryan_Miles]

@Alisdair85 I was told the rate limit is being rolled back to the original 10 per second today (Mar 7, 2024).

[Alisdair85]

Excellent news! Thanks @Ryan_Miles 

[GiacomoS · Accepted solution]

Hey folks,

This should now be resolved as of 9:40PM UTC. 

If you are still seeing issues please drop us a Support case via Dashboard.

Thank you so much for bearing with us with this hiccup!

Giac