Meraki API did not work for about 1 hour. Why?

ToryDav
Building a reputation

Meraki API did not work for about 1 hour. Why?

Hey All,

Did anyone experience problems with the Meraki API (Python Module, version 1.18.2) between 8 PM and 9 PM Eastern Time?

It was brief, but it occurred during a scheduled change window where it did not work. 

I didn't notice any change in my enviroment at all during this time, and I was on a call with no issues performing a migration, which relied on the API to execute changes and do pre/post checks.

Timeline of events:

6:30 PM on 2/24/2022 - API was working, ran my prechecks script.
8:05 PM Tried running the script multiple times over 1/2 hour period of time.
8:35 PM Decided that since the API was returning the same timeout message, that I would proceed manually.

2022-02-24 20:05:10       meraki:     INFO > Meraki dashboard API session initialized with these parameters: {'version': '1.18.2', 'api_key': '************************************a96f', 'base_url': 'https://api.meraki.com/api/v1', 'single_request_timeout': 60, 'certificate_path': '', 'requests_proxy': '', 'wait_on_rate_limit': True, 'nginx_429_retry_wait_time': 60, 'action_batch_retry_wait_time': 60, 'retry_4xx_error': False, 'retry_4xx_error_wait_time': 60, 'maximum_retries': 2, 'simulate': False, 'be_geo_id': None, 'caller': None, 'use_iterator_for_get_pages': False}
2022-02-24 20:05:10       meraki:    DEBUG > {'tags': ['organizations', 'configure'], 'operation': 'getOrganizations', 'method': 'GET', 'url': '/organizations', 'params': None}
2022-02-24 20:05:10       meraki:     INFO > GET https://api.meraki.com/api/v1/organizations
2022-02-24 20:06:10       meraki:  WARNING > organizations, getOrganizations - HTTPSConnectionPool(host='api.meraki.com', port=443): Read timed out. (read timeout=60), retrying in 1 second


At about 930 after the completing the work manually I tested again just for kicks, and started getting 200 OKs

 

2022-02-24 21:29:52       meraki:     INFO > Meraki dashboard API session initialized with these parameters: {'version': '1.18.2', 'api_key': '************************************a96f', 'base_url': 'https://api.meraki.com/api/v1', 'single_request_timeout': 60, 'certificate_path': '', 'requests_proxy': '', 'wait_on_rate_limit': True, 'nginx_429_retry_wait_time': 60, 'action_batch_retry_wait_time': 60, 'retry_4xx_error': False, 'retry_4xx_error_wait_time': 60, 'maximum_retries': 2, 'simulate': True, 'be_geo_id': None, 'caller': None, 'use_iterator_for_get_pages': False}
2022-02-24 21:29:52       meraki:    DEBUG > {'tags': ['organizations', 'configure'], 'operation': 'getOrganizations', 'method': 'GET', 'url': '/organizations', 'params': None}
2022-02-24 21:29:52       meraki:     INFO > GET https://api.meraki.com/api/v1/organizations
2022-02-24 21:29:53       meraki:     INFO > GET https://n61.meraki.com/api/v1/organizations
2022-02-24 21:29:53       meraki:     INFO > organizations, getOrganizations - 200 OK



Was this a Meraki issue? Routing over the Internet? Something else? I didn't notice anything on my end that was wrong. I've used this code 5 nights a week for the last few weeks without issue and my code is frozen right now, meaning there are no changes happening until the project ends. 


Thoughts?

2 Replies 2
Ryan_Miles
Meraki Employee
Meraki Employee

Thousand eyes outage map was showing some potentially impacted services in Dallas right around that time. I also had a problem claiming an order that same hour. All seems fine now though. Seems like a possible carrier issue from what I saw but I don’t know for certain. 

Ryan

If you found this post helpful, please give it Kudos. If my answer solves your problem please click Accept as Solution so others can benefit from it.
sungod
Kind of a big deal

We saw API failures between 0100-0200 UTC on both USA and EU hosted organizations.

 

Seemed to be transient, only a couple of API calls failed hard enough for our system to give up on back-off/retry.

 

Get notified when there are additional replies to this discussion.