Meraki API did not work for about 1 hour. Why?

ToryDav
Building a reputation

Meraki API did not work for about 1 hour. Why?

Hey All,

Did anyone experience problems with the Meraki API (Python Module, version 1.18.2) between 8 PM and 9 PM Eastern Time?

It was brief, but it occurred during a scheduled change window where it did not work. 

I didn't notice any change in my enviroment at all during this time, and I was on a call with no issues performing a migration, which relied on the API to execute changes and do pre/post checks.

Timeline of events:

6:30 PM on 2/24/2022 - API was working, ran my prechecks script.
8:05 PM Tried running the script multiple times over 1/2 hour period of time.
8:35 PM Decided that since the API was returning the same timeout message, that I would proceed manually.

2022-02-24 20:05:10       meraki:     INFO > Meraki dashboard API session initialized with these parameters: {'version': '1.18.2', 'api_key': '************************************a96f', 'base_url': 'https://api.meraki.com/api/v1', 'single_request_timeout': 60, 'certificate_path': '', 'requests_proxy': '', 'wait_on_rate_limit': True, 'nginx_429_retry_wait_time': 60, 'action_batch_retry_wait_time': 60, 'retry_4xx_error': False, 'retry_4xx_error_wait_time': 60, 'maximum_retries': 2, 'simulate': False, 'be_geo_id': None, 'caller': None, 'use_iterator_for_get_pages': False}
2022-02-24 20:05:10       meraki:    DEBUG > {'tags': ['organizations', 'configure'], 'operation': 'getOrganizations', 'method': 'GET', 'url': '/organizations', 'params': None}
2022-02-24 20:05:10       meraki:     INFO > GET https://api.meraki.com/api/v1/organizations
2022-02-24 20:06:10       meraki:  WARNING > organizations, getOrganizations - HTTPSConnectionPool(host='api.meraki.com', port=443): Read timed out. (read timeout=60), retrying in 1 second


At about 930 after the completing the work manually I tested again just for kicks, and started getting 200 OKs

 

2022-02-24 21:29:52       meraki:     INFO > Meraki dashboard API session initialized with these parameters: {'version': '1.18.2', 'api_key': '************************************a96f', 'base_url': 'https://api.meraki.com/api/v1', 'single_request_timeout': 60, 'certificate_path': '', 'requests_proxy': '', 'wait_on_rate_limit': True, 'nginx_429_retry_wait_time': 60, 'action_batch_retry_wait_time': 60, 'retry_4xx_error': False, 'retry_4xx_error_wait_time': 60, 'maximum_retries': 2, 'simulate': True, 'be_geo_id': None, 'caller': None, 'use_iterator_for_get_pages': False}
2022-02-24 21:29:52       meraki:    DEBUG > {'tags': ['organizations', 'configure'], 'operation': 'getOrganizations', 'method': 'GET', 'url': '/organizations', 'params': None}
2022-02-24 21:29:52       meraki:     INFO > GET https://api.meraki.com/api/v1/organizations
2022-02-24 21:29:53       meraki:     INFO > GET https://n61.meraki.com/api/v1/organizations
2022-02-24 21:29:53       meraki:     INFO > organizations, getOrganizations - 200 OK



Was this a Meraki issue? Routing over the Internet? Something else? I didn't notice anything on my end that was wrong. I've used this code 5 nights a week for the last few weeks without issue and my code is frozen right now, meaning there are no changes happening until the project ends. 


Thoughts?

2 REPLIES 2
Ryan_Miles
Meraki Employee
Meraki Employee

Thousand eyes outage map was showing some potentially impacted services in Dallas right around that time. I also had a problem claiming an order that same hour. All seems fine now though. Seems like a possible carrier issue from what I saw but I don’t know for certain. 

sungod
Head in the Cloud

We saw API failures between 0100-0200 UTC on both USA and EU hosted organizations.

 

Seemed to be transient, only a couple of API calls failed hard enough for our system to give up on back-off/retry.

 

Get notified when there are additional replies to this discussion.