Optimizing Meraki API Calls: How to Filter Changes Efficiently Without Missing Important Data?

[Ignazio]

Hello everyone,

I have a question about an issue I'm dealing with regarding the use of Meraki APIs.

Current Setup

Currently, I have software that calls all the Meraki GET APIs once a day to update the data in my inventory. I want to optimize this process by avoiding unnecessary API calls when only a few changes have occurred.

Initial Approach

To address this, I’ve implemented the Meraki "Organization Configuration Changes" API to identify which organizations have undergone modifications. Additionally, I use Redis to store the timestamp of the last changes, ensuring that only unprocessed changes are handled.

Challenges Encountered

Initially, I thought of using this API to filter networks based on the networkId field provided in the response for each change. However, I encountered difficulties in creating a reliable standard to determine which network has been modified. For example, when a new network is created, the networkId field is not populated in the response.

Current Solution

My current solution involves calling the "Organization Configuration Changes" API, and if no changes are detected, I skip processing that specific organization. However, this approach means that even minor changes, like updating a note on a device, would trigger processing of the entire organization, including all associated networks and devices.

Question

Is there an API or a better method to apply a more efficient filter and at the same time reduce API calls to Meraki without risking the loss of important information?

[PhilipDAth]

You could consider using this API call, which gets all devices (in all networks) in one go.

https://developer.cisco.com/meraki/api-v1/get-organization-inventory-devices/

[Ignazio]

Hi Philip, thanks for the advice. Unfortunately, there are some parameters we need that are not present in this call, but regardless, I appreciate the suggestion. Thanks to it, I'm still able to optimize part of the process.

[John_on_API]

Great question, and good call using the org changes operation. This is a super efficient way to determine if changes have happened in an org.

However, could I ask why even a minor change triggers processing of the entire organization?

[Ignazio]

If, for example, the name of a device in an organization changes, then in the response of the "get organization configuration changes" API call, I will receive the organization that contains that device. As a result, I need to reprocess the entire organization to save all the data, because it is not easy to determine who and what has changed from the API response. Therefore, I am forced to indiscriminately update all the data for that organization, including the devices.

[John_on_API]

I do not understand--you should not need to reprocess the entire organization if you are using the changelog operation (getOrganizationConfigurationChanges), which would give you, at the top of its response, an entry like this, if someone changes the name of a device:

    {
        "ts": "2024-09-10T23:13:53.834424Z",
        "adminName": "Person Name",
        "adminEmail": "your@email.address",
        "adminId": "0",
        "networkName": "NETWORK NAME",
        "networkId": "L_000000000000000000",
        "networkUrl": "/URL_HERE/manage/usage/list",
        "ssidName": null,
        "ssidNumber": null,
        "page": "Cellular Gateways",
        "label": "Updated name",
        "oldValue": "bc:db:09:00:00:00: OLD NAME",
        "newValue": "bc:db:09:00:00:00: NEW NAME",
        "oauthClientId": null
    }

With the old/new value attributes you not only have the MAC of the device, you also have both the new and the old names. 

Why would you have to reprocess the entire organization?