Improve retrieving all clients in a group policy - use getNetworkPoliciesByClient?

[Ilya_Kisel]

Hello Meraki team!

I need to map all clients to the corresponding group policies using API, could you please suggest the best endpoint to do it?

1. I found that GET /networks/{networkId}/clients response contains "groupPolicy8021x" for each client.
Can you confirm that the client's "groupPolicy8021x" is the client's group policy id?
Can you confirm that if "groupPolicy8021x" is null, then the client is not assigned to any group policy?

2. I found an API which looks like an ideal solution:  GET /networks/{networkId}/policies/byClient 
Can you confirm that getNetworkPoliciesByClient endpoint returns a list of all clients and their group policy Ids?


3. My current solution (the same as in https://community.meraki.com/t5/Developers-APIs/Get-all-clients-in-a-group-policy/m-p/83163😞
1) Get all clients in a network: GET /networks/{networkId}/clients
2) For each client retrieve it's policy - GET /networks/{networkId}/clients/{clientId}/policy

The current approach requires N + 1 requests in order to load policies for N clients.
I have 36000 clients connected to one Meraki network, and it takes 8 hours to carry out 36000 requests to map all clients to the corresponding group policy.

Thank you,
Ilya

[rhbirkelund]

Try this..

#! /usr/bin/env python3

from datetime import datetime
import asyncio
import meraki
import meraki.aio

async def GetWirelessUsageHistory(p_dashboard: meraki.aio.AsyncDashboardAPI,p_network_id,p_client):
    """https://developer.cisco.com/meraki/api-v1/#!get-network-client-policy"""

    try:
        ClientPolicy = await p_dashboard.networks.getNetworkClientPolicy(
            p_network_id, p_client['id']
        )
    except meraki.AsyncAPIError as e:
        print(f"Meraki API error {e}")
        return -1
    except Exception as e:
        print(f"Some other exception {e}")
        return -2
    
    if ClientPolicy['devicePolicy'] == "Group policy":
        p_client['groupPolicyId'] = ClientPolicy['groupPolicyId']
    else:
        p_client['groupPolicyId'] = None

    return p_client
    
async def main():
    """Main function routine"""
    network_id = ""
    AllNetworkClients = []

    dashboard = meraki.DashboardAPI(
        suppress_logging=True
    )

    GetAllNetworkClients = dashboard.networks.getNetworkClients(
        network_id, total_pages='all'
    )

    async with meraki.aio.AsyncDashboardAPI(
        maximum_retries=5,
        suppress_logging=True
    ) as aiodashboard:
        GetGroupPolicyTasks = [GetWirelessUsageHistory(aiodashboard,network_id,client) for client in GetAllNetworkClients]
        for task in asyncio.as_completed(GetGroupPolicyTasks):
            ClientData = await task
            AllNetworkClients.append(ClientData)
            
    return 0

if __name__ == "__main__":
    ScriptStart = datetime.now()
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    print("\n$Time Elapsed:",datetime.now()-ScriptStart)

[Ilya_Kisel]

Hi rbnielsen, thanks for your time, but your solution is the same as my current. 
Read please the question one more time.

[rhbirkelund]

Interesting, since I came up with it myself last night.

Are you also using Asyncio? Because that is not stated in your post, which is also why I made the suggestion.

But nevermind then, carry on. To my knowledge, there is no other way.

[Ilya_Kisel]

Hi rbnielsen, thank you.
Adding parallelism will definitely reduce the time to collect all client policies by about 5 times (5 because of Meraki rate limiter which allows ~5 requests per second).
Anyway, the overall complexity is still O(N), because O(N/5) = O(N)

In the topic, I wonder if it is possible to reduce complexity to constant O(1), which is 1 or 2 requests to fetch all data.
For example, via https://developer.cisco.com/meraki/api-v1/#!get-network-policies-by-client

[Ilya_Kisel · Accepted solution]

The best solution is:

1) Fetch all clients via GET /networks/{networkId}/clients|
2) Put the fetched clients into a hashtable (Map) with client.id as a key and client as a value.

3) Fetch all clients with group policies via GET /networks/{networkId}/policies/byClient 
4) Match clients with policies to clients by client.id, using the Map from step 2.

In the end, you will have all clients with their corresponding group policies collected in 2 requests to Meraki API.
Both getClients and getNetworkPoliciesByClient requests support pagination with page size up to 1000.