.networks.getNetworkClients fetch in batches

[RobHuijser]

Hi,

Because of the great amount of clients within our different networks in our organization(s) I want to fetch(get) client in small batches and not using [-1.all], but first 10 and then next 10 until end of all users.

I'm scarred about buffer overflow 🤔

Unfortunately I cannout find information how to do

.networks.getNetworkClients(networkID, total_pages=<-1,all>)

[Edgar-VO]

Good Morning,

As far as i know it is not possible.... I have been playing around with some parameters setting...

Only thing you can change is the lines per pages, timespan and direction. where prev are the last occurrance

I am also using this to monitor wifi usuage and things like that... until now, no overflows (yet)

The amount of datalines is the number of total_pages * perPage.

param = {}

param['timespan'] = 86400

param['perPage'] = 5

param['direction'] = 'next'

# param['direction'] = 'prev'

my_clients = dashboard.networks.getNetworkClients(network_id,total_pages=-1,**param)

for my_client in my_clients:

    print(my_client['id'])

[Greenberet]

I'm struggling with the same issue.

For the moment I can just say please be patient. I'm preparing a pull request for this, so that the methods will return an interator instead of the whole list/dict.

This should be better for speed and ram usage.

I hope I can finish this one this week. So if everything goes right, then it probably could be merged into the library.

[sungod · Accepted solution]

I just use this...

clients = await aiomeraki.networks.getNetworkClients(net['id'], timespan=t, perPage=1000, total_pages="all")

...run once every 24 hours with a timespan of a day. This is using API v1 Python package.

I use this in a script that gathers data across several organizations with hundreds of networks, some networks have thousands of clients. I add some additional handling for rate limiting as with so many networks and using async IO it is certain to rate limit.

If you only want to process data on 10 clients at a time, you will be generating 100x more API calls, far more likely that rate limiting will trigger.

But you can do it if you use the API call direct (i.e. rather than the Python package) and set perPage to 10, you'll need to add handling for pagination, API rate limiting etc.

This post has an example of using pagination (using the default 1000 page size, you'd need to add perPage parameter), you could adapt to get clients instead of devices, you'd still need to add handling for rate limiting back-off and retry...

https://community.meraki.com/t5/Developers-APIs/Python-pagination-get-8000-devices/td-p/101593

...but it seems simpler to just use the Python package.

[Greenberet]

Just for Information: Pull Request is out

[Greenberet]

The PR is already merged. It is not yet in a released version, but you can use it on the  v1 api directly from github.

In the creation of the Async/DashboardAPI object you have to specify the parameter 'use_iterator_for_get_pages = True'

examples: https://github.com/meraki/dashboard-api-python/blob/master/examples/get_pages_iterator.py & https://github.com/meraki/dashboard-api-python/blob/master/examples/aio_get_pages_iterator.py