Issues with getNetworkEvents

[RaphaelL]

Hi ,

Is there anyone currently using the endpoint getNetworkEvents ?

For some reason it is returning a dict instead of a list and the pagination is messed up ...  I don't understand how you handle pagination if the returned payload is a dict that contains a list.

You have to compile every list of events. 

{
    "message": "Some error",
    "pageStartAt": "2018-02-11T00:00:00.090210Z",
    "pageEndAt": "2018-02-11T00:00:00.090210Z",
    "events": [
        {
            "occurredAt": "2018-02-11T00:00:00.090210Z",
            "networkId": "N_24329156",
            "type": "association",
            "description": "802.11 association",
            "category": "80211",
            "clientId": "k74272e",
            "clientDescription": "Miles's phone",
            "clientMac": "22:33:44:55:66:77",
            "deviceSerial": "Q234-ABCD-5678",
            "deviceName": "My AP",
            "ssidNumber": 1,
            "eventData": {
                "radio": "1",
                "vap": "1",
                "client_mac": "22:33:44:55:66:77",
                "client_ip": "1.2.3.4",
                "channel": "36",
                "rssi": "12",
                "aid": "2104009183"
            }
        }
    ]
}

If I query something like : https://api.meraki.com/api/v1/networks/XXXXXXX/events?productType=appliance&perPage=3&includedEventT...

I will get 3 events and the pagination will contain a link that doesn't work at all : https://nxxx.meraki.com/api/v1/networks/xxxxx/events?includedEventTypes%5B%5D=bgp_session_unestablis...   will return 0 event. 

However if I use perPage=1000,  I get something like 100-120ish events and I still get a pagination that contains no event... 

So I did open a ticket but I'm just wondering if someone is really using this endpoint or not since it looks buggy as hell.

[rhbirkelund]

Do you set the startingAfter and endingBefore query parameters?

[RaphaelL]

Hi ,

These 2 parameters are from the 'next' link : A token used by the server to indicate the start of the page. Often this is a timestamp or an ID but it is not limited to those. This parameter should not be defined by client applications. The link for the first, last, prev, or next page in the HTTP Link header should define it.

So I'm not setting anything. Just query the first URL and follow the 'next' links until you have all the data. This works for all the other endpoints except that one.

[RicardoD]

That endpoint is supposed to return a dictionary and one of its "key:value" pairs is a list of all the events as you described.

When I'm running this code using the Python Library, I'm getting only one dictionary with 15 events (5 events perPage x 3 total_pages)

response = dashboard.networks.getNetworkEvents(network_id, productType='wireless', perPage=5, total_pages=3)

[RaphaelL]

Looking at the code I now understand why it fails in my case... 

From the SDK  : def getNetworkEvents(self, networkId: str, total_pages=1, direction='prev', event_log_end_time=None, **kwargs):

This is the first endpoint that I'm using that the pages go backwards. 

[sungod]

This call is one I have had multiple issues with, it may have improved since I last dug into it, in which case the following may not be up to date.

See https://community.meraki.com/t5/Developers-APIs/Events-same-event-different-API-return-also-call-nev...

a)

The API service itself was not complying with the documented pagination rules, I had a frustrating time with support to get this accepted, but eventually it was fixed.

b)

I'm not sure the separate issue with the Meraki Python library was ever fixed (I reported it on github but the issue was closed without it being fixed at that point, so I gave up trying), fwiw I still patch the library myself with the fix in the thread I linked just in case an eventless return still hangs the library call.

c)

I had to specify total_pages="all" to get all events, the documentation doesn't mention it, it's possible this was a side effect of the other issues, but I still add this when using the call, it doesn't hurt 🙂

For instance...

response = await aiomeraki.networks.getNetworkEvents(net['id'], productType="appliance", includedEventTypes = ["nbar_block", "cf_block", "sf_url_block", "sf_binary_block"], perPage=1000, total_pages="all")

That aside, a few comments about usage...

You do need to check the return for a non-null "message" element

If there's a message then there are no events.

The possible messages are not documented, an example of a real message is...

"message": "No matching events found between Dec 27 19:13 and Jan 27 19:13."

I use the occurredAt element to filter returned events to just the period I'm interested in.

I mainly use getNetworkEvents to get appliance security events (i.e. the *_block events), but not all security event types are returned by this call.

The other appliance security event types must be obtained with the getNetworkApplianceSecurityEvents call.

[RaphaelL]

I'm not a pro in python but I really dislike having the return info in a dict. Almost all other endpoints are returning in a list , so managing pagination is pretty easy , you just add all the list together. 

Also , setting pages=All will return a TON of logs depending on the event , having timestamp would be much easier to handle. 

I hate that endpoint.

[sungod]

Yes it's about my least favourite too.

I guess using array of dict just reflects the variation in elements for each event type, though some event types are not even consistent in the elements they contain.

Presumably the reverse pagination is intended to return the most recent events first, but I'd much rather be able to set start/end time period, overall the API is inconsistent in this sort of thing.

On the plus side, it's better than another orchestrator that I work with, which has a nasty habit of spontaneously generating millions of spurious events, one instance returned over 6GB of nonsense events last month, it makes even getNetworkEvents look good  😀