Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Register or Sign in
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
About skendric
skendric
Here to help
Member since Jan 7, 2020
Oct 18 2022
Community Record
19
Posts
5
Kudos
1
Solution
Badges
Jun 29 2022
3:07 PM
We didn't experiment with changing client assignment, so I don't know. FWIW: we replicated the same issue (MR32 quits forwarding ARP frames) for clients connecting at 2.4GHz ... we resolved this by disabling 2.4GHz network-wide --sk
... View more
Mar 22 2022
8:22 AM
OK, I have run for a least a week at a time under various combinations of code rev and Layer 2 Isolation enabled / disabled. Here are my conclusions: - The MR32, running v25.14, v26.82, or v26.83 hits a bug in which it intermittently quits forwarding ARP Frames (possibly IP frames as well). Rebooting the MR32 resolves the issue. Or, you can just wait: the issue auto-resolves after some number of hours, typically 1-10 hours - Disabling Layer 2 Isolation (we run our WAPs in Bridge mode) fixes the issue - The issue does not affect MR33 running v28.x I suspect that Meraki won't fix this bug -- the MR32 is well along its end-of-life trajectory. Time to disable Layer 2 Isolation or replace the MR32 --sk
... View more
Feb 23 2022
1:52 PM
Only (1) incident thus far, which is a substantial improvement. --sk
... View more
Feb 22 2022
11:27 AM
I have a building containing ~150 MR32/33/36. ~70 MR32. 5GHz only (we disabled 2.4GHz early this year) I believe that the MR32 sometimes quit forwarding ARPs and IP traffic - I have captured on the WiFi side; I can see ARP Requests for the IP default gateway arriving from the Client - I have captured on the Wired side: I don't see those ARP Requests being forwarded - I have (3) WiFi monitoring probes (Netscout nGeniusPulse 'nPoints'); poking through their logs, I estimate that two of them experience this "cannot send IP traffic" every day or two, for periods of ~1-5 hours, with the occasional larger window (as large as ~10 hours). The third one experiences this issue less frequently -- less than once/week I have demonstrated that rebooting the MR32 resolves the issue. Or, just waiting, per above I have a ticket open with Meraki; Support has suggested downgrading the MR32 flock from v28.5 to v25.14, which we have done, no obvious change in behavior. I suspect I have more work to do with Support. And we have a consultant joining us next week to verify my work. But in the meantime, I figured I would post here -- anyone seen this? --sk Stuart Kendrick
... View more
Labels:
- Labels:
-
Other
Sep 3 2021
3:59 AM
2 Kudos
Or syslog I have now scribbled together a reporting tool which counts which users connect on which bands (i.e.2.4 and 5) In my tiny environment (~700+ users in 2021) - 32 have only associated at 2.4GHz - 272 have only associated at 5GHz -~400+ have associated at both 2.4 and 5GHz Poking deeper at the 32 who have only ever associated at 2.4GHz ... only a single device has not only *associated* but has also *authenticated* ... and that device belongs to someone who left the company early this year ... the rest seem to be just devices that 'passed through' our building, without a need (or at least an attempt) to authenticate Sounds like it is time to disable 2.4GHz Thank you for pointing me in this direction! --sk
... View more
Aug 31 2021
3:54 PM
I seem to get 30 entries at a time ... is there some way to Download as CSV the last few thousand entries? Or is this limitation part of what you were referring to when you said 'Not a nice solution ...' 🙂 --sk
... View more
I have a flock of ~150 MR32 & MR33 contained in a single building. I have documented the issue with (3) clients: - OS X laptop - NetAlly EtherScope NG - NetAlly AirCheck2 I have anecdotal evidence of other clients being affected as well, but no pcaps to support these claims I have built up a library of simultaneous pcaps captured in the air next to the Client, at the WiFi side of the MR32, and at the Wired side of the MR32 ... showing what you report, i.e. that the DHCP frame gets dropped at the MR32. We have also tried statically assigning an IP address to the Client; at that point, we see ICMP Echos also dropped at the MR32 ... which suggests to us that when the MR32 has entered this state, it drops all WiFi-side IP frames We have tried and failed to replicate the problem as follows: (a) At 5GHz with the MR32 (b) At 2.4GHz with the MR33 OK, so your experience suggests a more narrow fingerprint than mine. But useful to me to know that at least one other customer has seen a variation on this issue -- thank you --sk
... View more
Aug 31 2021
7:11 AM
Is there a way to run a report listing all clients which have connected at 2.4GHz over the last x days? --sk
... View more
Labels:
- Labels:
-
Other
I believe that I'm seeing the following behavior. Can anyone else confirm this? Or poke holes in my story? - Some Clients seem to be able to dodge the "Dual-band operation with Band-Steering" option we have enabled, selecting 2.4GHz (perhaps because signal strength is stronger) over 5GHz - MR32s (v26.8.2) have a bug in them such that sometimes they quit forwarding DHCP (and possibly other IP protocols) - Thus, clients which are able to dodge Band-Steering, which then connect at 2.4GHz, and which associate to an MR32 which is experiencing this bug ... don't enjoy the experience Sometimes this bug clears up on its own; and rebooting the MR32 clears the bug for a 'while' (sometimes for just a few minutes) --sk
... View more
Labels:
- Labels:
-
Other
Good point, 'on the wire' is not correct -- I have edit my post above to replace 'on the wire' with 'on the air' But wait a minute, are you saying that you can search the Access Points page for a BSSID and get a hit? I have been unsuccessful in doing that ... but I'll try more times, if you tell me that you are successful. [We have <200 APs ..] --sk
... View more
Here is my recap WHAT I WANT - When I trouble-shoot with my NetAlly tools (AirCheck G2 and EtherScope NG), I want to see AP names, rather than BSSIDs - Why? Here is a use case. Client intermittently connects to AP X and succeeds and then moves to AP Y and fails. My NetAlly tools show me which AP it is connecting to ... by BSSID. Now, I want to map that BSSID back to an AP, so that I can examine that specific AP. One option would be to populate the hand-held tool with a table which maps BSSIDs to names. Another option would be to search for the BSSID in the Dashboard, retrieving the associated AP name - At the moment, there is no easy way to map BSSID back to AP, using the Dashboard UI ... no Search will find it. And I haven't had a programmatic way to build the one-to-many mapping of AP names to BSSIDs WHAT I HAVE DONE - I have asked Support to enable the AP to broadcast the name of the AP in the beacon. This only covers the SSID beaconing case; it does not cover all the times when BSSIDs appear 'on the air' (i.e. from the perspective of a radio-equipped tool) ... so it helps modestly, but still, most of the entries on a tool which displays all the 'APs' which it can see ... are BSSIDs, rather than AP names - I have opened a case with Support, and they have enabled viewing all the BSSIDs associated with an AP, from the Dashboard. But this still doesn't make them available via a Search function ... so, given a BSSID, I have to manually click on each AP and then scan the resulting list for the BSSID of interest ... not feasible - I have a "Made a Wish", asking that Search in the Access Points list support searching by BSSID TO DO - Ahh, but the Dashboard API supports the GetDeviceWirelessStatus call, which returns, amongst other things, all the BSSIDs which this particular AP is using: https://developer.cisco.com/meraki/api-v1/#!get-device-wireless-status - So, I can in theory write code to loop through my APs, emit this call, gather the result, strip out the BSSIDs, and build a table, for importation into the NetAlly tools, which will allow the NetAlly tools to display AP name instead of BSSID - Now I need to find time to write that code ... Thank you to all who helped me work through this. --sk
... View more
OK, so typing the BSSID into Tags works (so long as I remove the colons) ... but I cannot then search on it But typing the BSSID into Notes works ... although if I want to search on it, I have to enter it without the colons ... but once I've done that, i.e. entered the BSSID (sans colons) into the Notes field, then I can successfully search on it Is there a programmatic way to gather a list of all BSSIDs mapped to AP names? Or do I need to visit each AP individually carrying some hand-held tool which will list the BSSIDs broadcast from it?
... View more
Aug 25 2020
7:58 AM
Given a BSSID, how do I figure out which AP is broadcasting it? I would like to type the BSSID into some Search field in the Meraki Dashboard, and get back the AP name. Is there a way to do this?
... View more
Jan 8 2020
5:44 AM
1 Kudo
wrt to traffic shaping -- now that's an idea. In some ways, 'ip dhcp snooping limit rate 25' is a form of traffic shaping, however crude: allows a given AP (or, more precisely, the clients behind it) to emit no more than 25 DHCP Requests in a given second, and then shuts them down cold. You're suggesting a more sophisticated approach -- perhaps permit 20 DHCP Requests every second forever, as an example. Thank you -- I hadn't thought of that
... View more
Jan 8 2020
5:41 AM
wrt to increasing "ip dhcp snooping limit rate 25" to something larger, I did try 50. And then counted the number of events per week at '25' vs at '50' -- no difference Examining the one pcap I have, I can see that it emitted those 50 DHCP Requests within ~1/3 of a second. Which suggests that it can emit ~150/s. Then again, a couple of spurts during that 1/3 second see one DHCP Request every .00005s ... if it were to sustain that rate ... that translates into ~20,000 DHCP Requests / second. <aside>Normally, a client keeps its DHCP Transaction ID steady ... if the DHCP server doesn't respond, it sends another Request with the same Transaction ID. Over time, as it renews its lease, it will increment the Transaction ID by 1 (well, the examples I'm looking at currently behave this way). This pathological client, though, when it gets into this burst mode, employs random Transaction IDs, wildly random. Seems like another pointer toward bugginess to me.</aside> Still, one could make a case for increasing the threshold from 25 or 50 to, say, 200 But then, we're back to the deeper question: what happens to the Domain Controllers if one ... or more ... Clients are pounding them with DHCP Requests? I don't know -- maybe Windows handles this just fine. But this angle does seem worth considering, at least to me.
... View more
Jan 8 2020
5:35 AM
WRT to fixing buggy clients, this seems laborious to me. Seems to me that the work-flow looks something like this: - Capture DHCP traffic on the Layer 3 boxes (they have on-board sniffers ... although they quit capturing after 500,000 frames, so I have to manually restart them every day or two) - Correlate pcaps with log messages - Dig through the relevant pcaps, find the MAC address of the offending WiFi client - Search the Meraki dashboard for that MAC address, which usually results in a name ... "John's iPhone" is the only one I've done this with, thus far - Search the company directory for "John", then visit each one, asking if they have an iPhone. Hope that 'John' is a company employee, rather than a visitor. - If they do, compare the device's MAC address to the one from the pcap. If I find the right device, then investigate whether or not an OS update is available for installation, and if so, ask John to perform the upgrade - If the device is running the latest & greatest ... anyone know a way to contact Google or Apple to report such a bug? Anyway, this seems like a lot of work to me. Anyone see a less laborious approach? --sk
... View more
Jan 8 2020
5:22 AM
WRT to letting buggy clients spew lots of traffic at the DHCP Servers vs rebooting APs: It is an interesting trade-off, isn't it? I would like to know more about how the Domain Controllers would handle being pounded by DHCP Requests. Perhaps they would shrug this off just fine. On the other hand, if they became unhappy and quit performing other duties (like authentication), then perhaps better to reboot APs intermittently.
... View more
Jan 8 2020
5:19 AM
WRT investigating whether many / most / all of the clients behind a given AP have similar DHCP lease expiration dates ... that sounds like a lot of work. I would develop a 'database' of MAC addresses associated with APs ... and then correlate that with the inventory of DHCP addresses which the Domain Controllers maintain ... and then look for patterns like this one. What are you aiming for, with this suggestion?
... View more
Jan 7 2020
2:32 PM
Is anyone else seeing this issue? A few times a day ... sometimes a handful of times ... at least one of my WiFi clients starts emitting a surge of DHCP Requests -- I have in fact a pcap of 50 in a single second illustrating this behavior. Looks like a bug in the WiFi client to me. I manage a single building containing ~70 Meraki MR32/33, supporting ~750 WiFi clients per day 2020-01-07T07:20:24.528207-08:00 xxxxxx 8596: 008564: 000196: Jan 7 07:20:23.492 pst: %PM-4-ERR_DISABLE: dhcp-rate-limit error detected on Gi1/0/21, putting Gi1/0/21 in err-disable state (xxxxxx-1) 2020-01-07T07:22:25.078370-08:00 xxxxxx 8599: 008567: 000197: Jan 7 07:22:23.509 pst: %PM-4-ERR_RECOVER: Attempting to recover from dhcp-rate-limit err-disable state on Gi1/0/21 (xxxxxx-1) The access layer switches (Cisco Catalysts) have what I believe are vanilla protective mechanisms configured: interface GigabitEthernet5/0/1 switchport mode access vlan 123 storm-control broadcast level 1.00 storm-control multicast level 1.00 storm-control action shutdown storm-control action trap spanning-tree portfast edge spanning-tree guard root ip dhcp snooping limit rate 25 i.e. if broadcast traffic or multicast traffic exceeds 1% of the negotiated pipe size (typically 1000Mb/s) within a second, then the switch puts the port into err-disable. Similarly if the Catalyst sees a single port passing more than 25 DHCP transactions in a single second. And the ports servicing the Meraki WAPs have the same mechanisms installed: interface GigabitEthernet1/0/1 description Meraki AP switchport trunk native vlan 100 switchport mode trunk storm-control broadcast level 1.00 storm-control multicast level 1.00 storm-control action shutdown storm-control action trap spanning-tree portfast edge spanning-tree guard root ip dhcp snooping limit rate 25 Well, when one of these WiFi clients emits > 25 DHCP Requests in a single second, the Meraki AP forwards the first 25 of those Requests to the upstream Catalyst switch (which in turn forwards those Requests toward the DHCP Servers), and then err-disables the port. PoE shuts off, the Meraki AP goes dark. There is an automated recovery mechanism, which re-enables the port after (2) minutes ... PoE lights the AP again ... it reboots ... so within (10) minutes, the event is over, and the Meraki AP is back on-line. So this isn't a tragedy. But it happens enough that I would like to find another approach. Ideally, Meraki would implement the 'dhcp-rate-limit' function in their OS, and then the AP would automatically disassociate the client ... allowing it to re-associate perhaps a couple minutes later. I have submitted this Wish. Alternatively, I could remove the " ip dhcp snooping limit rate 25" protective mechanism and just let these buggy clients pound my DHCP Servers. Anyway, I figured I'd ask: is anyone else seeing this? --sk
... View more
My Accepted Solutions
| Subject | Views | Posted |
|---|---|---|
| 1227 | Mar 22 2022 8:22 AM |
My Top Kudoed Posts
| Subject | Kudos | Views |
|---|---|---|
| 2 | 2304 | |
| 2 | 1092 | |
| 1 | 6359 |
© 2024 Cisco Systems, Inc.
