Scanning API doesn't show some devices though they are all same hardware

[littleboots]

 I'm developing an application using Scanning API and noticed it doesn't pick up some devices though they are all same hardware. I got the same 30 devices and 17 don't show up in Scanning API. Do you experience similar problems?

 The device firmware and configuration is all the same. I bought them with one shot and there would not be hardware revision difference.
 I found this on community but it looks like it's different problem https://community.meraki.com/t5/Wireless-LAN/Iphones-and-Androids-Bluetooth-not-showing-on-BLE-scann....

[Nash]

Just to confirm, what's your Meraki dashboard structure for these devices? All in the same network?

If different network, is scanning api authorized on all networks?

[littleboots]

Yes they are all on the same single network. I tried to find any common thing like BD address bytes but there is nothing.

[ThibaultH]

Hi littleboots,

Welcome in the wonderful world of the scanning API 🚀

Could you please precise the type/model of hardware you're speaking about ?

Are we speaking about Smartphones, tablets, home-made sensors, ... ?

Are all devices seen as "connected" on the dashboard ?

What are the Wi-Fi capabilities of those devices ? 802.11n, ac ?

Have a lovely day

[littleboots]

It's Xiaomi Mi Band 3. They all are visible with Dashboard API but not with Scanning API. I tried to access them with other tool like BLE scanner on iPhone and gatttool on Raspberry Pi but I don't see any difference between the ones visible with Scanning API and the invisible ones. Looks like we have the same problem with Tile but haven't dug into it.

[ThibaultH]

Are all devices (mi band 3) within the same place ?

How many AccessPoints is there in this location ? Which models ?

Which firmware is installed on your AccessPoints ? 

Which version of the scanning API are you using ?

🤘

[littleboots]

*Are all devices (mi band 3) within the same place ?
Yes they are in the same place.
*How many AccessPoints is there in this location ? Which models ?
I got 2 neworks, one got 4 APs and the other got 25APs. They are all MR33. It doesn't matter which network I place Mi Band 3, if it's invisible at one network, it's the same at the other.
*Which firmware is installed on your AccessPoints ?
It's MR 25.13.
*Which version of the scanning API are you using ?
It's 2.0.

Are you seeing the same problem in your network? If so, which device are you using?

[ThibaultH]

Fortunately I don't have this problem in my network, but trying to help you figure this out 😁

Could you first try to update your firmware to the latest stable release candidate ?

When you go on your dashboard on one Mi Band 3 that is not seen by the scanning API, what do you see on the right section within the Map ?

[littleboots]

We see firmware update a couple of time since we found this problem and they didn't take any effect. MR33s are in production and we can't update it soon.

I think you are talking about Connectivity on the screen and they are all green, which is weird because it it's green the device should be found by Scanning API.

[BrunoCarneiro]

Have you solved this issue?
I'm experiencing same thing.

[Sakul]

I have same issue. I have 15 of BLE tags (Brand: Kontakt, all tags are same model, same firmware level).

Dashboard > Wireless > Monitor > Bluetooth Clients can see all 15 tags correctly.

But Scanning API only POST out 8 tags.
So, now I buy 4 more BLE tags from different brand (Brand: AprilBeacon). Only 2 BLE tags get POST from API.

As a result, i bought a total of 19 BLE tags, and only 10 tags can work with Scanning API...I am scratching my head.

[littleboots]

Hi, I found out the reason why we are seeing this. As a conclusion it's a bug of Meraki. If the 7th bit from top of 1st hex of BD address is 0, it's visible from Scanning API but if it's 1, it's not visible. For example, I got a MEDiTAG that BD address is DF:81:64:80:6B:A4. 1st hex is DF = bin 110111"1"1, which 7th bit from top is 1, and it's not visible from Scanning API. I got another one with BD address F4:89:62:0A:6C:B3, 1st hex is F4 = bin 111101"0"0, which 7th bit from top is 0 and it's visible.
Could you check if this is applicable with your devices? You can check with =MID(HEX2BIN(MID(<BD address cell>,1,2),8),7,1) in Excel.

[colinster]

Wow that's a great catch @littleboots. Escalate your findings to the Meraki support team! 

@TonyC  

[Sakul]

Hi littleboots, 

Super.... How could you figure this out ? I checked my Tags and concur your finding. Only the Tags with "0" on 7th bit show on Scanning API. Have you openned a helpdesk ticket with Meraki, btw ?  

[littleboots]

Hi Sakul, great to hear your devices are on the same rule. I was thinking there must be something in common for devices that are not visible from Scanning API because I saw visible/invisible changes as I did factory reset which comes with BD address change. So I compared what's in common at BD address then found it.
I opened a case about this a couple month ago and will follow up, but they said I need to call them having visible and invisible devices. Now my office is locked down because of Corona virus and it's hard to do it, but will do something.

[Sakul]

Hi Littlebots... Stay Safe... I will also open a case for this. I still have the Tags (both visible and non-visible) with me so Meraki engineer can check from my Dashboard anytime.

[littleboots]

Hi Sakul, thanks and you too. Yes please open a case and let's fix this together.

[littleboots]

Hi all, I reopened a case and got an answer from Meraki support it's spec not a bug. They are saying BD address with 7th bit of 1st hex is 1 is Locally adminisrered addess(LAA) which user can alter from original BD address, Universally Administered Addresses (UAA). I explained it's not the case with Bluetooth devices, end user cannot change BD address and as a result half of devices are non-visible with Scanning API, which end up users throwing away Meraki from window. But they still insist they won't fix it since it's a feature request and all I can do is to "Make a wish" from web GUI and escalate to Meraki sales person.
Any good ideas to push them to fix this?

[colinster]

Oh no! It's a feature, no it's a bug, no it's a feature. That can be frustrating, but don't worry. Where there is a will, there is a way. Cisco likes to support their tech partners (apps.meraki.com) and will listen to your wish / bug report / feature request if you phrase it as a solid use case. You can send it to support, your sales person, and on the dashboard's wish feature. Feel free to reach out to me directly, colin@lowenberg.com if you want help with this process.

I can see this was reported long ago on the forums, and likely seen by support. Locally administered addresses are not supported by Meraki. This is why a lot of BLE devices do not appear, because they are in "privacy" mode and not using a proper MAC Address. This is to prevent the randomized "privacy" MAC addresses configured locally on the device from filling up the database and sending junk in the API. A lot of device makers are not buying MAC addresses from the registry, and instead are using locally assigned MAC addresses. Also some device makers have algorithms that use random local MAC addresses and still are able to see and track their devices.

If you have a specific use case, like tracking employee badges, typically those badges would use a proper MAC address. If your use case is tracking FitBits, you will be out of luck as the newer FitBits now randomize MACs. Cisco is not going to violate the privacy-mode features of consumer devices, and it's not feasible to identify the random MACs. 

Other forum posts:

https://community.meraki.com/t5/Wireless-LAN/Location-API-and-randomized-MAC-addresses/td-p/8443

https://community.meraki.com/t5/Wireless-LAN/Iphones-and-Androids-Bluetooth-not-showing-on-BLE-scann...

[littleboots]

Hi @colinster, thanks for your reply. I've already done "Make a wish" and escalated to Meraki sales person. Web page you mentioned are very informative, looks like Meraki engineers are living in a perfect world not a real world. It's funny Dashboard API(Wireless -> Bluetooth Client) detect LAA BD addresses though Scanning API don't.

[colinster]

Tell me about your use case! I want to know more, and sometimes I can help. calendly.com/colinster

[littleboots]

Hi @colinster, I'm trying to map out people and equipment based on Scanning API. For people I use non-Apple wearable devices and for equipment BLE tags. It's not that complex but I don't have any control for BD addresses.

[Sakul]

Hi @littleboots I created a support ticket, first the support told me to call local Meraki telephone number. But I insisted that he escalate this case to Meraki Dev team. I also create a "make a wish" request for allowing Scanning API for BLE with LAA.

Hi @colinster In my case, I am developing Asset Tracking for my customer's warehouse. I plan to deploy about 50 of BLE Tags attached to their pallets. So far, i bought 20 BLE Tags but Meraki API allow me to see only half of them.

Thanks.

[littleboots]

Hi @Sakul, I got a response back from Meraki sales person that Meraki is not going to fix this as they believe current implementation is right and benefit everyone. So for now we have to give it up for a quick fix but we need to keep on pushing because nothing is going to happen if we don't do nothing.

[Sakul]

 The good news is that Meraki BLE API version3 can now produce LAA tags. I just tested and saw it (LAA BLE tag) posted via the API. 

[littleboots]

Thanks for the great news @Sakul ! My systems are built with V2 and will try V3 on next system.