When BlueZ has connected once to a BLE device, it uses the Device Name characteristic for the device name
The title of this article may seem trivial, but it represents the conclusion I reached after being astonished for a long time while debugging some issues related to decoding Bluetooth Low Energy (BLE) advertisements that rely on the device name. Like many solutions, the actual solution seemed simple in hindsight.
For Theengs Decoder I wrote decoders for two of Shelly's devices: the ShellyBLU Button1 and the ShellyBLU Door/Window sensor. Both devices broadcast BLE advertisements using the BTHome v2 format. Before we can decode these advertisements, we first need to identify the device that is sending a specific advertisement.
In Theengs Decoder, device detection is achieved using a model condition. The Button1 model condition is as follows:
"condition":["servicedata", "=", 14, "index", 0, "40", "|", "servicedata", "=", 14, "index", 0, "44", "&", "uuid", "index", 0, "fcd2", "&", "name", "index", 0, "SBBT-002C"],
This means: the service data (an array of bytes) consists of 14 hexadecimal characters and begins with the byte 0x40 (the first firmware versions) or 0x44 (the latest firmware). Additionally, the service data UUID is fcd2 and the name starts with SBBT-002C.
The Door/Window model condition is similar:
"condition":["servicedata", "=", 28, "index", 0, "44", "&", "uuid", "index", 0, "fcd2", "&", "name", "index", 0, "SBDW-002C"],
Here, the service data consists of 28 hexadecimal characters and starts with the byte 0x44. The service data UUID is fcd2 and the name starts with SBDW-002C.
Therefore, when decoding BLE advertisements, Theengs Decoder tries all model conditions for the supported devices. If any of the conditions is true, the properties defined in the associated decoder are extracted from the advertised data.
After writing these decoders, I tested them with Theengs Gateway on my Linux laptop. Theengs Gateway uses the cross-platform Python library Bleak, which on Linux uses BlueZ, for receiving BLE advertisements. It then decodes them using Theengs Decoder and publishes the decoded properties (in this case a button push or the state of the contact sensor) to an MQTT broker. The decoders worked.
It's important to know that Theengs Decoder purely works on BLE advertisements, which are broadcasted. This is the most basic way to communicate between BLE devices. All information is extracted from these advertisements: manufacturer-specific data, service data, and the local name. [1]
However, when I later experimented with additional functionality of both devices, I connected to them. A couple of days later, I noticed that the decoders no longer worked. When I looked into it, I saw that Theengs Gateway had detected the devices with different names: Shelly Blue Button 1 and Shelly BLE DW. Naturally, the previous model conditions no longer applied, because they were checking for the names SBBT-002C and SBDW-002C, respectively. Interestingly, the same version of Theengs Gateway running on a Raspberry Pi still managed to detect and decode both devices correctly, because it recognized them by their original names. I didn't understand this difference.
It took me a while before I remembered that I had previously connected to both devices on my laptop. So then I started exploring their GATT characteristics. Each BLE device that allows connections has a Generic Access Profile (GAP) service with mandatory Device Name and Appearance characteristics. I read the Device Name characteristic from both devices, and to my surprise they were identified as Shelly Blue Button 1 and Shelly BLE DW.
So that's when I started to connect the dots. Once I had connected to both devices from my laptop, BlueZ seemed to remember their names based on the Device Name characteristic, even after disconnecting. Apparently, when BlueZ later encounters advertisements from the same Bluetooth address, it doesn't use the advertised name (Complete Local Name or Shortened Local Name data type) to identify this device. Instead, it relies on the name it had previously stored from the Device Name characteristic. As a result, when Bleak requests the local name from BlueZ, it receives the device's Device Name characteristic instead of the advertised local name. Consequently, Theengs Gateway sent the 'wrong' name to Theengs Decoder, which caused the devices to remain undetected.
You can check this with bluetoothctl:
$ bluetoothctl [bluetooth]# info 5C:C7:C1:XX:XX:XX Device 5C:C7:C1:XX:XX:XX (public) Name: Shelly Blue Button 1 Alias: Shelly Blue Button 1 Appearance: 0x8001 Paired: no Trusted: yes Blocked: no Connected: no LegacyPairing: no UUID: Generic Access Profile (00001800-0000-1000-8000-00805f9b34fb) UUID: Generic Attribute Profile (00001801-0000-1000-8000-00805f9b34fb) UUID: Device Information (0000180a-0000-1000-8000-00805f9b34fb) UUID: Vendor specific (1d14d6ee-fd63-4fa1-bfa4-8f47b42119f0) UUID: Vendor specific (de8a5aac-a99b-c315-0c80-60d4cbb51225) ManufacturerData Key: 0x0ba9 ManufacturerData Value: 01 09 00 0b 01 00 0a aa bb cc c1 c7 5c ............\ ServiceData Key: 0000fcd2-0000-1000-8000-00805f9b34fb ServiceData Value: 40 00 19 01 5a 3a 01 @...Z:. RSSI: -68 AdvertisingFlags: 06 .
Fortunately, once I had figured this out, the solution was quite simple: BlueZ needed to forget the devices. You can remove a device from BlueZ's cache with the following command:
Alternatively, you can manually delete the directory where BlueZ stores device information. First, find your Bluetooth adapter's address:
$ hciconfig hci0: Type: Primary Bus: USB BD Address: 9C:FC:E8:XX:XX:XX ACL MTU: 1021:4 SCO MTU: 96:6 UP RUNNING RX bytes:1203288 acl:31 sco:0 events:34298 errors:0 TX bytes:898242 acl:29 sco:0 commands:11700 errors:0
Then delete the following directory, based on the adapter address and the device address:
If you're curious about the cached information, first have a look at the culprit before removing the directory:
$ sudo cat /var/lib/bluetooth/9C\:FC\:E8\:XX\:XX\:XX/3C:2E:F5:XX:XX:XX/info [General] Name=Shelly BLE DW AddressType=public SupportedTechnologies=LE; Trusted=true Blocked=false Services=00001800-0000-1000-8000-00805f9b34fb;00001801-0000-1000-8000-00805f9b34fb;0000180a-0000-1000-8000-00805f9b34fb;1d14d6ee-fd63-4fa1-bfa4-8f47b42119f0;de8a5aac-a99b-c315-0c80-60d4cbb51225; Appearance=0x8001
It's this value of the Name field that BlueZ returns to Bleak, and subsequently to Theengs Gateway and Theengs Decoder, leading to the wrong detection. After removing the device from BlueZ's cache, the advertised local name was detected correctly, and the decoders worked again.
So now I'm wondering: is it possible to get the real local name advertised by a device? Bleak uses the Name property of a device in its BlueZ backend for scanning. Does BlueZ expose the local name, and could Bleak be adapted to use this instead? I couldn't find the answer to this question.