Develop your own Bluetooth Low Energy applications

Koen Vervloesem

2022-06-12 20:54

This week my new book has been published, Develop your own Bluetooth Low Energy Applications for Raspberry Pi, ESP32 and nRF52 with Python, Arduino and Zephyr.

Bluetooth Low Energy (BLE) is one of the most accessible wireless communication standards. You don't need any expensive equipment to develop BLE devices such as wireless sensor boards, proximity beacons, or heart rate monitors. All you need is a computer or a Raspberry Pi, an ESP32 microcontroller board, or a development board with a Nordic Semiconductor nRF5 (or an equivalent BLE SoC from another manufacturer).

On the software side, BLE is similarly accessible. Many development platforms, most of them open source, offer an API (application programming interface) to assist you in developing your own BLE applications. This book shows you the ropes of Bluetooth Low Energy programming with Python and the Bleak library on a Raspberry Pi or PC, with C++ and NimBLE-Arduino on Espressif's ESP32 development boards, and with C on one of the development boards supported by the Zephyr real-time operating system, such as Nordic Semiconductor's nRF52 boards.

While Bluetooth Low Energy is a complex technology with a comprehensive specification, getting started with the basics is relatively easy. This book takes a practical approach to BLE programming to make the technology even more approachable. With a minimal amount of theory, you'll develop code right from the start. After you've completed this book, you'll know enough to create your own BLE applications.

What is Bluetooth Low Energy?

Bluetooth is a wireless communication standard in the 2.4 GHz Industrial, Scientific, and Medical (ISM) frequency band. These days, if you hear about Bluetooth support in a product, this almost always is Bluetooth Low Energy (BLE). It's a radical departure from the original Bluetooth standard, which is now called Classic Bluetooth.

Bluetooth Low Energy and Classic Bluetooth are actually different protocols. Classic Bluetooth is essentially a wireless version of the traditional serial connection. If you want to print a document, transfer a file or stream audio, you want this to happen as fast as possible. Therefore, the focus of development in Classic Bluetooth was on attaining faster and faster speeds with every new version.

However, Classic Bluetooth wasn't a good fit for devices with low power consumption, for instance those powered by batteries. That's why Nokia adapted the Bluetooth standard to enable it to work in low-power scenarios. In 2006, they released their resulting technology onto the market, dubbed Wibree.

The Bluetooth Special Interest Group (SIG), the organization that maintains the Bluetooth specifications, showed interest in this new development. After consulting with Nokia, they decided to adopt Wibree as part of Bluetooth 4.0, with a new name, Bluetooth Low Energy. Classic Bluetooth remained available for high-throughput applications.

Layered architecture

The Bluetooth Core Specification is more than 3200 pages long. And this is only the core specification; there are many supplemental documents for BLE. However, BLE has a layered architecture. Many end-user applications only use the upper layers, so you don't need to know the details of the architecture's lower layers.

The BLE architecture consists of three main blocks: controller, host, and application.

Controller: This has the lower-level layers: the Physical Layer (PHY), Link Layer (LL) and Direct Test Mode (DTM). These are the layers where the Bluetooth radio does its work. The controller communicates with the outside world using the antenna, in a frequency band around 2.4 GHz. It communicates with the host using a standardized interface between the two blocks: the Host Controller Interface (HCI). [1]
Host: This is the block with which the end user or application developer comes in contact. The Logical Link Control and Adaptation Protocol (L2CAP) defines channels and signaling commands. On top of it, the Security Manager Protocol (SMP) handles secure connections (with authentication and encryption), and the Attribute Protocol (ATT) defines how to expose and access data as attributes. The Generic Attribute Profile (GATT) [2] builds on the Attribute Protocol to define how to discover services and their characteristics and how to read and write their values. The upper layer of the Host block is the Generic Access Profile (GAP), which defines how devices can discover other devices and connect, pair, and bond to them. The host communicates with the controller using its part of the host controller interface, and applications communicate with the host depending on the APIs exposed by the operating system.
Application: This layer builds on top of the Generic Attribute Profile to implement application-specific characteristics, services, and profiles. A characteristic defines a specific type of data, such as an Alert Level. A service defines a set of characteristics and their behaviors, such as the Link Loss Service. A profile is a specification that describes how two or more devices with one or more services communicate with each other. An example is the Proximity profile, which has two roles: Proximity Monitor and Proximity Reporter.

The three blocks don't have to run on the same processor. In fact, there are three common configurations --- one single-chip and two dual-chip:

Single-chip (SoC): Controller, host and application code run on the same chip. The host and controller communicate through function calls and queues in the chip's RAM. Most simple devices such as BLE sensors use this configuration; it keeps the cost down. Some smartphones also use this configuration if they have a SoC with Bluetooth built in.
Dual-chip over HCI: A dual-chip solution with application and host on one chip, and the controller on another chip, communicates over HCI. Because HCI is a standardized interface, it lets you combine different platforms. For instance, on a Raspberry Pi, the Wi-Fi and BLE chip implements a BLE controller. If you connect a BLE dongle to an older Raspberry Pi, this dongle also implements a BLE controller. [3] BlueZ, the Raspberry Pi Linux kernel's Bluetooth stack, implements a BLE host. So BlueZ communicates with the BLE controller in the built-in BLE chip or the BLE dongle. In the former case, the HCI uses SDIO, and in the latter, UART over USB. [4] Many smartphones and tablets also use the dual-chip over HCI configuration, with a powerful processor running the host and a Bluetooth chip running the controller.
Dual-chip with connectivity device: Another dual-chip solution is one with the application running on one chip and the host and controller on another chip. The latter is then called the connectivity device because it adds BLE connectivity to the other device. This approach is useful if you have an existing hardware device that you want to extend with BLE connectivity. Because there's no standardized interface in this case, the communication between the application processor and the connectivity device needs to make use of a proprietary protocol implemented by the connectivity device.

A three-chip solution with controller, host, and application each running on its own chip is also possible. However, because of the associated cost, this is typically only done for development systems.

How to communicate with BLE devices?

Bluetooth Low Energy has two ways to communicate between devices: with and without a connection.

Without a connection

Without a connection means that the device just broadcasts information in an advertisement. Every BLE device in the neighborhood is able to receive this information.

Some examples of BLE devices broadcasting data are:

Proximity beacons: These devices, often following Apple's iBeacon standard, broadcast their ID. Receivers calculate their approximate distance to the beacons based on the advertisement's Received Signal Strength Indicator (RSSI).
Sensors: Many temperature and humidity sensors broadcast their sensor values. Most devices do this in an unencrypted fashion, but some of them encrypt the data to prevent it being read by every device in the neighborhood.
Mobile phones: After the COVID-19 pandemic started in 2020, Google and Apple collaborated on the Exposure Notifications standard for contact tracing. As part of this technology, Android phones and iPhones broadcast unique (but anonymous) numbers. Other phones can pick up these numbers and use them later to warn users that they have been in contact with someone who is known to have had COVID-19.

With a connection

The other way to communicate between BLE devices is with a connection. One device (the client) scans for BLE advertisements to find the device it wants to connect to. Then, optionally, it may do an active scan to ask the device (the server) which services are offered.

After the client connects to the server, the client can use the server's services. Each BLE service is a container of specific data from the server. You can read this data, or (with some services) write a value to the server.

Some examples of BLE devices using a connection are:

Fitness trackers: Your smartphone can connect to a fitness tracker and read your heart rate, the tracker's battery level, and other measurements.
Sensors: Some environmental sensors let you read their sensor values over a BLE connection.
Proximity reporters: These devices sound an alert when their connection to another device is lost.

Advantages of BLE

Low power consumption

As its name implies, Bluetooth Low Energy is optimized for low-power applications. Its whole architecture is designed to reduce power consumption. For instance, setting up a connection, reading or writing data, and disconnecting happens in a couple of milliseconds. The radio is often the most energy-consuming part of a device. Therefore, the idea is to turn on the Bluetooth radio, create a connection, read or write data, disconnect, and turn off the radio again until the next time the device has to communicate.

This way, a well-designed BLE temperature sensor is able to work on a coin cell for ten years or more. You can use the same approach with other wireless technologies, such as Wi-Fi, but they require more power and more time to set up a connection.

Ubiquitous

BLE radio chips are ubiquitous. You can find them in smartphones, tablets, and laptops. This means that all those devices can talk to your BLE sensors or lightbulbs. Most manufacturers create mobile apps to control their BLE devices.

You can also find BLE radios in many single-board computers, such as the Raspberry Pi, and in popular microcontroller platforms such as the ESP32. [5] This makes it quite easy for you to create your own gateways for BLE devices. And, platforms such as the Nordic Semiconductor nRF5 series of microcontrollers with BLE radio even make it possible to create your own battery-powered BLE devices.

Low cost

There's no cost to access the official BLE specifications. Moreover, BLE chips are cheap, and the available development boards (based on an nRF5 or ESP32) and Raspberry Pis are quite affordable. This means you can just start with BLE programming at minimal cost.

Disadvantages of BLE

Short range

BLE has a short range (for most devices, less than 10 meters) compared to other wireless networks, such as Zigbee, Z-Wave, and Thread. It's not a coincidence that these competitors all have a mesh architecture, in which devices can forward their neighbors' messages in order to improve range. Low-power wide area networks (LPWANs), such as LoRaWAN, Sigfox, and NB-IoT, have even longer ranges.

In 2017, the Bluetooth SIG added Bluetooth Mesh, a mesh protocol. This builds upon BLE's physical and link layers with a whole new stack above them. However, Bluetooth Mesh isn't as well-established as the core BLE protocol, at least not for home use.

Limited speed

The BLE radio has a limited transmission speed. For Bluetooth 4.2 and earlier, this is 1 Mbps, while for Bluetooth 5 and later, this can be up to 2 Mbps. This makes BLE unsuitable for high-bandwidth applications.

You need a gateway

Wi-Fi devices have their own IP addresses, so you can communicate with them directly from other IP-based devices, and they're integrated in your LAN (local area network). Bluetooth doesn't have this: to integrate your BLE devices with other network devices, you need a gateway. This device has to translate Bluetooth packets to IP-based protocols such as MQTT (Message Queuing Telemetry Transport). That's why many BLE device manufacturers have smartphone apps that function as device gateways. [6]

Platforms used in this book

This book focuses on Bluetooth Low Energy programming on three platforms:

BLE platforms used in this book
Programming language	Library	Software platform	Hardware platform
Python	Bleak	Windows, Linux, macOS	Raspberry Pi or PC
C++	NimBLE-Arduino	Arduino framework	ESP32
C	/ [7]	Zephyr	nRF52

These choices were made in order to demonstrate a wide range of applications compatible with many software and hardware platforms.

Python/Bleak (Raspberry Pi, PC)

Python is an easy-to-use programming language that works on all major operating systems. There are a lot of Python Bluetooth Low Energy libraries, but many of them support only a single operating system. Bleak, which stands for Bluetooth Low Energy platform Agnostic Klient, is a welcome exception. It supports:

Windows 10, version 16299 (Fall Creators Update) or higher
Linux distributions with BlueZ 5.43 or higher (also on a Raspberry Pi)
OS X 10.11 (El Capitan) or macOS 10.12+

Bleak is a GATT client: it's able to connect to BLE devices that act as GATT servers. It supports reading, writing, and getting notifications from GATT servers, and it's also able to discover BLE devices and read advertising data broadcast by them.

Bleak doesn't implement a GATT server. In practice this isn't a big limitation. GATT servers are typically implemented on constrained devices, so, for this purpose, the ESP32 and nRF52 hardware platforms are a better match. [8]

C++/NimBLE-Arduino (ESP32)

If you're looking at microcontrollers, the Arduino framework has become quite popular, not only for the original Arduino boards, which didn't have BLE functionality, but also on ESP32 development boards, which do.

Programming for the Arduino framework is done in a variant of C++, but the framework and many Arduino libraries hide much of C++'s complexity. Even if you only know some C (which is much less complex than C++), you'll be able to use the Arduino framework.

One of the more popular BLE libraries for Arduino on the ESP32 is NimBLE-Arduino. It's a fork of NimBLE, which is part of the Apache Mynewt real-time operating system. With NimBLE-Arduino, you can easily create your own GATT server or client.

C/Zephyr (nRF52)

For even more constrained devices, typically battery-powered, you need a specialized real-time operating system (RTOS). This book uses the Zephyr Project on nRF52840-based devices from Nordic Semiconductor. Zephyr has a completely open-source Bluetooth Low Energy stack.

Zephyr's BLE stack is highly configurable. You can build Zephyr firmware for three configuration types:

Combined build: Builds the BLE controller, BLE host, and your application for a one-chip configuration.
Host build: Builds the BLE host and your application, along with an HCI driver to let your device communicate with an external BLE controller on another chip. [9]
Controller build: Builds the BLE controller with an HCI driver to let your device communicate with an external BLE host on another chip.

With some basic knowledge of C, you can create your own BLE devices with Zephyr, such as BLE beacons, sensor boards, and proximity reporters. Zephyr has extensive documentation of its Bluetooth API, as well as a lot of ready-to-use examples that you can build upon.

Recommended hardware

You need some (affordable) hardware for the examples in this book:

The Python examples are hardware-agnostic due to the use of Bleak. So, whether you want to use a Raspberry Pi or another Linux computer, a Windows computer, or a Mac, it won't make much difference.
For the NimBLE-Arduino examples, any ESP32 development board will suffice. [10]
For the Zephyr examples, I recommend a Nordic Semiconductor nRF52840 Dongle, but you can use any BLE microcontroller supported by Zephyr. Some other manufacturers have variants of the dongle, such as the April USB Dongle 52840 (with an external antenna) or the nRF52840 MDK USB Dongle from makerdiary (with a case).

If you're really serious about BLE development with Zephyr, I also recommend Nordic Semiconductor's nRF52840 Development Kit. This is the bigger (and slightly less affordable) brother of the nRF52840 Dongle, offering more capabilities to debug your code easily. If you have one of these, you can do your initial development on it, and move your code to the nRF52840 Dongle when you've solved the most complex issues.

So, at the minimum, you need your computer, one ESP32 development board and one BLE microcontroller supported by Zephyr.