Allow using allowlist for BLE scanner #6752

feens · 2024-05-15T01:51:47Z

What does this implement/fix?

Enables using the esp-idf BLE scanner's whitelist functionality. This is useful in cases where there's a lot of BLE devices nearby and they overwhelm the scanner, especially in active mode.

My own example is using this along with the Bluetooth Proxy component to enable the Home Assistant Yale Bluetooth integration. I have 3 locks, 2 close to one proxy, one close to another, but the extra BLE traffic can cause significant delays. I have no other BLE devices that I'm integrating at this point, so filtering down to those devices is very beneficial.

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Other

Related issue or feature (if applicable): fixes

I think this would basically complete esphome/feature-requests#2686 as well (the ask for was for the proxy, but it relies on this component for scanning anyway)

Pull request in esphome-docs with documentation (if applicable): esphome/esphome-docs#3848

Test Environment

Example entry for `config.yaml`:

# Example config.yaml
substitutions:
  name: olimex-bluetooth-proxy-00886c
  friendly_name: Bluetooth Proxy (POE)
packages:
  esphome.bluetooth-proxy: github://esphome/firmware/bluetooth-proxy/olimex-esp32-poe-iso.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
api:

esp32:
  board: esp32-poe-iso
  framework:
    type: esp-idf
    version: 4.4.7

ethernet:
  type: LAN8720
  mdc_pin: GPIO23
  mdio_pin: GPIO18
  clk_mode: GPIO17_OUT
  phy_addr: 0
  power_pin: GPIO12

logger:

ota:

dashboard_import:
  package_import_url: github://esphome/firmware/bluetooth-proxy/olimex-esp32-poe-iso.yaml@main

external_components:
  - source: github://feens/esphome_official
    refresh: 1minutes
    components: [ esp32_ble_tracker ]

esp32_ble_tracker:
  scan_parameters:
    interval: 1100ms
    window: 1100ms
    active: true
    allowlist_address: 

      - 78:9C:85:09:72:10
      - 78:9C:85:10:89:42

bluetooth_proxy:
  active: true

Checklist:

The code change is tested and works locally.
Tests have been added to verify that the new code works (under tests/ folder).

If user exposed functionality or configuration variables are added/changed:

Documentation added/updated in esphome-docs.

codecov-commenter · 2024-05-15T01:53:20Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 54.09%. Comparing base (4d8b5ed) to head (cdfbd98).
Report is 635 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #6752      +/-   ##
==========================================
+ Coverage   53.70%   54.09%   +0.38%     
==========================================
  Files          50       50              
  Lines        9408     9627     +219     
  Branches     1654     1701      +47     
==========================================
+ Hits         5053     5208     +155     
- Misses       4056     4092      +36     
- Partials      299      327      +28

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

bdraco · 2024-05-16T22:16:24Z

esphome/components/esp32_ble_tracker/esp32_ble_tracker.h

@@ -213,6 +214,15 @@ class ESP32BLETracker : public Component,
  void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) override;
  void ble_before_disabled_event_handler() override;

+  static void uint64_to_bd_addr(uint64_t address, esp_bd_addr_t bd_addr) {


We should avoid duplicating the code since this function already exists in bluetooth_proxy.h

ya...I was trying tot think of the best approach for that. Can I easily pull in the proxy one, or should the proxy one switch to referencing this one? The latter seems better as the proxy requires this component anyway.

since proxy depends on esp32_ble_tracker it probably makes sense to move it here.

I'd do that in another PR and than rebase this one on top once its merged

feens · 2024-05-17T01:26:20Z

esphome/components/esp32_ble_tracker/__init__.py

@@ -30,6 +30,9 @@
 CONF_WINDOW = "window"
 CONF_CONTINUOUS = "continuous"
 CONF_ON_SCAN_END = "on_scan_end"
+CONF_ALLOWLIST_ADDRESS = "allowlist_address"
+MAX_ALLOWLIST = 10


The max depends on which board is being used, so I picked what should be a more conservative amount.

The max allowlist is hardcoded? How does the user know about this? Any way to change this value?

Eg. Using 12 Xiaomi temp/hum sensors and a BLE lock...

@nagyrobi thus far this is the only spot where I've read anything conclusive: espressif/esp-idf#9205

Should use the max possible for every chip type, or at least make it configurable.

I don't disagree, but I don't have the time to figure out what's possible for every chip (or even know where to look). Could be a good follow-up PR. I feel like this at least adds a beneficial feature, and the risk of making it configurable is that if an incorrect value is used by the end-user it won't work properly, so this feels like safer approach to me.

Why does this feature have to depend on IDF?

@nagyrobi the methods used are for the ESP-IDF API. Arduino may have similar functionality but I can't find much info on it.

I expect they are also available on Arduino. Switching the framework might be all that is needed to verify

The esp-idf apis are all available even when using Arduino (version dependent for some things)

I'll give Arduino a try when I get a moment and adjust as necessary

feens · 2024-05-17T01:29:17Z

esphome/components/esp32_ble_tracker/esp32_ble_tracker.cpp

  esp_err_t err = esp_ble_gap_set_scan_params(&this->scan_params_);
  if (err != ESP_OK) {
    ESP_LOGE(TAG, "esp_ble_gap_set_scan_params failed: %d", err);
    return;
  }
+
+  if (!this->allowlist_address_vec_.empty() && !this->allowlist_populated_) {


I wanted to run this in setup() as it really only needs to run the first time...but then it wasn't working at all, so I'm just using a flag to save it from running on successive passes.

feens · 2024-05-20T01:01:05Z

@bdraco docs changes are up, should be ready for a proper review now.

probot-esphome bot added the integration: esp32_ble_tracker label May 15, 2024

feens mentioned this pull request May 15, 2024

August Bluetooth lock constantly disconnecting using yale Bluetooth integration. home-assistant/core#116366

Open

feens force-pushed the dev branch from ca35658 to c0bdf3f Compare May 16, 2024 01:51

bdraco reviewed May 16, 2024

View reviewed changes

feens force-pushed the dev branch 2 times, most recently from 9a098fb to a36b8fd Compare May 17, 2024 01:24

feens commented May 17, 2024

View reviewed changes

feens force-pushed the dev branch 3 times, most recently from ca9532d to 4921b8c Compare May 17, 2024 01:38

Allow using whitelist for BLE scanner

6b9efaf

feens force-pushed the dev branch from 4921b8c to 6b9efaf Compare May 17, 2024 01:46

feens mentioned this pull request May 20, 2024

Allowlist docs esphome/esphome-docs#3848

Open

3 tasks

feens marked this pull request as ready for review May 20, 2024 01:00

Merge branch 'dev' into dev

cdfbd98

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow using allowlist for BLE scanner #6752

Allow using allowlist for BLE scanner #6752

feens commented May 15, 2024 •

edited

codecov-commenter commented May 15, 2024 •

edited

bdraco May 16, 2024

feens May 17, 2024 •

edited

bdraco May 20, 2024

feens May 17, 2024

nagyrobi May 20, 2024 •

edited

feens May 21, 2024

nagyrobi May 21, 2024 •

edited

feens May 21, 2024

nagyrobi May 21, 2024

feens May 22, 2024

bdraco May 22, 2024

jesserockz May 22, 2024

feens May 22, 2024

feens May 17, 2024

feens commented May 20, 2024

Allow using allowlist for BLE scanner #6752

Are you sure you want to change the base?

Allow using allowlist for BLE scanner #6752

Conversation

feens commented May 15, 2024 • edited

What does this implement/fix?

Types of changes

Test Environment

Example entry for config.yaml:

Checklist:

codecov-commenter commented May 15, 2024 • edited

Codecov Report

Choose a reason for hiding this comment

feens May 17, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nagyrobi May 20, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nagyrobi May 21, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

feens commented May 20, 2024

feens commented May 15, 2024 •

edited

Example entry for `config.yaml`:

codecov-commenter commented May 15, 2024 •

edited

feens May 17, 2024 •

edited

nagyrobi May 20, 2024 •

edited

nagyrobi May 21, 2024 •

edited