Researched the exact modules the user is building with and fixed three
concrete issues the earlier speculative firmware got wrong:
1. WiFi: confirmed the target board (Waveshare ESP32-P4-Module-DEV-KIT,
chip ESP32-P4NRW32) bridges WiFi through an onboard ESP32-C6
co-processor over a fixed 7-pin SDIO link (CLK18/CMD19/D0-14/D1-15/
D2-16/D3-17/RESET54, cross-confirmed against Espressif's own
esp-hosted-mcu docs). wifi_manager.c's esp_wifi_init()/esp_wifi_start()
calls don't need to change — esp_wifi_remote/esp_hosted provide a
drop-in-compatible API — but the component manifest (new
main/idf_component.yml) and sdkconfig.defaults were missing entirely.
2. Real pin conflict: the presence sensor's original UART pins (17/18)
directly collided with the SDIO CLK/D3 pins above — wiring it there
would have broken WiFi, the sensor, or both. Moved to GPIO4/5.
3. Swapped placeholder parts for the user's actual hardware:
- BME280 -> BMP280 (GY-BMP280 module): temp+pressure only, no humidity.
Rewrote the driver rather than just renaming it — the old code would
have read nonexistent humidity registers and reported garbage
forever. 3.3V-only wiring note added (the BME280 assumption of
5V-tolerant logic doesn't hold for this specific breakout).
- LD2410 -> RD-03E (Ai-Thinker, not Hi-Link — a different manufacturer
with a different, incompatible UART protocol). Rewrote the frame
parser against the RD-03E's actual (if less-documented) 5-byte
simple-report format. Reports numeric distance instead of a boolean,
which better fits both the hardware's actual output and the backend's
statistical anomaly detector.
README, CMakeLists.txt, and all cross-references updated to match.
616 lines
32 KiB
Markdown
616 lines
32 KiB
Markdown
# Quantumancy ESP32-P4 Sensor Node
|
||
|
||
Firmware for the paired hardware sensor node described in
|
||
[`docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md`](../../docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md)
|
||
("Workstream I — firmware, ESP-IDF C — core sensor node"). Real ESP-IDF C
|
||
(FreeRTOS-based), not Arduino, not pseudocode. Connects to the seeker's home
|
||
Wi-Fi, samples a small set of sensors, and POSTs readings to the Quantumancy
|
||
backend's `POST /api/device/telemetry` endpoint, which feeds them into the
|
||
séance's live anomaly-detection pipeline as a sixth signal source alongside
|
||
`wire`/`evp`/`radio`/`emf`.
|
||
|
||
**Target board:** [Waveshare ESP32-P4-Module-DEV-KIT](https://www.waveshare.com/esp32-p4-module-dev-kit.htm)
|
||
(chip: **ESP32-P4NRW32**, 16MB NOR flash, onboard **ESP32-C6** Wi-Fi
|
||
6/Bluetooth 5 co-processor over SDIO, USB OTG 2.0 HS, 40-pin 2×20 header
|
||
with 28 programmable GPIOs). All pin assignments and the Wi-Fi bring-up
|
||
approach below are specific to this board — see the Wi-Fi section for the
|
||
reserved SDIO pins and [Wiring / pinout](#wiring--pinout) for sensor pins.
|
||
|
||
## Honesty policy — READ THIS FIRST
|
||
|
||
> This app's whole ethos is "real signal processing on real data, and it
|
||
> says so when something is unverified."
|
||
|
||
**Nobody working on this had physical ESP32-P4 hardware, a BMP280, or an
|
||
RD-03E module to flash and test against.** Everything in this directory is
|
||
real, structurally-correct ESP-IDF C, written against ESP-IDF's documented
|
||
APIs and each sensor's public datasheet/protocol documentation, and reasoned
|
||
about carefully — but it has **never been compiled with a real ESP-IDF
|
||
toolchain, never been flashed, and never talked to real hardware.** Treat
|
||
every claim below as "should work, per the docs" rather than "confirmed
|
||
working." See [What's verified vs. not](#whats-verified-vs-not) for the
|
||
specific, itemized breakdown — this mirrors the same convention
|
||
`frontend/src/lib/sdr.ts`'s `HARDWARE PASS REQUIRED` header comment uses
|
||
elsewhere in this repo.
|
||
|
||
## Directory layout
|
||
|
||
```
|
||
firmware/esp32p4-sensor-node/
|
||
├── CMakeLists.txt top-level ESP-IDF project file
|
||
├── sdkconfig.defaults seed config (idf.py generates the real sdkconfig)
|
||
├── README.md this file
|
||
├── components/
|
||
│ ├── README.md
|
||
│ └── rtlsdr_experimental/ Workstream J — opt-in USB-host RTL-SDR module
|
||
└── main/
|
||
├── CMakeLists.txt component registration
|
||
├── idf_component.yml managed deps: esp_wifi_remote, esp_hosted
|
||
├── app_main.c entry point / boot sequence
|
||
├── device_config.h.example template you copy to device_config.h
|
||
├── wifi_manager.{h,c} Wi-Fi station mode connect/reconnect (via
|
||
│ the onboard ESP32-C6 co-processor — see below)
|
||
├── telemetry_client.{h,c} HTTP POST task -> /api/device/telemetry
|
||
├── sensor_driver.h the sensor_driver_t registry interface
|
||
├── sensor_registry.{h,c} the concrete list of compiled-in drivers
|
||
├── bmp280.{h,c} temperature/pressure over I2C
|
||
└── rd03e.{h,c} presence/distance/gesture over UART
|
||
```
|
||
|
||
## Build instructions
|
||
|
||
Requires an ESP-IDF install (v5.3 or newer — ESP32-P4 target support landed
|
||
around v5.2/5.3; this was written without a toolchain available to pin an
|
||
exact tested version, see honesty section). With `idf.py` on your `PATH`
|
||
(e.g. after sourcing ESP-IDF's `export.sh`):
|
||
|
||
```sh
|
||
cd firmware/esp32p4-sensor-node
|
||
|
||
# 1. Fill in your Wi-Fi + pairing details (see next section) — the build
|
||
# will fail on a missing #include until you do this.
|
||
cp main/device_config.h.example main/device_config.h
|
||
$EDITOR main/device_config.h
|
||
|
||
# 2. Target and build.
|
||
idf.py set-target esp32p4
|
||
idf.py build
|
||
|
||
# 3. Flash + monitor (adjust the port for your machine).
|
||
idf.py -p /dev/ttyUSB0 flash monitor
|
||
```
|
||
|
||
## Manual configuration (no provisioning UI — by design)
|
||
|
||
A full BLE/Wi-Fi-AP provisioning flow is explicitly out of scope for this
|
||
spec (see the spec's "Explicitly out of scope" section). Instead, you
|
||
hand-edit one header before building:
|
||
|
||
1. In the Quantumancy web app, sign in and create a device from your
|
||
account (name + optional sensor-type hint). The backend shows you a
|
||
**raw pairing token exactly once** — copy it immediately, it cannot be
|
||
retrieved again (same one-time-secret convention as the site's session
|
||
tokens).
|
||
2. `cp main/device_config.h.example main/device_config.h`
|
||
3. Edit `main/device_config.h` and fill in:
|
||
- `DEVICE_WIFI_SSID` / `DEVICE_WIFI_PASSWORD` — your home Wi-Fi.
|
||
- `DEVICE_BACKEND_BASE_URL` — the backend's base URL, no trailing slash.
|
||
- `DEVICE_PAIRING_TOKEN` — the raw token from step 1.
|
||
- `DEVICE_REPORT_INTERVAL_SEC` — optional, defaults to 15s.
|
||
4. `main/device_config.h` is listed in `.gitignore` — it will never be
|
||
committed. Never put real credentials in `device_config.h.example`
|
||
itself; that file is the template everyone else copies.
|
||
|
||
There is deliberately no other config path (no NVS-based captive portal, no
|
||
BLE provisioning) in this build — see the spec's scope boundary.
|
||
|
||
## What's verified vs. not
|
||
|
||
**Structurally verified** (reasoned through carefully against ESP-IDF's
|
||
documented API surface and each sensor's public protocol docs; internally
|
||
consistent; no known syntax errors or obviously-wrong API usage):
|
||
|
||
- Project skeleton (`CMakeLists.txt` × 2, `sdkconfig.defaults`,
|
||
`idf_component_register` call) follows ESP-IDF's standard project layout.
|
||
- Wi-Fi station-mode connect/reconnect (`wifi_manager.c`) follows ESP-IDF's
|
||
documented event-driven pattern (`WIFI_EVENT`/`IP_EVENT` handlers +
|
||
`EventGroupHandle_t`), extended with an exponential-backoff reconnect
|
||
timer instead of giving up after N tries.
|
||
- HTTP client (`telemetry_client.c`) builds the exact JSON shape the spec's
|
||
contract defines and POSTs it via `esp_http_client` with
|
||
`Authorization: Bearer <token>` and `Content-Type: application/json`.
|
||
- BMP280 driver (`bmp280.c`): register map and the double-precision
|
||
compensation formulas are transcribed from Bosch's public BMP280
|
||
datasheet (rev 1.23, §3.11.1–3.11.3) — this is well-trodden, publicly
|
||
documented territory, and the formulas are checkable line-by-line against
|
||
the datasheet. Uses ESP-IDF's newer `driver/i2c_master.h` API (the
|
||
current idiomatic choice; the older `driver/i2c.h` is being phased out).
|
||
Temperature + pressure only — the part in use (a GY-BMP280 breakout) has
|
||
no humidity sensor, unlike its BME280 sibling.
|
||
- RD-03E driver (`rd03e.c`): the 5-byte "simple report" UART frame
|
||
(`0xAA` header, gesture byte, little-endian distance, `0x55 0x55`
|
||
footer) is reconstructed from a third-party bring-up write-up, not
|
||
Ai-Thinker's own datasheet (not available while writing this) — **the
|
||
single least-certain piece of code in this entire firmware.** The
|
||
gesture byte's exact value-to-meaning mapping is unconfirmed, so the
|
||
driver reports it as a raw code in `metadata` rather than guessing at a
|
||
translated label. Cross-confirmed from multiple sources: 256000 baud,
|
||
8N1 UART framing.
|
||
- Sensor driver registry (`sensor_driver.h`, `sensor_registry.c`): a
|
||
`sensor_driver_t { name, init, read }` struct, a compile-time array of
|
||
them, and generic init/collect functions that `app_main.c` and
|
||
`telemetry_client.c` call without knowing which concrete sensors exist.
|
||
|
||
**NOT verified — requires real hardware bring-up:**
|
||
|
||
- `idf.py build` has never actually been run in this environment (no
|
||
ESP-IDF toolchain installed here) — there could be a typo, a missing
|
||
include, or an API signature mismatch against whatever exact ESP-IDF
|
||
version you build with that only a real compile will surface.
|
||
- I2C timing/electricals: pull-up resistor values, bus speed headroom,
|
||
cable length — none of this has been bench-tested.
|
||
- BMP280 compensation formula correctness in practice: the math is
|
||
transcribed carefully, but "matches the datasheet" and "produces a
|
||
plausible number when this exact C runs on this exact silicon" are
|
||
different claims until someone compares a real reading to a reference
|
||
thermometer/barometer.
|
||
- RD-03E frame parsing, as above — this one especially, since the frame
|
||
format itself (not just the implementation) is reconstructed from a
|
||
third-party source rather than an official datasheet. Verify against a
|
||
logic analyzer capture before trusting field values, and treat the
|
||
gesture code's meaning as genuinely unknown until cross-checked.
|
||
- Wi-Fi reconnect behavior under real-world conditions (router reboot,
|
||
weak signal, captive portals) — the backoff logic is reasoned about, not
|
||
soak-tested.
|
||
- HTTP client behavior against the real backend: TLS handshake against
|
||
its actual certificate, real latency, real error responses. The
|
||
`sdkconfig.defaults` enables mbedTLS's full certificate bundle for this,
|
||
but that's untested against the live deploy.
|
||
- Timing/power: task stack sizes (`telemetry_task`'s 8192 words, etc.) are
|
||
reasonable guesses, not measured high-water-marks from a real run.
|
||
- Everything under [Wiring / pinout](#wiring--pinout) and the Wi-Fi section
|
||
below — confirmed against the actual target board's published specs, but
|
||
never checked against the physical hardware itself.
|
||
|
||
## Wi-Fi: ESP32-P4 has no integrated radio — confirmed target hardware
|
||
|
||
The ESP32-P4 SoC has **no built-in 2.4GHz radio**. This was flagged
|
||
speculatively in an earlier pass; it's now confirmed against the actual
|
||
target hardware — **Waveshare ESP32-P4-Module-DEV-KIT**, chip
|
||
**ESP32-P4NRW32** — which solves this the way Espressif's own reference
|
||
design does: an onboard **ESP32-C6** co-processor, wired to the P4 over a
|
||
**fixed 7-pin SDIO link**, running Wi-Fi 6 + Bluetooth 5 on the seeker's
|
||
behalf. These 7 pins are physically fixed by the board's own PCB routing —
|
||
**do not reuse them for sensors**:
|
||
|
||
| Signal | GPIO |
|
||
|---|---|
|
||
| SDIO CLK | 18 |
|
||
| SDIO CMD | 19 |
|
||
| SDIO D0 | 14 |
|
||
| SDIO D1 | 15 |
|
||
| SDIO D2 | 16 |
|
||
| SDIO D3 | 17 |
|
||
| C6 Reset | 54 |
|
||
|
||
(Sourced from Waveshare's own documentation and independently cross-checked
|
||
against Espressif's `esp-hosted-mcu` reference docs for their nearly
|
||
identical ESP32-P4-Function-EV-Board, which uses the same 7 pins — strong
|
||
agreement across two independent sources, though still not a substitute for
|
||
checking the physical board's silkscreen.)
|
||
|
||
Wi-Fi is brought up via two managed components — `espressif/esp_wifi_remote`
|
||
and `espressif/esp_hosted` — declared in `main/idf_component.yml` and
|
||
configured in `sdkconfig.defaults` (`CONFIG_SLAVE_IDF_TARGET_ESP32C6`,
|
||
`CONFIG_ESP_HOSTED_CP_TARGET_ESP32C6`, `CONFIG_ESP_HOSTED_P4_DEV_BOARD_FUNC_BOARD`,
|
||
plus buffer/window tuning). Critically, these components provide a
|
||
**drop-in-compatible** `esp_wifi_*`/`esp_netif_*` API — `wifi_manager.c`'s
|
||
`esp_wifi_init()`/`esp_wifi_start()` calls are exactly what they'd be on a
|
||
Wi-Fi-native chip; only the component manifest and sdkconfig differ, not
|
||
the application code. `idf.py build` will need network access the first
|
||
time to fetch these two components from the ESP Component Registry.
|
||
|
||
**What's still unverified here specifically:** whether
|
||
`CONFIG_ESP_HOSTED_P4_DEV_BOARD_FUNC_BOARD` (a preset named for Espressif's
|
||
own eval board) configures correctly for Waveshare's board given they
|
||
share the same SDIO pin assignment — likely fine, but the first thing to
|
||
check in `idf.py menuconfig` if Wi-Fi bring-up misbehaves is the Wi-Fi
|
||
Remote / ESP-Hosted config screens directly rather than trusting the preset
|
||
blindly.
|
||
|
||
## Wiring / pinout
|
||
|
||
Confirmed against the target board (Waveshare ESP32-P4-Module-DEV-KIT):
|
||
GPIOs 14-19 and 54 are reserved for the onboard Wi-Fi co-processor's SDIO
|
||
link (see the Wi-Fi section above) — never wire a sensor to those pins on
|
||
this board.
|
||
|
||
### BMP280 (I2C) — temperature, pressure
|
||
|
||
Confirmed against the actual part in use: a **GY-BMP280** breakout module
|
||
(Bosch BMP280 — temperature + pressure only, no humidity). GPIO8/9 don't
|
||
conflict with the board's reserved SDIO range. **3.3V only** — the
|
||
GY-BMP280 module is not 5V tolerant, unlike the RD-03E below.
|
||
|
||
| BMP280 pin | Connects to |
|
||
|------------|---------------------------------------|
|
||
| VCC | 3V3 (3.3V ONLY — do not connect to 5V)|
|
||
| GND | GND |
|
||
| SDA | GPIO8 (`BMP280_I2C_SDA_GPIO`) |
|
||
| SCL | GPIO9 (`BMP280_I2C_SCL_GPIO`) |
|
||
| CSB | VCC (selects I2C mode, not SPI) |
|
||
| SDO | GND → I2C address `0x76` (default assumed; tie to VCC + change `BMP280_I2C_ADDR` for `0x77`) |
|
||
|
||
GPIO numbers are `#define`s at the top of `bmp280.h` — override them there
|
||
(or via a future `idf.py menuconfig` entry) to match your actual wiring.
|
||
100kHz I2C clock by default (`BMP280_I2C_CLK_HZ`); the part supports faster
|
||
modes if your wiring/pull-ups support it.
|
||
|
||
### RD-03E (UART) — presence, distance, gesture
|
||
|
||
Confirmed against the actual part in use: an **Ai-Thinker RD-03E** 24GHz
|
||
mmWave radar ("Human Gesture Recognition... Precise Ranging & Positioning
|
||
Radar Sensor Module"). Originally speculatively planned as a Hi-Link
|
||
LD2410 — a **different manufacturer with a different, incompatible UART
|
||
protocol** — so this driver was rewritten from scratch against the RD-03E's
|
||
actual (if less-documented) frame format rather than adapted from the
|
||
LD2410 code. Reports ranged distance (not just a boolean), fitting a
|
||
handheld "sense a presence at a distance" device well — see the honesty
|
||
note in [What's verified vs. not](#whats-verified-vs-not) about the frame
|
||
parser being the single least-certain code in this firmware.
|
||
|
||
| RD-03E pin | Connects to |
|
||
|------------|----------------------------------------|
|
||
| VCC | 5V (module power; UART logic is 0-3.3V, ESP32-safe) |
|
||
| GND | GND |
|
||
| OT1 | GPIO4 (`RD03E_UART_RX_GPIO`, ESP32 RX) — module's UART TX output |
|
||
| RX | GPIO5 (`RD03E_UART_TX_GPIO`, ESP32 TX) — module's UART RX input |
|
||
| OT2 | not connected (reserved on the module, unused here) |
|
||
|
||
(GPIO4/5 avoid this board's reserved SDIO pins and the BMP280's I2C pins —
|
||
a reasonable, currently-unused pick, not verified against the physical
|
||
board's full pin map beyond confirming it's outside those known-reserved
|
||
ranges.)
|
||
|
||
Default UART settings: 256000 baud, 8N1 (module factory default), reading
|
||
only the module's free-running "simple report" frames — no configuration
|
||
handshake is sent or required. GPIO numbers and baud rate are `#define`s
|
||
at the top of `rd03e.h`.
|
||
|
||
## Sensor driver registry — the extensibility pattern
|
||
|
||
`sensor_driver.h` defines:
|
||
|
||
```c
|
||
typedef struct {
|
||
char sensor_type[SENSOR_READING_TYPE_MAXLEN];
|
||
double value;
|
||
char unit[SENSOR_READING_UNIT_MAXLEN];
|
||
cJSON *metadata; // nullable; NULL serializes as {}
|
||
} sensor_reading_t;
|
||
|
||
typedef struct sensor_driver {
|
||
const char *name;
|
||
esp_err_t (*init)(void);
|
||
esp_err_t (*read)(sensor_reading_t *out, size_t max_out, size_t *out_count);
|
||
} sensor_driver_t;
|
||
```
|
||
|
||
`sensor_registry.c` holds a compile-time array of these (currently BMP280
|
||
and RD-03E) and two generic functions, `sensor_registry_init_all()` and
|
||
`sensor_registry_collect()`, that `app_main.c` and `telemetry_client.c`
|
||
call without ever referencing `bmp280.c`/`rd03e.c` directly. One driver
|
||
failing `init()` or `read()` is logged and skipped — it doesn't take the
|
||
whole node offline.
|
||
|
||
### Adding a new sensor
|
||
|
||
1. Write `main/my_sensor.h` / `main/my_sensor.c` implementing `init()` and
|
||
`read()` matching `sensor_driver_t`'s function pointer signatures.
|
||
2. Add `"my_sensor.c"` to the `SRCS` list in `main/CMakeLists.txt`.
|
||
3. `#include "my_sensor.h"` in `sensor_registry.c` and add one line to the
|
||
`s_drivers[]` array:
|
||
```c
|
||
{ .name = "my_sensor", .init = my_sensor_init, .read = my_sensor_read },
|
||
```
|
||
|
||
Nothing in `app_main.c`, `telemetry_client.c`, or the main reporting loop's
|
||
control flow needs to change — that's the whole point of this structure per
|
||
the spec.
|
||
|
||
## Backend contract this firmware targets
|
||
|
||
From the spec (binding, see the spec file for the authoritative version):
|
||
|
||
```json
|
||
POST /api/device/telemetry
|
||
Authorization: Bearer <raw pairing token>
|
||
Content-Type: application/json
|
||
|
||
{
|
||
"readings": [
|
||
{"sensor_type": "presence", "value": 1, "unit": "bool", "metadata": {}},
|
||
{"sensor_type": "temperature", "value": 21.4, "unit": "c", "metadata": {}},
|
||
{"sensor_type": "humidity", "value": 47.2, "unit": "pct", "metadata": {}},
|
||
{"sensor_type": "pressure", "value": 1013.2, "unit": "hpa", "metadata": {}}
|
||
]
|
||
}
|
||
```
|
||
|
||
This firmware's BMP280 driver emits `temperature`/`pressure` (no
|
||
`humidity` — see above); its RD-03E driver emits `presence` as a numeric
|
||
distance in centimeters (`unit: "cm"`, not a `0`/`1` boolean) so the
|
||
backend's statistical anomaly detector can treat "something got suddenly
|
||
close" as the signal, with the module's raw gesture code folded into
|
||
`metadata` (`gesture_code`) rather than a boolean transition detector.
|
||
|
||
## Out of scope here
|
||
|
||
Per the spec: thermal camera support, a full BLE/Wi-Fi-AP provisioning UX,
|
||
on-device spectrum analysis/FFT (the RTL-SDR module below forwards raw IQ
|
||
upstream rather than analyzing on-device), and anything on the
|
||
backend/frontend side (Workstreams G, H, K).
|
||
|
||
---
|
||
|
||
## Workstream J — RTL-SDR experimental module
|
||
|
||
**⚠️ This is the least certain part of the entire firmware build. Read
|
||
this whole section before touching it.**
|
||
|
||
### What this is
|
||
|
||
A clearly-separated, opt-in, experimental module exploring USB-host
|
||
communication with an RTL2832U-based SDR dongle over the ESP32-P4's
|
||
native USB-OTG host controller (USB Host Library, `usb_host.h` — a real,
|
||
documented capability of this specific chip, unlike most ESP32 variants).
|
||
The idea, per the design spec: attach a cheap RTL-SDR dongle to the
|
||
sensor node, treat "spirit radio scanning" as a hardware-backed mode
|
||
instead of only a browser WebUSB feature.
|
||
|
||
**Be honest about the real constraint (this is the spec's framing, and
|
||
it's correct):** wideband IQ sample rates and FFT processing are
|
||
demanding relative to an MCU's compute, even one with ESP32-P4's AI
|
||
accelerator. **On-device spectrum analysis is not what this module
|
||
attempts.** The realistic architecture — and the one implemented here —
|
||
is: pull raw IQ samples off the dongle via USB host, and forward them
|
||
upstream for the *backend* to FFT/analyze (the same job the browser
|
||
already does client-side today via `frontend/src/lib/fft.ts` +
|
||
`frontend/src/lib/sdr.ts`'s `SpectrumAnomalyDetector`). Even that
|
||
"just pass the bytes through" architecture needs real USB throughput
|
||
numbers to know if it's viable — see "What's unverified" below.
|
||
|
||
### Primary reference
|
||
|
||
`frontend/src/lib/sdr.ts` — this repo's existing browser-based (WebUSB)
|
||
RTL2832U + R820T driver. It's real, already-researched protocol detail
|
||
against the public librtlsdr register documentation (vendor commands,
|
||
I2C-repeater tuner access, the demod/tuner init sequence), itself marked
|
||
`HARDWARE PASS REQUIRED` since it's never been run against a real dongle
|
||
either. This firmware module is a direct **port** of that file's control-
|
||
transfer sequence from WebUSB JS calls to ESP-IDF USB Host Library C
|
||
calls — line-by-line correspondences are called out in code comments
|
||
(e.g. `rtlsdr_run_init_sequence()` mirrors `open()` in sdr.ts almost
|
||
register-for-register). It intentionally does not re-derive any register
|
||
math from scratch; wherever sdr.ts says "simplified" or "HARDWARE PASS
|
||
REQUIRED" (e.g. the R820T PLL frequency math, the demod resample-ratio
|
||
math), this module carries the exact same simplification forward with the
|
||
exact same caveat, rather than inventing new unverified math on top of
|
||
already-unverified math.
|
||
|
||
### What's implemented (structurally — see caveats below)
|
||
|
||
1. **USB Host Library lifecycle** (`rtlsdr_exp_start()` /
|
||
`rtlsdr_usb_lib_daemon_task()` / `rtlsdr_exp_client_task()`):
|
||
`usb_host_install()`, `usb_host_client_register()` with an async event
|
||
callback, and the two-task pump pattern the USB Host Library's async
|
||
model requires (one task for `usb_host_lib_handle_events()`, one for
|
||
`usb_host_client_handle_events()` — the latter also being how this
|
||
module's own control- and bulk-transfer completion callbacks get
|
||
dispatched, since the Library calls them synchronously from whichever
|
||
task is pumping events, not from a hidden thread or ISR).
|
||
2. **RTL2832U/Terratec device enumeration and vendor-ID matching**
|
||
(`rtlsdr_try_bring_up()`, `rtlsdr_vendor_id_matches()`): on a
|
||
`USB_HOST_CLIENT_EVENT_NEW_DEV` event, opens the device, reads its
|
||
device descriptor, and matches `idVendor` against `0x0BDA`
|
||
(RTL2832U) / `0x0CCD` (Terratec-rebadged) — ported directly from
|
||
`sdr.ts`'s `requestDevice()` filter list (`RTL2832U_VENDOR`,
|
||
`TERRATEC_VENDOR`). Faithfully carries over that file's specific
|
||
choice to filter by **vendor ID only**, not product ID (its comment:
|
||
"many dongles report product ids outside the handful we know, so
|
||
filtering by productId hides them from the picker") — the known
|
||
product-id list (`RTLSDR_EXP_KNOWN_PRODUCT_IDS`) is kept as an
|
||
informational log line only, never a hard filter, exactly mirroring
|
||
how `RTL2832U_PRODUCTS` is exported-but-unused-as-a-filter in
|
||
`sdr.ts`.
|
||
3. **RTL2832U init vendor-command sequence** over USB control transfers
|
||
(`rtlsdr_run_init_sequence()`, `rtlsdr_demod_write()`,
|
||
`rtlsdr_reg_write()`, `rtlsdr_i2c_write()`, `rtlsdr_exp_set_frequency()`,
|
||
`rtlsdr_exp_set_sample_rate()`): the same demod soft-reset → demod_ctl/
|
||
suspend-off block → standby-off → AGC-mode → R820T tuner power-up
|
||
(through the I2C repeater) → sample-rate program → initial tune →
|
||
streaming-endpoint reset → test-mode-off sequence as `sdr.ts`'s
|
||
`open()`, with the same register addresses/values and the same
|
||
`wValue`/`wIndex` encoding (`(block<<8)|0x10` / `(page<<8)|address`),
|
||
translated from `USBDevice.controlTransferOut()` to
|
||
`usb_host_transfer_submit_control()` with a manually-built
|
||
`usb_setup_packet_t`.
|
||
4. **A basic bulk-transfer read loop structure** for pulling IQ sample
|
||
data off the device (`rtlsdr_start_bulk_streaming()`,
|
||
`rtlsdr_bulk_xfer_cb()`): unlike `sdr.ts` (which does one-shot
|
||
`await dev.transferIn(...)` calls from inside its own async `sweep()`
|
||
loop), the USB Host Library is callback-driven, so continuous
|
||
streaming here is a small pipeline — N transfers (`bulk_read_queue_
|
||
depth`, default 4) of a fixed chunk size (`bulk_read_chunk_bytes`,
|
||
default 16384B, enforced as a multiple of 512 the same way `sdr.ts`'s
|
||
`readSamples()` comment requires) are kept perpetually in flight; each
|
||
completion callback copies the received bytes into a heap block, hands
|
||
it to a FreeRTOS queue for the forward task, and immediately resubmits
|
||
itself to keep the pipe full. **This is the part most likely to need
|
||
real-hardware tuning** — see below.
|
||
5. **A stub/structure for forwarding raw IQ data upstream**
|
||
(`rtlsdr_exp_forward_iq_block()`, `rtlsdr_forward_task()`) — see the
|
||
architecture decision writeup immediately below.
|
||
|
||
### IQ-forwarding architecture decision
|
||
|
||
The spec explicitly leaves this as a product-judgment call ("your call on
|
||
whether this warrants a separate endpoint/stream vs. reusing the sensor
|
||
telemetry shape with a binary/base64 payload"). Decision made here:
|
||
**a separate binary stream/endpoint, not a reuse of the JSON telemetry
|
||
shape.** Reasoning:
|
||
|
||
- **Raw throughput is the wrong order of magnitude for the telemetry
|
||
contract.** Even a modest 2.048 Msps capture at 8 bits/sample/channel
|
||
(RTL2832U's native ADC format) is interleaved I/Q bytes at roughly
|
||
4.1 MB/s. The design spec's `POST /api/device/telemetry` contract caps
|
||
the `readings` array at ~64 entries, caps the request body at ~16KB,
|
||
and rate-limits ingestion to roughly 1 request/second sustained per
|
||
device — sized for a handful of scalar sensor readings (temperature,
|
||
humidity, presence booleans), not a continuous multi-megabyte/second
|
||
binary stream. Forcing IQ data through that shape would mean either
|
||
violating those caps (defeating their whole purpose — bounding what a
|
||
misbehaving/malicious device can push at the backend) or chopping IQ
|
||
into thousands of tiny requests per second, which is worse for both
|
||
sides than one continuous stream.
|
||
- **Base64-in-JSON adds ~33% size overhead** on top of a payload that's
|
||
already too large for the telemetry shape, for no benefit — there's no
|
||
reason to pay text-encoding tax on a payload nothing needs to
|
||
eyeball as text.
|
||
- **It's a fundamentally different kind of data with different backend
|
||
handling needs.** Telemetry readings feed the anomaly-detection
|
||
baseline/threshold pipeline directly and cheaply (a few floats per
|
||
sensor). IQ data needs FFT processing before it's useful for anything
|
||
— an entirely different backend code path (closer to how the browser's
|
||
own `powerSpectrumDb()` + `SpectrumAnomalyDetector` work today,
|
||
server-side instead of client-side). Conflating the two request shapes
|
||
would couple two things that should scale, rate-limit, and fail
|
||
independently.
|
||
|
||
Given that, this module's stub (`rtlsdr_exp_forward_iq_block()`) targets
|
||
a **separate, currently-hypothetical** endpoint (`iq_upload_url` in
|
||
`rtlsdr_exp_config_t`, suggested path `/api/device/iq-stream` in code
|
||
comments) carrying a small fixed binary header (magic, sequence number,
|
||
center frequency, sample rate, payload length — see
|
||
`rtlsdr_iq_chunk_header_t`) followed by the raw IQ bytes, POSTed as
|
||
`application/octet-stream` with the same `Authorization: Bearer <device
|
||
token>` auth as the regular telemetry loop. **This endpoint does not
|
||
exist anywhere in this repo.** Building it is explicitly out of scope for
|
||
this workstream (it's backend work, not firmware, and the spec doesn't
|
||
assign a backend workstream to receive IQ data at all — only to receive
|
||
scalar telemetry). The stub is inert by default (`rtlsdr_exp_forward_
|
||
iq_block()` returns `ESP_OK` immediately unless `iq_upload_url` is
|
||
configured) specifically so this module can be compiled/enabled for its
|
||
USB-host/bring-up behavior without requiring a backend that doesn't
|
||
exist.
|
||
|
||
A real product decision here — genuinely open, not resolved by this
|
||
workstream — is whether a POST-per-chunk model is even right versus a
|
||
persistent WebSocket stream (the spec's `/ws/device-feed` design already
|
||
establishes a WS pub/sub pattern on the backend for telemetry; a
|
||
`/ws/device-iq` sibling might fit that architecture better than repeated
|
||
HTTP POSTs, especially if backpressure/flow-control matters, which for a
|
||
continuous stream it very much does). That's flagged here rather than
|
||
silently decided, because it depends on backend design judgment as much
|
||
as firmware judgment.
|
||
|
||
### What's unverified — an honest, specific list for real hardware bring-up
|
||
|
||
Nothing below has run against real hardware. In rough order of "most
|
||
likely to break first":
|
||
|
||
1. **USB Host Library API surface.** Function names, struct field names,
|
||
and callback signatures (`usb_host_client_config_t`'s `.async.
|
||
client_event_callback` shape in particular — ESP-IDF has changed this
|
||
API's shape across versions) are written from documented/remembered
|
||
API shape, not checked against a real ESP-IDF checkout (none available
|
||
in this environment). **First thing a real bring-up needs: does this
|
||
even compile against the ESP-IDF version Workstream I's project
|
||
targets?**
|
||
2. **RTL2832U enumeration over a real ESP32-P4 USB-OTG host port** — does
|
||
`usb_host_device_open()` / `usb_host_get_device_descriptor()` actually
|
||
see the dongle at all on this specific chip's USB-OTG controller (power
|
||
delivery to the dongle over USB-host mode is itself a hardware
|
||
question — does the ESP32-P4 dev board supply VBUS in host mode, does
|
||
the dongle draw more current than it can supply).
|
||
3. **The init vendor-command sequence itself** — carried over unchanged
|
||
from `sdr.ts`, which is *itself* unverified. Two layers of "reasoned,
|
||
not tested." A logic analyzer / USB protocol analyzer trace against a
|
||
real dongle (or cross-checking against real librtlsdr `-T` verbose
|
||
output) is needed to confirm register values, not just transfer
|
||
plumbing.
|
||
4. **R820T PLL frequency math and demod resample-ratio math** — both are
|
||
the same simplified integer-N approximation `sdr.ts` uses (its own
|
||
comment: "real librtlsdr computes the exact sdm/vco from a 28.8MHz
|
||
crystal reference. Simplified... Marked for hardware."). Likely wrong
|
||
or imprecise until replaced with the real librtlsdr formula and
|
||
checked against an actual received signal.
|
||
5. **Bulk transfer chunk size, pipeline depth, and timeout=0 choice**
|
||
(`RTLSDR_EXP_BULK_CHUNK_BYTES`/`RTLSDR_EXP_BULK_QUEUE_DEPTH` Kconfig,
|
||
defaults 16384B / depth 4) — these are guesses. Real ESP32-P4 USB
|
||
Host Library heap/DMA limits, achievable sustained throughput at
|
||
2.048 Msps (~4.1 MB/s), and whether `timeout_ms = 0` (no timeout, i.e.
|
||
"block until data or disconnect") is even the right transfer mode for
|
||
this endpoint all need real measurement.
|
||
6. **Drop-on-full backpressure policy** (`iq_blocks_dropped` stat) — is
|
||
silently dropping IQ blocks when the forward task falls behind
|
||
acceptable, or does it need real flow control (e.g. throttling the
|
||
bulk read rate itself)? Depends on (5) and on real WiFi uplink
|
||
bandwidth from a real device on a real seeker's home network.
|
||
7. **The proposed IQ-forwarding wire format and endpoint** — entirely
|
||
hypothetical (see architecture section above); no backend exists to
|
||
validate the framing against, and the POST-vs-WebSocket question above
|
||
is unresolved.
|
||
8. **Concurrent USB Host Library ownership with the rest of the firmware
|
||
project.** `rtlsdr_exp_start()`/`rtlsdr_exp_stop()` call `usb_host_
|
||
install()`/`usb_host_uninstall()` directly, assuming this module is the
|
||
*only* USB Host client in the project. If Workstream I's skeleton (or
|
||
anything else) also needs USB host for something, this needs to change
|
||
to a shared-ownership model (install once, both modules register as
|
||
clients) — impossible to resolve without seeing that code, flagged
|
||
here for whoever does the merge.
|
||
9. **Memory footprint.** Multiple in-flight 16KB bulk transfer buffers
|
||
plus a forward queue plus `esp_http_client` buffers, alongside whatever
|
||
Workstream I's WiFi/HTTP/BMP280/RD-03E stack already needs, on a
|
||
single ESP32-P4's RAM — not sized or measured against a real linker
|
||
map.
|
||
|
||
### Build / integration notes
|
||
|
||
This component is **off by default** (`RTLSDR_EXP_ENABLE` Kconfig option,
|
||
default `n`) so it cannot affect Workstream I's core build unless
|
||
explicitly turned on via `idf.py menuconfig` → "RTL-SDR Experimental
|
||
Module." To include it in a real project once Workstream I's skeleton
|
||
exists:
|
||
|
||
1. Add this directory's `components/` to the consuming project's
|
||
`EXTRA_COMPONENT_DIRS` in the top-level `CMakeLists.txt` (or copy
|
||
`components/rtlsdr_experimental/` into that project's own
|
||
`components/`).
|
||
2. From wherever `app_main()` sets up its other sensor drivers, e.g.:
|
||
```c
|
||
#include "rtlsdr_experimental.h"
|
||
|
||
rtlsdr_exp_config_t sdr_cfg;
|
||
rtlsdr_exp_config_default(&sdr_cfg);
|
||
sdr_cfg.iq_upload_url = NULL; /* leave unset until a real backend
|
||
endpoint exists — see README */
|
||
sdr_cfg.device_bearer_token = DEVICE_BEARER_TOKEN; /* from
|
||
main/device_config.h, same token
|
||
used for the telemetry POST loop */
|
||
rtlsdr_exp_handle_t sdr_handle;
|
||
if (rtlsdr_exp_init(&sdr_cfg, &sdr_handle) == ESP_OK) {
|
||
rtlsdr_exp_start(sdr_handle); /* non-blocking; watches for a dongle */
|
||
}
|
||
```
|
||
3. `idf.py build` — **not run in this environment** (no ESP-IDF
|
||
toolchain available); this module has only been reasoned about, not
|
||
compiled. Treat "compiles" as an open question for the next person
|
||
with a real toolchain, not a claim made here.
|
||
|
||
Verified in this environment: none of it, against hardware or a real
|
||
compiler. What *has* been done: careful structural translation of a
|
||
real, already-partially-researched protocol (`sdr.ts`) to the documented
|
||
shape of a real, chip-specific API (ESP32-P4's USB Host Library), with
|
||
every simplification and open question called out rather than hidden.
|