Files
qtalker---/firmware/esp32p4-sensor-node/README.md
Indiana 6e627bae15 firmware: verify against actual target hardware, fix real conflicts
Researched the exact modules the user is building with and fixed three
concrete issues the earlier speculative firmware got wrong:

1. WiFi: confirmed the target board (Waveshare ESP32-P4-Module-DEV-KIT,
   chip ESP32-P4NRW32) bridges WiFi through an onboard ESP32-C6
   co-processor over a fixed 7-pin SDIO link (CLK18/CMD19/D0-14/D1-15/
   D2-16/D3-17/RESET54, cross-confirmed against Espressif's own
   esp-hosted-mcu docs). wifi_manager.c's esp_wifi_init()/esp_wifi_start()
   calls don't need to change — esp_wifi_remote/esp_hosted provide a
   drop-in-compatible API — but the component manifest (new
   main/idf_component.yml) and sdkconfig.defaults were missing entirely.

2. Real pin conflict: the presence sensor's original UART pins (17/18)
   directly collided with the SDIO CLK/D3 pins above — wiring it there
   would have broken WiFi, the sensor, or both. Moved to GPIO4/5.

3. Swapped placeholder parts for the user's actual hardware:
   - BME280 -> BMP280 (GY-BMP280 module): temp+pressure only, no humidity.
     Rewrote the driver rather than just renaming it — the old code would
     have read nonexistent humidity registers and reported garbage
     forever. 3.3V-only wiring note added (the BME280 assumption of
     5V-tolerant logic doesn't hold for this specific breakout).
   - LD2410 -> RD-03E (Ai-Thinker, not Hi-Link — a different manufacturer
     with a different, incompatible UART protocol). Rewrote the frame
     parser against the RD-03E's actual (if less-documented) 5-byte
     simple-report format. Reports numeric distance instead of a boolean,
     which better fits both the hardware's actual output and the backend's
     statistical anomaly detector.

README, CMakeLists.txt, and all cross-references updated to match.
2026-07-24 21:30:37 +00:00

616 lines
32 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Quantumancy ESP32-P4 Sensor Node
Firmware for the paired hardware sensor node described in
[`docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md`](../../docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md)
("Workstream I — firmware, ESP-IDF C — core sensor node"). Real ESP-IDF C
(FreeRTOS-based), not Arduino, not pseudocode. Connects to the seeker's home
Wi-Fi, samples a small set of sensors, and POSTs readings to the Quantumancy
backend's `POST /api/device/telemetry` endpoint, which feeds them into the
séance's live anomaly-detection pipeline as a sixth signal source alongside
`wire`/`evp`/`radio`/`emf`.
**Target board:** [Waveshare ESP32-P4-Module-DEV-KIT](https://www.waveshare.com/esp32-p4-module-dev-kit.htm)
(chip: **ESP32-P4NRW32**, 16MB NOR flash, onboard **ESP32-C6** Wi-Fi
6/Bluetooth 5 co-processor over SDIO, USB OTG 2.0 HS, 40-pin 2×20 header
with 28 programmable GPIOs). All pin assignments and the Wi-Fi bring-up
approach below are specific to this board — see the Wi-Fi section for the
reserved SDIO pins and [Wiring / pinout](#wiring--pinout) for sensor pins.
## Honesty policy — READ THIS FIRST
> This app's whole ethos is "real signal processing on real data, and it
> says so when something is unverified."
**Nobody working on this had physical ESP32-P4 hardware, a BMP280, or an
RD-03E module to flash and test against.** Everything in this directory is
real, structurally-correct ESP-IDF C, written against ESP-IDF's documented
APIs and each sensor's public datasheet/protocol documentation, and reasoned
about carefully — but it has **never been compiled with a real ESP-IDF
toolchain, never been flashed, and never talked to real hardware.** Treat
every claim below as "should work, per the docs" rather than "confirmed
working." See [What's verified vs. not](#whats-verified-vs-not) for the
specific, itemized breakdown — this mirrors the same convention
`frontend/src/lib/sdr.ts`'s `HARDWARE PASS REQUIRED` header comment uses
elsewhere in this repo.
## Directory layout
```
firmware/esp32p4-sensor-node/
├── CMakeLists.txt top-level ESP-IDF project file
├── sdkconfig.defaults seed config (idf.py generates the real sdkconfig)
├── README.md this file
├── components/
│ ├── README.md
│ └── rtlsdr_experimental/ Workstream J — opt-in USB-host RTL-SDR module
└── main/
├── CMakeLists.txt component registration
├── idf_component.yml managed deps: esp_wifi_remote, esp_hosted
├── app_main.c entry point / boot sequence
├── device_config.h.example template you copy to device_config.h
├── wifi_manager.{h,c} Wi-Fi station mode connect/reconnect (via
│ the onboard ESP32-C6 co-processor — see below)
├── telemetry_client.{h,c} HTTP POST task -> /api/device/telemetry
├── sensor_driver.h the sensor_driver_t registry interface
├── sensor_registry.{h,c} the concrete list of compiled-in drivers
├── bmp280.{h,c} temperature/pressure over I2C
└── rd03e.{h,c} presence/distance/gesture over UART
```
## Build instructions
Requires an ESP-IDF install (v5.3 or newer — ESP32-P4 target support landed
around v5.2/5.3; this was written without a toolchain available to pin an
exact tested version, see honesty section). With `idf.py` on your `PATH`
(e.g. after sourcing ESP-IDF's `export.sh`):
```sh
cd firmware/esp32p4-sensor-node
# 1. Fill in your Wi-Fi + pairing details (see next section) — the build
# will fail on a missing #include until you do this.
cp main/device_config.h.example main/device_config.h
$EDITOR main/device_config.h
# 2. Target and build.
idf.py set-target esp32p4
idf.py build
# 3. Flash + monitor (adjust the port for your machine).
idf.py -p /dev/ttyUSB0 flash monitor
```
## Manual configuration (no provisioning UI — by design)
A full BLE/Wi-Fi-AP provisioning flow is explicitly out of scope for this
spec (see the spec's "Explicitly out of scope" section). Instead, you
hand-edit one header before building:
1. In the Quantumancy web app, sign in and create a device from your
account (name + optional sensor-type hint). The backend shows you a
**raw pairing token exactly once** — copy it immediately, it cannot be
retrieved again (same one-time-secret convention as the site's session
tokens).
2. `cp main/device_config.h.example main/device_config.h`
3. Edit `main/device_config.h` and fill in:
- `DEVICE_WIFI_SSID` / `DEVICE_WIFI_PASSWORD` — your home Wi-Fi.
- `DEVICE_BACKEND_BASE_URL` — the backend's base URL, no trailing slash.
- `DEVICE_PAIRING_TOKEN` — the raw token from step 1.
- `DEVICE_REPORT_INTERVAL_SEC` — optional, defaults to 15s.
4. `main/device_config.h` is listed in `.gitignore` — it will never be
committed. Never put real credentials in `device_config.h.example`
itself; that file is the template everyone else copies.
There is deliberately no other config path (no NVS-based captive portal, no
BLE provisioning) in this build — see the spec's scope boundary.
## What's verified vs. not
**Structurally verified** (reasoned through carefully against ESP-IDF's
documented API surface and each sensor's public protocol docs; internally
consistent; no known syntax errors or obviously-wrong API usage):
- Project skeleton (`CMakeLists.txt` × 2, `sdkconfig.defaults`,
`idf_component_register` call) follows ESP-IDF's standard project layout.
- Wi-Fi station-mode connect/reconnect (`wifi_manager.c`) follows ESP-IDF's
documented event-driven pattern (`WIFI_EVENT`/`IP_EVENT` handlers +
`EventGroupHandle_t`), extended with an exponential-backoff reconnect
timer instead of giving up after N tries.
- HTTP client (`telemetry_client.c`) builds the exact JSON shape the spec's
contract defines and POSTs it via `esp_http_client` with
`Authorization: Bearer <token>` and `Content-Type: application/json`.
- BMP280 driver (`bmp280.c`): register map and the double-precision
compensation formulas are transcribed from Bosch's public BMP280
datasheet (rev 1.23, §3.11.1–3.11.3) — this is well-trodden, publicly
documented territory, and the formulas are checkable line-by-line against
the datasheet. Uses ESP-IDF's newer `driver/i2c_master.h` API (the
current idiomatic choice; the older `driver/i2c.h` is being phased out).
Temperature + pressure only — the part in use (a GY-BMP280 breakout) has
no humidity sensor, unlike its BME280 sibling.
- RD-03E driver (`rd03e.c`): the 5-byte "simple report" UART frame
(`0xAA` header, gesture byte, little-endian distance, `0x55 0x55`
footer) is reconstructed from a third-party bring-up write-up, not
Ai-Thinker's own datasheet (not available while writing this) — **the
single least-certain piece of code in this entire firmware.** The
gesture byte's exact value-to-meaning mapping is unconfirmed, so the
driver reports it as a raw code in `metadata` rather than guessing at a
translated label. Cross-confirmed from multiple sources: 256000 baud,
8N1 UART framing.
- Sensor driver registry (`sensor_driver.h`, `sensor_registry.c`): a
`sensor_driver_t { name, init, read }` struct, a compile-time array of
them, and generic init/collect functions that `app_main.c` and
`telemetry_client.c` call without knowing which concrete sensors exist.
**NOT verified — requires real hardware bring-up:**
- `idf.py build` has never actually been run in this environment (no
ESP-IDF toolchain installed here) — there could be a typo, a missing
include, or an API signature mismatch against whatever exact ESP-IDF
version you build with that only a real compile will surface.
- I2C timing/electricals: pull-up resistor values, bus speed headroom,
cable length — none of this has been bench-tested.
- BMP280 compensation formula correctness in practice: the math is
transcribed carefully, but "matches the datasheet" and "produces a
plausible number when this exact C runs on this exact silicon" are
different claims until someone compares a real reading to a reference
thermometer/barometer.
- RD-03E frame parsing, as above — this one especially, since the frame
format itself (not just the implementation) is reconstructed from a
third-party source rather than an official datasheet. Verify against a
logic analyzer capture before trusting field values, and treat the
gesture code's meaning as genuinely unknown until cross-checked.
- Wi-Fi reconnect behavior under real-world conditions (router reboot,
weak signal, captive portals) — the backoff logic is reasoned about, not
soak-tested.
- HTTP client behavior against the real backend: TLS handshake against
its actual certificate, real latency, real error responses. The
`sdkconfig.defaults` enables mbedTLS's full certificate bundle for this,
but that's untested against the live deploy.
- Timing/power: task stack sizes (`telemetry_task`'s 8192 words, etc.) are
reasonable guesses, not measured high-water-marks from a real run.
- Everything under [Wiring / pinout](#wiring--pinout) and the Wi-Fi section
below — confirmed against the actual target board's published specs, but
never checked against the physical hardware itself.
## Wi-Fi: ESP32-P4 has no integrated radio — confirmed target hardware
The ESP32-P4 SoC has **no built-in 2.4GHz radio**. This was flagged
speculatively in an earlier pass; it's now confirmed against the actual
target hardware — **Waveshare ESP32-P4-Module-DEV-KIT**, chip
**ESP32-P4NRW32** — which solves this the way Espressif's own reference
design does: an onboard **ESP32-C6** co-processor, wired to the P4 over a
**fixed 7-pin SDIO link**, running Wi-Fi 6 + Bluetooth 5 on the seeker's
behalf. These 7 pins are physically fixed by the board's own PCB routing —
**do not reuse them for sensors**:
| Signal | GPIO |
|---|---|
| SDIO CLK | 18 |
| SDIO CMD | 19 |
| SDIO D0 | 14 |
| SDIO D1 | 15 |
| SDIO D2 | 16 |
| SDIO D3 | 17 |
| C6 Reset | 54 |
(Sourced from Waveshare's own documentation and independently cross-checked
against Espressif's `esp-hosted-mcu` reference docs for their nearly
identical ESP32-P4-Function-EV-Board, which uses the same 7 pins — strong
agreement across two independent sources, though still not a substitute for
checking the physical board's silkscreen.)
Wi-Fi is brought up via two managed components — `espressif/esp_wifi_remote`
and `espressif/esp_hosted` — declared in `main/idf_component.yml` and
configured in `sdkconfig.defaults` (`CONFIG_SLAVE_IDF_TARGET_ESP32C6`,
`CONFIG_ESP_HOSTED_CP_TARGET_ESP32C6`, `CONFIG_ESP_HOSTED_P4_DEV_BOARD_FUNC_BOARD`,
plus buffer/window tuning). Critically, these components provide a
**drop-in-compatible** `esp_wifi_*`/`esp_netif_*` API — `wifi_manager.c`'s
`esp_wifi_init()`/`esp_wifi_start()` calls are exactly what they'd be on a
Wi-Fi-native chip; only the component manifest and sdkconfig differ, not
the application code. `idf.py build` will need network access the first
time to fetch these two components from the ESP Component Registry.
**What's still unverified here specifically:** whether
`CONFIG_ESP_HOSTED_P4_DEV_BOARD_FUNC_BOARD` (a preset named for Espressif's
own eval board) configures correctly for Waveshare's board given they
share the same SDIO pin assignment — likely fine, but the first thing to
check in `idf.py menuconfig` if Wi-Fi bring-up misbehaves is the Wi-Fi
Remote / ESP-Hosted config screens directly rather than trusting the preset
blindly.
## Wiring / pinout
Confirmed against the target board (Waveshare ESP32-P4-Module-DEV-KIT):
GPIOs 14-19 and 54 are reserved for the onboard Wi-Fi co-processor's SDIO
link (see the Wi-Fi section above) — never wire a sensor to those pins on
this board.
### BMP280 (I2C) — temperature, pressure
Confirmed against the actual part in use: a **GY-BMP280** breakout module
(Bosch BMP280 — temperature + pressure only, no humidity). GPIO8/9 don't
conflict with the board's reserved SDIO range. **3.3V only** — the
GY-BMP280 module is not 5V tolerant, unlike the RD-03E below.
| BMP280 pin | Connects to |
|------------|---------------------------------------|
| VCC | 3V3 (3.3V ONLY — do not connect to 5V)|
| GND | GND |
| SDA | GPIO8 (`BMP280_I2C_SDA_GPIO`) |
| SCL | GPIO9 (`BMP280_I2C_SCL_GPIO`) |
| CSB | VCC (selects I2C mode, not SPI) |
| SDO | GND → I2C address `0x76` (default assumed; tie to VCC + change `BMP280_I2C_ADDR` for `0x77`) |
GPIO numbers are `#define`s at the top of `bmp280.h` — override them there
(or via a future `idf.py menuconfig` entry) to match your actual wiring.
100kHz I2C clock by default (`BMP280_I2C_CLK_HZ`); the part supports faster
modes if your wiring/pull-ups support it.
### RD-03E (UART) — presence, distance, gesture
Confirmed against the actual part in use: an **Ai-Thinker RD-03E** 24GHz
mmWave radar ("Human Gesture Recognition... Precise Ranging & Positioning
Radar Sensor Module"). Originally speculatively planned as a Hi-Link
LD2410 — a **different manufacturer with a different, incompatible UART
protocol** — so this driver was rewritten from scratch against the RD-03E's
actual (if less-documented) frame format rather than adapted from the
LD2410 code. Reports ranged distance (not just a boolean), fitting a
handheld "sense a presence at a distance" device well — see the honesty
note in [What's verified vs. not](#whats-verified-vs-not) about the frame
parser being the single least-certain code in this firmware.
| RD-03E pin | Connects to |
|------------|----------------------------------------|
| VCC | 5V (module power; UART logic is 0-3.3V, ESP32-safe) |
| GND | GND |
| OT1 | GPIO4 (`RD03E_UART_RX_GPIO`, ESP32 RX) — module's UART TX output |
| RX | GPIO5 (`RD03E_UART_TX_GPIO`, ESP32 TX) — module's UART RX input |
| OT2 | not connected (reserved on the module, unused here) |
(GPIO4/5 avoid this board's reserved SDIO pins and the BMP280's I2C pins —
a reasonable, currently-unused pick, not verified against the physical
board's full pin map beyond confirming it's outside those known-reserved
ranges.)
Default UART settings: 256000 baud, 8N1 (module factory default), reading
only the module's free-running "simple report" frames — no configuration
handshake is sent or required. GPIO numbers and baud rate are `#define`s
at the top of `rd03e.h`.
## Sensor driver registry — the extensibility pattern
`sensor_driver.h` defines:
```c
typedef struct {
char sensor_type[SENSOR_READING_TYPE_MAXLEN];
double value;
char unit[SENSOR_READING_UNIT_MAXLEN];
cJSON *metadata; // nullable; NULL serializes as {}
} sensor_reading_t;
typedef struct sensor_driver {
const char *name;
esp_err_t (*init)(void);
esp_err_t (*read)(sensor_reading_t *out, size_t max_out, size_t *out_count);
} sensor_driver_t;
```
`sensor_registry.c` holds a compile-time array of these (currently BMP280
and RD-03E) and two generic functions, `sensor_registry_init_all()` and
`sensor_registry_collect()`, that `app_main.c` and `telemetry_client.c`
call without ever referencing `bmp280.c`/`rd03e.c` directly. One driver
failing `init()` or `read()` is logged and skipped — it doesn't take the
whole node offline.
### Adding a new sensor
1. Write `main/my_sensor.h` / `main/my_sensor.c` implementing `init()` and
`read()` matching `sensor_driver_t`'s function pointer signatures.
2. Add `"my_sensor.c"` to the `SRCS` list in `main/CMakeLists.txt`.
3. `#include "my_sensor.h"` in `sensor_registry.c` and add one line to the
`s_drivers[]` array:
```c
{ .name = "my_sensor", .init = my_sensor_init, .read = my_sensor_read },
```
Nothing in `app_main.c`, `telemetry_client.c`, or the main reporting loop's
control flow needs to change — that's the whole point of this structure per
the spec.
## Backend contract this firmware targets
From the spec (binding, see the spec file for the authoritative version):
```json
POST /api/device/telemetry
Authorization: Bearer <raw pairing token>
Content-Type: application/json
{
"readings": [
{"sensor_type": "presence", "value": 1, "unit": "bool", "metadata": {}},
{"sensor_type": "temperature", "value": 21.4, "unit": "c", "metadata": {}},
{"sensor_type": "humidity", "value": 47.2, "unit": "pct", "metadata": {}},
{"sensor_type": "pressure", "value": 1013.2, "unit": "hpa", "metadata": {}}
]
}
```
This firmware's BMP280 driver emits `temperature`/`pressure` (no
`humidity` — see above); its RD-03E driver emits `presence` as a numeric
distance in centimeters (`unit: "cm"`, not a `0`/`1` boolean) so the
backend's statistical anomaly detector can treat "something got suddenly
close" as the signal, with the module's raw gesture code folded into
`metadata` (`gesture_code`) rather than a boolean transition detector.
## Out of scope here
Per the spec: thermal camera support, a full BLE/Wi-Fi-AP provisioning UX,
on-device spectrum analysis/FFT (the RTL-SDR module below forwards raw IQ
upstream rather than analyzing on-device), and anything on the
backend/frontend side (Workstreams G, H, K).
---
## Workstream J — RTL-SDR experimental module
**⚠️ This is the least certain part of the entire firmware build. Read
this whole section before touching it.**
### What this is
A clearly-separated, opt-in, experimental module exploring USB-host
communication with an RTL2832U-based SDR dongle over the ESP32-P4's
native USB-OTG host controller (USB Host Library, `usb_host.h` — a real,
documented capability of this specific chip, unlike most ESP32 variants).
The idea, per the design spec: attach a cheap RTL-SDR dongle to the
sensor node, treat "spirit radio scanning" as a hardware-backed mode
instead of only a browser WebUSB feature.
**Be honest about the real constraint (this is the spec's framing, and
it's correct):** wideband IQ sample rates and FFT processing are
demanding relative to an MCU's compute, even one with ESP32-P4's AI
accelerator. **On-device spectrum analysis is not what this module
attempts.** The realistic architecture — and the one implemented here —
is: pull raw IQ samples off the dongle via USB host, and forward them
upstream for the *backend* to FFT/analyze (the same job the browser
already does client-side today via `frontend/src/lib/fft.ts` +
`frontend/src/lib/sdr.ts`'s `SpectrumAnomalyDetector`). Even that
"just pass the bytes through" architecture needs real USB throughput
numbers to know if it's viable — see "What's unverified" below.
### Primary reference
`frontend/src/lib/sdr.ts` — this repo's existing browser-based (WebUSB)
RTL2832U + R820T driver. It's real, already-researched protocol detail
against the public librtlsdr register documentation (vendor commands,
I2C-repeater tuner access, the demod/tuner init sequence), itself marked
`HARDWARE PASS REQUIRED` since it's never been run against a real dongle
either. This firmware module is a direct **port** of that file's control-
transfer sequence from WebUSB JS calls to ESP-IDF USB Host Library C
calls — line-by-line correspondences are called out in code comments
(e.g. `rtlsdr_run_init_sequence()` mirrors `open()` in sdr.ts almost
register-for-register). It intentionally does not re-derive any register
math from scratch; wherever sdr.ts says "simplified" or "HARDWARE PASS
REQUIRED" (e.g. the R820T PLL frequency math, the demod resample-ratio
math), this module carries the exact same simplification forward with the
exact same caveat, rather than inventing new unverified math on top of
already-unverified math.
### What's implemented (structurally — see caveats below)
1. **USB Host Library lifecycle** (`rtlsdr_exp_start()` /
`rtlsdr_usb_lib_daemon_task()` / `rtlsdr_exp_client_task()`):
`usb_host_install()`, `usb_host_client_register()` with an async event
callback, and the two-task pump pattern the USB Host Library's async
model requires (one task for `usb_host_lib_handle_events()`, one for
`usb_host_client_handle_events()` — the latter also being how this
module's own control- and bulk-transfer completion callbacks get
dispatched, since the Library calls them synchronously from whichever
task is pumping events, not from a hidden thread or ISR).
2. **RTL2832U/Terratec device enumeration and vendor-ID matching**
(`rtlsdr_try_bring_up()`, `rtlsdr_vendor_id_matches()`): on a
`USB_HOST_CLIENT_EVENT_NEW_DEV` event, opens the device, reads its
device descriptor, and matches `idVendor` against `0x0BDA`
(RTL2832U) / `0x0CCD` (Terratec-rebadged) — ported directly from
`sdr.ts`'s `requestDevice()` filter list (`RTL2832U_VENDOR`,
`TERRATEC_VENDOR`). Faithfully carries over that file's specific
choice to filter by **vendor ID only**, not product ID (its comment:
"many dongles report product ids outside the handful we know, so
filtering by productId hides them from the picker") — the known
product-id list (`RTLSDR_EXP_KNOWN_PRODUCT_IDS`) is kept as an
informational log line only, never a hard filter, exactly mirroring
how `RTL2832U_PRODUCTS` is exported-but-unused-as-a-filter in
`sdr.ts`.
3. **RTL2832U init vendor-command sequence** over USB control transfers
(`rtlsdr_run_init_sequence()`, `rtlsdr_demod_write()`,
`rtlsdr_reg_write()`, `rtlsdr_i2c_write()`, `rtlsdr_exp_set_frequency()`,
`rtlsdr_exp_set_sample_rate()`): the same demod soft-reset → demod_ctl/
suspend-off block → standby-off → AGC-mode → R820T tuner power-up
(through the I2C repeater) → sample-rate program → initial tune →
streaming-endpoint reset → test-mode-off sequence as `sdr.ts`'s
`open()`, with the same register addresses/values and the same
`wValue`/`wIndex` encoding (`(block<<8)|0x10` / `(page<<8)|address`),
translated from `USBDevice.controlTransferOut()` to
`usb_host_transfer_submit_control()` with a manually-built
`usb_setup_packet_t`.
4. **A basic bulk-transfer read loop structure** for pulling IQ sample
data off the device (`rtlsdr_start_bulk_streaming()`,
`rtlsdr_bulk_xfer_cb()`): unlike `sdr.ts` (which does one-shot
`await dev.transferIn(...)` calls from inside its own async `sweep()`
loop), the USB Host Library is callback-driven, so continuous
streaming here is a small pipeline — N transfers (`bulk_read_queue_
depth`, default 4) of a fixed chunk size (`bulk_read_chunk_bytes`,
default 16384B, enforced as a multiple of 512 the same way `sdr.ts`'s
`readSamples()` comment requires) are kept perpetually in flight; each
completion callback copies the received bytes into a heap block, hands
it to a FreeRTOS queue for the forward task, and immediately resubmits
itself to keep the pipe full. **This is the part most likely to need
real-hardware tuning** — see below.
5. **A stub/structure for forwarding raw IQ data upstream**
(`rtlsdr_exp_forward_iq_block()`, `rtlsdr_forward_task()`) — see the
architecture decision writeup immediately below.
### IQ-forwarding architecture decision
The spec explicitly leaves this as a product-judgment call ("your call on
whether this warrants a separate endpoint/stream vs. reusing the sensor
telemetry shape with a binary/base64 payload"). Decision made here:
**a separate binary stream/endpoint, not a reuse of the JSON telemetry
shape.** Reasoning:
- **Raw throughput is the wrong order of magnitude for the telemetry
contract.** Even a modest 2.048 Msps capture at 8 bits/sample/channel
(RTL2832U's native ADC format) is interleaved I/Q bytes at roughly
4.1 MB/s. The design spec's `POST /api/device/telemetry` contract caps
the `readings` array at ~64 entries, caps the request body at ~16KB,
and rate-limits ingestion to roughly 1 request/second sustained per
device — sized for a handful of scalar sensor readings (temperature,
humidity, presence booleans), not a continuous multi-megabyte/second
binary stream. Forcing IQ data through that shape would mean either
violating those caps (defeating their whole purpose — bounding what a
misbehaving/malicious device can push at the backend) or chopping IQ
into thousands of tiny requests per second, which is worse for both
sides than one continuous stream.
- **Base64-in-JSON adds ~33% size overhead** on top of a payload that's
already too large for the telemetry shape, for no benefit — there's no
reason to pay text-encoding tax on a payload nothing needs to
eyeball as text.
- **It's a fundamentally different kind of data with different backend
handling needs.** Telemetry readings feed the anomaly-detection
baseline/threshold pipeline directly and cheaply (a few floats per
sensor). IQ data needs FFT processing before it's useful for anything
— an entirely different backend code path (closer to how the browser's
own `powerSpectrumDb()` + `SpectrumAnomalyDetector` work today,
server-side instead of client-side). Conflating the two request shapes
would couple two things that should scale, rate-limit, and fail
independently.
Given that, this module's stub (`rtlsdr_exp_forward_iq_block()`) targets
a **separate, currently-hypothetical** endpoint (`iq_upload_url` in
`rtlsdr_exp_config_t`, suggested path `/api/device/iq-stream` in code
comments) carrying a small fixed binary header (magic, sequence number,
center frequency, sample rate, payload length — see
`rtlsdr_iq_chunk_header_t`) followed by the raw IQ bytes, POSTed as
`application/octet-stream` with the same `Authorization: Bearer <device
token>` auth as the regular telemetry loop. **This endpoint does not
exist anywhere in this repo.** Building it is explicitly out of scope for
this workstream (it's backend work, not firmware, and the spec doesn't
assign a backend workstream to receive IQ data at all — only to receive
scalar telemetry). The stub is inert by default (`rtlsdr_exp_forward_
iq_block()` returns `ESP_OK` immediately unless `iq_upload_url` is
configured) specifically so this module can be compiled/enabled for its
USB-host/bring-up behavior without requiring a backend that doesn't
exist.
A real product decision here — genuinely open, not resolved by this
workstream — is whether a POST-per-chunk model is even right versus a
persistent WebSocket stream (the spec's `/ws/device-feed` design already
establishes a WS pub/sub pattern on the backend for telemetry; a
`/ws/device-iq` sibling might fit that architecture better than repeated
HTTP POSTs, especially if backpressure/flow-control matters, which for a
continuous stream it very much does). That's flagged here rather than
silently decided, because it depends on backend design judgment as much
as firmware judgment.
### What's unverified — an honest, specific list for real hardware bring-up
Nothing below has run against real hardware. In rough order of "most
likely to break first":
1. **USB Host Library API surface.** Function names, struct field names,
and callback signatures (`usb_host_client_config_t`'s `.async.
client_event_callback` shape in particular — ESP-IDF has changed this
API's shape across versions) are written from documented/remembered
API shape, not checked against a real ESP-IDF checkout (none available
in this environment). **First thing a real bring-up needs: does this
even compile against the ESP-IDF version Workstream I's project
targets?**
2. **RTL2832U enumeration over a real ESP32-P4 USB-OTG host port** — does
`usb_host_device_open()` / `usb_host_get_device_descriptor()` actually
see the dongle at all on this specific chip's USB-OTG controller (power
delivery to the dongle over USB-host mode is itself a hardware
question — does the ESP32-P4 dev board supply VBUS in host mode, does
the dongle draw more current than it can supply).
3. **The init vendor-command sequence itself** — carried over unchanged
from `sdr.ts`, which is *itself* unverified. Two layers of "reasoned,
not tested." A logic analyzer / USB protocol analyzer trace against a
real dongle (or cross-checking against real librtlsdr `-T` verbose
output) is needed to confirm register values, not just transfer
plumbing.
4. **R820T PLL frequency math and demod resample-ratio math** — both are
the same simplified integer-N approximation `sdr.ts` uses (its own
comment: "real librtlsdr computes the exact sdm/vco from a 28.8MHz
crystal reference. Simplified... Marked for hardware."). Likely wrong
or imprecise until replaced with the real librtlsdr formula and
checked against an actual received signal.
5. **Bulk transfer chunk size, pipeline depth, and timeout=0 choice**
(`RTLSDR_EXP_BULK_CHUNK_BYTES`/`RTLSDR_EXP_BULK_QUEUE_DEPTH` Kconfig,
defaults 16384B / depth 4) — these are guesses. Real ESP32-P4 USB
Host Library heap/DMA limits, achievable sustained throughput at
2.048 Msps (~4.1 MB/s), and whether `timeout_ms = 0` (no timeout, i.e.
"block until data or disconnect") is even the right transfer mode for
this endpoint all need real measurement.
6. **Drop-on-full backpressure policy** (`iq_blocks_dropped` stat) — is
silently dropping IQ blocks when the forward task falls behind
acceptable, or does it need real flow control (e.g. throttling the
bulk read rate itself)? Depends on (5) and on real WiFi uplink
bandwidth from a real device on a real seeker's home network.
7. **The proposed IQ-forwarding wire format and endpoint** — entirely
hypothetical (see architecture section above); no backend exists to
validate the framing against, and the POST-vs-WebSocket question above
is unresolved.
8. **Concurrent USB Host Library ownership with the rest of the firmware
project.** `rtlsdr_exp_start()`/`rtlsdr_exp_stop()` call `usb_host_
install()`/`usb_host_uninstall()` directly, assuming this module is the
*only* USB Host client in the project. If Workstream I's skeleton (or
anything else) also needs USB host for something, this needs to change
to a shared-ownership model (install once, both modules register as
clients) — impossible to resolve without seeing that code, flagged
here for whoever does the merge.
9. **Memory footprint.** Multiple in-flight 16KB bulk transfer buffers
plus a forward queue plus `esp_http_client` buffers, alongside whatever
Workstream I's WiFi/HTTP/BMP280/RD-03E stack already needs, on a
single ESP32-P4's RAM — not sized or measured against a real linker
map.
### Build / integration notes
This component is **off by default** (`RTLSDR_EXP_ENABLE` Kconfig option,
default `n`) so it cannot affect Workstream I's core build unless
explicitly turned on via `idf.py menuconfig` → "RTL-SDR Experimental
Module." To include it in a real project once Workstream I's skeleton
exists:
1. Add this directory's `components/` to the consuming project's
`EXTRA_COMPONENT_DIRS` in the top-level `CMakeLists.txt` (or copy
`components/rtlsdr_experimental/` into that project's own
`components/`).
2. From wherever `app_main()` sets up its other sensor drivers, e.g.:
```c
#include "rtlsdr_experimental.h"
rtlsdr_exp_config_t sdr_cfg;
rtlsdr_exp_config_default(&sdr_cfg);
sdr_cfg.iq_upload_url = NULL; /* leave unset until a real backend
endpoint exists — see README */
sdr_cfg.device_bearer_token = DEVICE_BEARER_TOKEN; /* from
main/device_config.h, same token
used for the telemetry POST loop */
rtlsdr_exp_handle_t sdr_handle;
if (rtlsdr_exp_init(&sdr_cfg, &sdr_handle) == ESP_OK) {
rtlsdr_exp_start(sdr_handle); /* non-blocking; watches for a dongle */
}
```
3. `idf.py build` — **not run in this environment** (no ESP-IDF
toolchain available); this module has only been reasoned about, not
compiled. Treat "compiles" as an open question for the next person
with a real toolchain, not a claim made here.
Verified in this environment: none of it, against hardware or a real
compiler. What *has* been done: careful structural translation of a
real, already-partially-researched protocol (`sdr.ts`) to the documented
shape of a real, chip-specific API (ESP32-P4's USB Host Library), with
every simplification and open question called out rather than hidden.