qtalker---/firmware/esp32p4-sensor-node/README.md

Quantumancy ESP32-P4 Sensor Node

Firmware for the paired hardware sensor node described in docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md ("Workstream I — firmware, ESP-IDF C — core sensor node"). Real ESP-IDF C (FreeRTOS-based), not Arduino, not pseudocode. Connects to the seeker's home Wi-Fi, samples a small set of sensors, and POSTs readings to the Quantumancy backend's POST /api/device/telemetry endpoint, which feeds them into the séance's live anomaly-detection pipeline as a sixth signal source alongside wire/evp/radio/emf.

Target board: Waveshare ESP32-P4-Module-DEV-KIT (chip: ESP32-P4NRW32, 16MB NOR flash, onboard ESP32-C6 Wi-Fi 6/Bluetooth 5 co-processor over SDIO, USB OTG 2.0 HS, 40-pin 2×20 header with 28 programmable GPIOs). All pin assignments and the Wi-Fi bring-up approach below are specific to this board — see the Wi-Fi section for the reserved SDIO pins and Wiring / pinout for sensor pins.

Target sensors: a GY-BMP280 breakout (temperature/pressure, I2C), an Ai-Thinker RD-03E 24GHz mmWave radar (presence/distance/gesture, UART), and an I2S digital MEMS microphone (INMP441-family pinout — L/R/WS/SCK/SD — EVP-style audio level). See Wiring / pinout for exact connections.

Honesty policy — READ THIS FIRST

This app's whole ethos is "real signal processing on real data, and it says so when something is unverified."

Nobody working on this had physical ESP32-P4 hardware, a BMP280, or an RD-03E module to flash and test against. Everything in this directory is real, structurally-correct ESP-IDF C, written against ESP-IDF's documented APIs and each sensor's public datasheet/protocol documentation, and reasoned about carefully — but it has never been compiled with a real ESP-IDF toolchain, never been flashed, and never talked to real hardware. Treat every claim below as "should work, per the docs" rather than "confirmed working." See What's verified vs. not for the specific, itemized breakdown — this mirrors the same convention frontend/src/lib/sdr.ts's HARDWARE PASS REQUIRED header comment uses elsewhere in this repo.

One narrow exception, added deliberately. The pure logic — frame parsing, byte order, compensation maths, level maths — has been extracted into ESP-IDF-free units under main/ and is now covered by a host test suite you can run anywhere gcc exists:

./run_tests.sh          # from the repo root, or test/run_tests.sh from here

That suite is machine-verified, not reasoned about. It is also strictly about logic: it never touches a bus, a pin, or ESP-IDF, and it cannot tell you whether the protocols it implements match the real modules. Read the first subsection of What's verified vs. not for exactly what it does and does not establish.

Directory layout

firmware/esp32p4-sensor-node/
├── CMakeLists.txt              top-level ESP-IDF project file
├── sdkconfig.defaults          seed config (idf.py generates the real sdkconfig)
├── README.md                   this file
├── test/                       host test harness (gcc only, no ESP-IDF)
│   ├── run_tests.sh            build + run; non-zero exit on failure
│   ├── Makefile                same build, for `make check`
│   └── test_*.c                plain-assert tests, zero dependencies
├── components/
│   ├── README.md
│   └── rtlsdr_experimental/    Workstream J — opt-in USB-host RTL-SDR module
└── main/
    ├── CMakeLists.txt          component registration
    ├── idf_component.yml       managed deps: esp_wifi_remote, esp_hosted
    ├── app_main.c              entry point / boot sequence
    ├── device_config.h.example template you copy to device_config.h
    ├── wifi_manager.{h,c}      Wi-Fi station mode connect/reconnect (via
    │                           the onboard ESP32-C6 co-processor — see below)
    ├── telemetry_client.{h,c}  HTTP POST task -> /api/device/telemetry
    ├── sensor_driver.h         the sensor_driver_t registry interface
    ├── sensor_registry.{h,c}   the concrete list of compiled-in drivers
    ├── bmp280.{h,c}            temperature/pressure over I2C (I2C traffic)
    ├── bmp280_compensate.{h,c} PURE: calib/ADC decode + Bosch compensation
    ├── rd03e.{h,c}             presence/distance/gesture over UART (UART I/O)
    ├── rd03e_parse.{h,c}       PURE: the 6-byte frame scanner
    ├── mems_mic.{h,c}          EVP-style audio level over I2S (I2S traffic)
    └── mems_level.{h,c}        PURE: RMS -> dBFS maths

The units marked PURE include only <stdint.h>/<stddef.h>/<math.h> — no ESP-IDF, no FreeRTOS, no logging — so test/ can compile them with plain gcc. The drivers alongside them own the bus I/O and call in. Any new decision that is pure arithmetic or byte handling belongs in a PURE unit, where it can be tested before it reaches a board.

Build instructions

Requires an ESP-IDF install (v5.3 or newer — ESP32-P4 target support landed around v5.2/5.3; this was written without a toolchain available to pin an exact tested version, see honesty section). With idf.py on your PATH (e.g. after sourcing ESP-IDF's export.sh):

cd firmware/esp32p4-sensor-node

# 1. Fill in your Wi-Fi + pairing details (see next section) — the build
#    will fail on a missing #include until you do this.
cp main/device_config.h.example main/device_config.h
$EDITOR main/device_config.h

# 2. Target and build.
idf.py set-target esp32p4
idf.py build

# 3. Flash + monitor (adjust the port for your machine).
idf.py -p /dev/ttyUSB0 flash monitor

Manual configuration (no provisioning UI — by design)

A full BLE/Wi-Fi-AP provisioning flow is explicitly out of scope for this spec (see the spec's "Explicitly out of scope" section). Instead, you hand-edit one header before building:

1. In the Quantumancy web app, sign in and create a device from your account (name + optional sensor-type hint). The backend shows you a raw pairing token exactly once — copy it immediately, it cannot be retrieved again (same one-time-secret convention as the site's session tokens).
2. cp main/device_config.h.example main/device_config.h
3. Edit main/device_config.h and fill in:
   • DEVICE_WIFI_SSID / DEVICE_WIFI_PASSWORD — your home Wi-Fi.
   • DEVICE_BACKEND_BASE_URL — the backend's base URL, no trailing slash.
   • DEVICE_PAIRING_TOKEN — the raw token from step 1.
   • DEVICE_REPORT_INTERVAL_SEC — optional, defaults to 15s.
4. main/device_config.h is listed in .gitignore — it will never be committed. Never put real credentials in device_config.h.example itself; that file is the template everyone else copies.

There is deliberately no other config path (no NVS-based captive portal, no BLE provisioning) in this build — see the spec's scope boundary.

What's verified vs. not

Machine-verified on a host: the pure logic (test/)

Run it with ./run_tests.sh (from the repo root, or test/run_tests.sh here). It needs gcc and nothing else — no ESP-IDF, no toolchain, no board, no network. It compiles with -Wall -Wextra -Werror and exits non-zero on any failure. Current status: 175 checks, 0 failures.

This exists because a real bug shipped in this firmware and sat there undetected: RD03E_FRAME_LEN was 5 for a 6-byte frame, so the footer check compared the distance high byte against 0x55 instead of the second footer byte. Frames only "validated" when the high byte happened to be 0x55, and every distance reading came back as lo | 0x5500 — roughly 218 metres, always. That was pure arithmetic with zero hardware dependency and it should have been catchable on a laptop. The three pure units below were split out of their drivers precisely so that class of bug now is.

The ESP-IDF-free units, and what the tests actually prove about each:

• main/rd03e_parse.c — the frame scanner. Proven: a well-formed frame yields the exact expected gesture and distance; 0x2C 0x01 is 300 cm, little-endian (the shipped bug produced 21804 cm here); RD03E_FRAME_LEN really is 6 and two back-to-back frames occupy exactly 12 bytes without desynchronising; two frames in one buffer report the newest; a wrong byte in either footer position is rejected; a truncated trailing frame is ignored and never read past; garbage (including a stray 0xAA) before a valid frame is skipped; a 0xAA that is really a payload byte does not fool the scanner; the full 16-bit distance range decodes with the correct byte order; NULL/short/empty inputs return "no frame" rather than crashing.
• main/bmp280_compensate.c — calibration/ADC decoding plus the Bosch §3.11.3 compensation maths. Proven: all twelve calibration coefficients decode little-endian with signedness preserved; the 20-bit ADC words decode with pressure first, temperature second, and the XLSB's low nibble discarded; the datasheet's own worked reference values (adc_T=519888, adc_P=415148 with the published calibration set) come out at ~25.08 °C and ~100653 Pa; temperature rises with raw ADC; pressure falls monotonically across an ADC sweep and stays in a physically plausible band; the var1 == 0 guard returns exactly 0.0 rather than inf/NaN when the calibration block is all zeros (i.e. a silently-failed I2C read).
• main/mems_level.c — RMS → dBFS. Proven: a full-scale block reads ~0 dBFS; silence reads the −120 floor and is never -inf or NaN (which would poison the JSON the backend receives); sub-LSB noise in the padding bits stays at the floor; halving amplitude costs ~6 dB; negative samples carry the same energy as positive; the level rises monotonically with amplitude and stays inside [−120, 0]; empty/NULL blocks do not divide by zero.

What this does NOT prove — and the distinction matters. These tests verify the firmware's logic against the protocol and datasheet as this repo understands them. They cannot verify that understanding. If the RD-03E's real frame format differs from the reconstruction below, every test still passes and every reading is still wrong. Nothing here touches a UART, an I2C bus, an I2S clock, a GPIO, or ESP-IDF itself. See the hardware list further down — it is unchanged by these tests.

Structurally verified only

(Reasoned through carefully against ESP-IDF's documented API surface and each sensor's public protocol docs; internally consistent; no known syntax errors or obviously-wrong API usage. Not compiled, not run.)

• Project skeleton (CMakeLists.txt × 2, sdkconfig.defaults, idf_component_register call) follows ESP-IDF's standard project layout.
• Wi-Fi station-mode connect/reconnect (wifi_manager.c) follows ESP-IDF's documented event-driven pattern (WIFI_EVENT/IP_EVENT handlers + EventGroupHandle_t), extended with an exponential-backoff reconnect timer instead of giving up after N tries.
• HTTP client (telemetry_client.c) builds the exact JSON shape the spec's contract defines and POSTs it via esp_http_client with Authorization: Bearer <token> and Content-Type: application/json.
• BMP280 driver (bmp280.c + bmp280_compensate.c): register map and the double-precision compensation formulas are transcribed from Bosch's public BMP280 datasheet (rev 1.23, §3.11.1–3.11.3) — this is well-trodden, publicly documented territory, and the formulas are checkable line-by-line against the datasheet. Uses ESP-IDF's newer driver/i2c_master.h API (the current idiomatic choice; the older driver/i2c.h is being phased out). Temperature + pressure only — the part in use (a GY-BMP280 breakout) has no humidity sensor, unlike its BME280 sibling.
• RD-03E driver (rd03e.c + rd03e_parse.c): the 6-byte "simple report" UART frame (0xAA header, gesture byte, little-endian distance low/high, 0x55 0x55 footer) is reconstructed from a third-party bring-up write-up, not Ai-Thinker's own datasheet (not available while writing this) — the single least-certain piece of this entire firmware. The parser itself is now host-tested (above); the format it parses is still a reconstruction, and that is the risk that remains. The gesture byte's exact value-to-meaning mapping is unconfirmed, so the driver reports it as a raw code in metadata rather than guessing at a translated label. Cross-confirmed from multiple sources: 256000 baud, 8N1 UART framing.
• I2S MEMS microphone driver (mems_mic.c + mems_level.c): uses ESP-IDF's current driver/i2s_std.h API (standard/Philips mode, mono, 32-bit slot). The 24-bit-in-32-bit-slot right-shift and dBFS reference level are the commonly-documented values for the INMP441 family this module's pinout (L/R/WS/SCK/SD) matches. Deliberately does RMS-level reporting only, not on-device voice-band FFT — see its header comment for why.
• Sensor driver registry (sensor_driver.h, sensor_registry.c): a sensor_driver_t { name, init, read } struct, a compile-time array of them, and generic init/collect functions that app_main.c and telemetry_client.c call without knowing which concrete sensors exist.

NOT verified — requires real hardware bring-up:

None of the following is touched by run_tests.sh. The host tests cover arithmetic and byte handling; everything in this list is about wiring, timing, and what the silicon actually does.

• idf.py build has never actually been run in this environment (no ESP-IDF toolchain installed here) — there could be a typo, a missing include, or an API signature mismatch against whatever exact ESP-IDF version you build with that only a real compile will surface. The host harness deliberately does not compile rd03e.c / bmp280.c / mems_mic.c (they need ESP-IDF headers), so it cannot catch this.
• The firmware has still never been flashed to a board. Nothing below has been observed; it has only been reasoned about.
• I2C timing/electricals: pull-up resistor values, bus speed headroom, cable length — none of this has been bench-tested.
• BMP280 readings in practice: the maths now reproduces the datasheet's own worked reference values on a host, but "matches the datasheet's reference numbers" and "a real BMP280 on this bus returns the register contents we assume, and the result matches a reference thermometer/barometer" are different claims. Also unverified: the forced-mode ctrl_meas value, the status-polling loop, the reset delay, and the burst-read register addresses — all of that is I2C traffic the host tests never execute.
• The RD-03E frame format itself — not the parser, which is now host-tested, but the reconstruction it implements. This is the gap the tests cannot close: they assert the parser matches the format in rd03e_parse.h, and that format came from a third-party write-up rather than an official datasheet. If it is wrong, the tests pass and the readings are garbage. Verify against a logic analyzer capture before trusting field values, and treat the gesture code's meaning as genuinely unknown until cross-checked.
• I2S mic timing/levels: the BCLK/WS timing relationship, and whether the 24-bit-in-32-bit-slot shift is exactly right for this specific module revision. The host tests prove the shift-and-RMS arithmetic is self-consistent; they say nothing about whether shifting by 8 is the correct alignment for the real bitstream, or whether the dBFS numbers land in a sane, usable range once a real mic is feeding them — none of this has been bench-tested. Confirm by talking near the mic and checking the reported level actually rises before trusting it unattended.
• Wi-Fi reconnect behavior under real-world conditions (router reboot, weak signal, captive portals) — the backoff logic is reasoned about, not soak-tested.
• HTTP client behavior against the real backend: TLS handshake against its actual certificate, real latency, real error responses. The sdkconfig.defaults enables mbedTLS's full certificate bundle for this, but that's untested against the live deploy.
• Timing/power: task stack sizes (telemetry_task's 8192 words, etc.) are reasonable guesses, not measured high-water-marks from a real run.
• Everything under Wiring / pinout and the Wi-Fi section below — confirmed against the actual target board's published specs, but never checked against the physical hardware itself.

Wi-Fi: ESP32-P4 has no integrated radio — confirmed target hardware

The ESP32-P4 SoC has no built-in 2.4GHz radio. This was flagged speculatively in an earlier pass; it's now confirmed against the actual target hardware — Waveshare ESP32-P4-Module-DEV-KIT, chip ESP32-P4NRW32 — which solves this the way Espressif's own reference design does: an onboard ESP32-C6 co-processor, wired to the P4 over a fixed 7-pin SDIO link, running Wi-Fi 6 + Bluetooth 5 on the seeker's behalf. These 7 pins are physically fixed by the board's own PCB routing — do not reuse them for sensors:

Signal	GPIO
SDIO CLK	18
SDIO CMD	19
SDIO D0	14
SDIO D1	15
SDIO D2	16
SDIO D3	17
C6 Reset	54

(Sourced from Waveshare's own documentation and independently cross-checked against Espressif's esp-hosted-mcu reference docs for their nearly identical ESP32-P4-Function-EV-Board, which uses the same 7 pins — strong agreement across two independent sources, though still not a substitute for checking the physical board's silkscreen.)

Wi-Fi is brought up via two managed components — espressif/esp_wifi_remote and espressif/esp_hosted — declared in main/idf_component.yml and configured in sdkconfig.defaults (CONFIG_SLAVE_IDF_TARGET_ESP32C6, CONFIG_ESP_HOSTED_CP_TARGET_ESP32C6, CONFIG_ESP_HOSTED_P4_DEV_BOARD_FUNC_BOARD, plus buffer/window tuning). Critically, these components provide a drop-in-compatible esp_wifi_*/esp_netif_* API — wifi_manager.c's esp_wifi_init()/esp_wifi_start() calls are exactly what they'd be on a Wi-Fi-native chip; only the component manifest and sdkconfig differ, not the application code. idf.py build will need network access the first time to fetch these two components from the ESP Component Registry.

What's still unverified here specifically: whether CONFIG_ESP_HOSTED_P4_DEV_BOARD_FUNC_BOARD (a preset named for Espressif's own eval board) configures correctly for Waveshare's board given they share the same SDIO pin assignment — likely fine, but the first thing to check in idf.py menuconfig if Wi-Fi bring-up misbehaves is the Wi-Fi Remote / ESP-Hosted config screens directly rather than trusting the preset blindly.

Wiring / pinout

Confirmed against the target board (Waveshare ESP32-P4-Module-DEV-KIT): GPIOs 14-19 and 54 are reserved for the onboard Wi-Fi co-processor's SDIO link (see the Wi-Fi section above) — never wire a sensor to those pins on this board.

BMP280 (I2C) — temperature, pressure

Confirmed against the actual part in use: a GY-BMP280 breakout module (Bosch BMP280 — temperature + pressure only, no humidity). GPIO8/9 don't conflict with the board's reserved SDIO range. 3.3V only — the GY-BMP280 module is not 5V tolerant, unlike the RD-03E below.

BMP280 pin	Connects to
VCC	3V3 (3.3V ONLY — do not connect to 5V)
GND	GND
SDA	GPIO8 (BMP280_I2C_SDA_GPIO)
SCL	GPIO9 (BMP280_I2C_SCL_GPIO)
CSB	VCC (selects I2C mode, not SPI)
SDO	GND → I2C address 0x76 (default assumed; tie to VCC + change BMP280_I2C_ADDR for 0x77)

GPIO numbers are #defines at the top of bmp280.h — override them there (or via a future idf.py menuconfig entry) to match your actual wiring. 100kHz I2C clock by default (BMP280_I2C_CLK_HZ); the part supports faster modes if your wiring/pull-ups support it.

RD-03E (UART) — presence, distance, gesture

Confirmed against the actual part in use: an Ai-Thinker RD-03E 24GHz mmWave radar ("Human Gesture Recognition... Precise Ranging & Positioning Radar Sensor Module"). Originally speculatively planned as a Hi-Link LD2410 — a different manufacturer with a different, incompatible UART protocol — so this driver was rewritten from scratch against the RD-03E's actual (if less-documented) frame format rather than adapted from the LD2410 code. Reports ranged distance (not just a boolean), fitting a handheld "sense a presence at a distance" device well — see the honesty note in What's verified vs. not about the frame parser being the single least-certain code in this firmware.

RD-03E pin	Connects to
VCC	5V (module power; UART logic is 0-3.3V, ESP32-safe)
GND	GND
OT1	GPIO4 (RD03E_UART_RX_GPIO, ESP32 RX) — module's UART TX output
RX	GPIO5 (RD03E_UART_TX_GPIO, ESP32 TX) — module's UART RX input
OT2	not connected (reserved on the module, unused here)

(GPIO4/5 avoid this board's reserved SDIO pins and the BMP280's I2C pins — a reasonable, currently-unused pick, not verified against the physical board's full pin map beyond confirming it's outside those known-reserved ranges.)

Default UART settings: 256000 baud, 8N1 (module factory default), reading only the module's free-running "simple report" frames — no configuration handshake is sent or required. GPIO numbers and baud rate are #defines at the top of rd03e.h.

I2S MEMS microphone — EVP-style audio level

Confirmed against the actual part in use via its pinout (L/R, WS, SCK, SD, VCC, GND — the standard INMP441-family I2S digital MEMS mic breakout naming). Unlike the browser-based EVP mode's client-side voice-band FFT, this driver does not attempt on-device spectral analysis — it samples a short audio block per cycle, computes RMS level in dBFS, and reports that as a plain numeric reading. The backend's existing statistical anomaly detector (the same one already used for temperature/pressure) does the spike detection from there — simpler and more honest than pretending to replicate real voice-band filtering without ever having tested it.

Mic pin	Connects to
VCC	3V3
GND	GND
L/R	GND (selects left-channel output — tie to 3V3 instead for right, either works, just match the driver's I2S_STD_SLOT_LEFT default or change it)
WS	GPIO11 (MEMS_MIC_I2S_WS_GPIO) — word select / LRCLK
SCK	GPIO10 (MEMS_MIC_I2S_BCLK_GPIO) — bit clock
SD	GPIO12 (MEMS_MIC_I2S_DIN_GPIO) — serial data, mic OUT to ESP32 IN

(GPIO10/11/12 avoid this board's reserved SDIO range, the BMP280's I2C pins, and the RD-03E's UART pins — a reasonable, currently-unused pick, same verification caveat as the other sensors' pins above.)

16kHz sample rate, 256ms sample block per reporting cycle (4096 samples) — #defines at the top of mems_mic.h. Reports sensor_type: "evp", unit: "dbfs".

Sensor driver registry — the extensibility pattern

sensor_driver.h defines:

typedef struct {
    char sensor_type[SENSOR_READING_TYPE_MAXLEN];
    double value;
    char unit[SENSOR_READING_UNIT_MAXLEN];
    cJSON *metadata; // nullable; NULL serializes as {}
} sensor_reading_t;

typedef struct sensor_driver {
    const char *name;
    esp_err_t (*init)(void);
    esp_err_t (*read)(sensor_reading_t *out, size_t max_out, size_t *out_count);
} sensor_driver_t;

sensor_registry.c holds a compile-time array of these (currently BMP280 and RD-03E) and two generic functions, sensor_registry_init_all() and sensor_registry_collect(), that app_main.c and telemetry_client.c call without ever referencing bmp280.c/rd03e.c directly. One driver failing init() or read() is logged and skipped — it doesn't take the whole node offline.

Adding a new sensor

1. Write main/my_sensor.h / main/my_sensor.c implementing init() and read() matching sensor_driver_t's function pointer signatures.
2. Add "my_sensor.c" to the SRCS list in main/CMakeLists.txt.
3. #include "my_sensor.h" in sensor_registry.c and add one line to the s_drivers[] array:

   { .name = "my_sensor", .init = my_sensor_init, .read = my_sensor_read },
   

Nothing in app_main.c, telemetry_client.c, or the main reporting loop's control flow needs to change — that's the whole point of this structure per the spec.

Backend contract this firmware targets

From the spec (binding, see the spec file for the authoritative version):

POST /api/device/telemetry
Authorization: Bearer <raw pairing token>
Content-Type: application/json

{
  "readings": [
    {"sensor_type": "presence", "value": 1, "unit": "bool", "metadata": {}},
    {"sensor_type": "temperature", "value": 21.4, "unit": "c", "metadata": {}},
    {"sensor_type": "humidity", "value": 47.2, "unit": "pct", "metadata": {}},
    {"sensor_type": "pressure", "value": 1013.2, "unit": "hpa", "metadata": {}}
  ]
}

This firmware's BMP280 driver emits temperature/pressure (no humidity — see above); its RD-03E driver emits presence as a numeric distance in centimeters (unit: "cm", not a 0/1 boolean) so the backend's statistical anomaly detector can treat "something got suddenly close" as the signal, with the module's raw gesture code folded into metadata (gesture_code) rather than a boolean transition detector.

Out of scope here

Per the spec: thermal camera support, a full BLE/Wi-Fi-AP provisioning UX, on-device spectrum analysis/FFT (the RTL-SDR module below forwards raw IQ upstream rather than analyzing on-device), and anything on the backend/frontend side (Workstreams G, H, K).

---

Workstream J — RTL-SDR experimental module

⚠️ This is the least certain part of the entire firmware build. Read this whole section before touching it.

What this is

A clearly-separated, opt-in, experimental module exploring USB-host communication with an RTL2832U-based SDR dongle over the ESP32-P4's native USB-OTG host controller (USB Host Library, usb_host.h — a real, documented capability of this specific chip, unlike most ESP32 variants). The idea, per the design spec: attach a cheap RTL-SDR dongle to the sensor node, treat "spirit radio scanning" as a hardware-backed mode instead of only a browser WebUSB feature.

Be honest about the real constraint (this is the spec's framing, and it's correct): wideband IQ sample rates and FFT processing are demanding relative to an MCU's compute, even one with ESP32-P4's AI accelerator. On-device spectrum analysis is not what this module attempts. The realistic architecture — and the one implemented here — is: pull raw IQ samples off the dongle via USB host, and forward them upstream for the backend to FFT/analyze (the same job the browser already does client-side today via frontend/src/lib/fft.ts + frontend/src/lib/sdr.ts's SpectrumAnomalyDetector). Even that "just pass the bytes through" architecture needs real USB throughput numbers to know if it's viable — see "What's unverified" below.

Primary reference

frontend/src/lib/sdr.ts — this repo's existing browser-based (WebUSB) RTL2832U + R820T driver. It's real, already-researched protocol detail against the public librtlsdr register documentation (vendor commands, I2C-repeater tuner access, the demod/tuner init sequence), itself marked HARDWARE PASS REQUIRED since it's never been run against a real dongle either. This firmware module is a direct port of that file's control- transfer sequence from WebUSB JS calls to ESP-IDF USB Host Library C calls — line-by-line correspondences are called out in code comments (e.g. rtlsdr_run_init_sequence() mirrors open() in sdr.ts almost register-for-register). It intentionally does not re-derive any register math from scratch; wherever sdr.ts says "simplified" or "HARDWARE PASS REQUIRED" (e.g. the R820T PLL frequency math, the demod resample-ratio math), this module carries the exact same simplification forward with the exact same caveat, rather than inventing new unverified math on top of already-unverified math.

What's implemented (structurally — see caveats below)

1. USB Host Library lifecycle (rtlsdr_exp_start() / rtlsdr_usb_lib_daemon_task() / rtlsdr_exp_client_task()): usb_host_install(), usb_host_client_register() with an async event callback, and the two-task pump pattern the USB Host Library's async model requires (one task for usb_host_lib_handle_events(), one for usb_host_client_handle_events() — the latter also being how this module's own control- and bulk-transfer completion callbacks get dispatched, since the Library calls them synchronously from whichever task is pumping events, not from a hidden thread or ISR).
2. RTL2832U/Terratec device enumeration and vendor-ID matching (rtlsdr_try_bring_up(), rtlsdr_vendor_id_matches()): on a USB_HOST_CLIENT_EVENT_NEW_DEV event, opens the device, reads its device descriptor, and matches idVendor against 0x0BDA (RTL2832U) / 0x0CCD (Terratec-rebadged) — ported directly from sdr.ts's requestDevice() filter list (RTL2832U_VENDOR, TERRATEC_VENDOR). Faithfully carries over that file's specific choice to filter by vendor ID only, not product ID (its comment: "many dongles report product ids outside the handful we know, so filtering by productId hides them from the picker") — the known product-id list (RTLSDR_EXP_KNOWN_PRODUCT_IDS) is kept as an informational log line only, never a hard filter, exactly mirroring how RTL2832U_PRODUCTS is exported-but-unused-as-a-filter in sdr.ts.
3. RTL2832U init vendor-command sequence over USB control transfers (rtlsdr_run_init_sequence(), rtlsdr_demod_write(), rtlsdr_reg_write(), rtlsdr_i2c_write(), rtlsdr_exp_set_frequency(), rtlsdr_exp_set_sample_rate()): the same demod soft-reset → demod_ctl/ suspend-off block → standby-off → AGC-mode → R820T tuner power-up (through the I2C repeater) → sample-rate program → initial tune → streaming-endpoint reset → test-mode-off sequence as sdr.ts's open(), with the same register addresses/values and the same wValue/wIndex encoding ((block<<8)|0x10 / (page<<8)|address), translated from USBDevice.controlTransferOut() to usb_host_transfer_submit_control() with a manually-built usb_setup_packet_t.
4. A basic bulk-transfer read loop structure for pulling IQ sample data off the device (rtlsdr_start_bulk_streaming(), rtlsdr_bulk_xfer_cb()): unlike sdr.ts (which does one-shot await dev.transferIn(...) calls from inside its own async sweep() loop), the USB Host Library is callback-driven, so continuous streaming here is a small pipeline — N transfers (bulk_read_queue_ depth, default 4) of a fixed chunk size (bulk_read_chunk_bytes, default 16384B, enforced as a multiple of 512 the same way sdr.ts's readSamples() comment requires) are kept perpetually in flight; each completion callback copies the received bytes into a heap block, hands it to a FreeRTOS queue for the forward task, and immediately resubmits itself to keep the pipe full. This is the part most likely to need real-hardware tuning — see below.
5. A stub/structure for forwarding raw IQ data upstream (rtlsdr_exp_forward_iq_block(), rtlsdr_forward_task()) — see the architecture decision writeup immediately below.

IQ-forwarding architecture decision

The spec explicitly leaves this as a product-judgment call ("your call on whether this warrants a separate endpoint/stream vs. reusing the sensor telemetry shape with a binary/base64 payload"). Decision made here: a separate binary stream/endpoint, not a reuse of the JSON telemetry shape. Reasoning:

• Raw throughput is the wrong order of magnitude for the telemetry contract. Even a modest 2.048 Msps capture at 8 bits/sample/channel (RTL2832U's native ADC format) is interleaved I/Q bytes at roughly 4.1 MB/s. The design spec's POST /api/device/telemetry contract caps the readings array at ~64 entries, caps the request body at ~16KB, and rate-limits ingestion to roughly 1 request/second sustained per device — sized for a handful of scalar sensor readings (temperature, humidity, presence booleans), not a continuous multi-megabyte/second binary stream. Forcing IQ data through that shape would mean either violating those caps (defeating their whole purpose — bounding what a misbehaving/malicious device can push at the backend) or chopping IQ into thousands of tiny requests per second, which is worse for both sides than one continuous stream.
• Base64-in-JSON adds ~33% size overhead on top of a payload that's already too large for the telemetry shape, for no benefit — there's no reason to pay text-encoding tax on a payload nothing needs to eyeball as text.
• It's a fundamentally different kind of data with different backend handling needs. Telemetry readings feed the anomaly-detection baseline/threshold pipeline directly and cheaply (a few floats per sensor). IQ data needs FFT processing before it's useful for anything — an entirely different backend code path (closer to how the browser's own powerSpectrumDb() + SpectrumAnomalyDetector work today, server-side instead of client-side). Conflating the two request shapes would couple two things that should scale, rate-limit, and fail independently.

Given that, this module's stub (rtlsdr_exp_forward_iq_block()) targets a separate, currently-hypothetical endpoint (iq_upload_url in rtlsdr_exp_config_t, suggested path /api/device/iq-stream in code comments) carrying a small fixed binary header (magic, sequence number, center frequency, sample rate, payload length — see rtlsdr_iq_chunk_header_t) followed by the raw IQ bytes, POSTed as application/octet-stream with the same Authorization: Bearer <device token> auth as the regular telemetry loop. This endpoint does not exist anywhere in this repo. Building it is explicitly out of scope for this workstream (it's backend work, not firmware, and the spec doesn't assign a backend workstream to receive IQ data at all — only to receive scalar telemetry). The stub is inert by default (rtlsdr_exp_forward_ iq_block() returns ESP_OK immediately unless iq_upload_url is configured) specifically so this module can be compiled/enabled for its USB-host/bring-up behavior without requiring a backend that doesn't exist.

A real product decision here — genuinely open, not resolved by this workstream — is whether a POST-per-chunk model is even right versus a persistent WebSocket stream (the spec's /ws/device-feed design already establishes a WS pub/sub pattern on the backend for telemetry; a /ws/device-iq sibling might fit that architecture better than repeated HTTP POSTs, especially if backpressure/flow-control matters, which for a continuous stream it very much does). That's flagged here rather than silently decided, because it depends on backend design judgment as much as firmware judgment.

What's unverified — an honest, specific list for real hardware bring-up

Nothing below has run against real hardware. In rough order of "most likely to break first":

1. USB Host Library API surface. Function names, struct field names, and callback signatures (usb_host_client_config_t's .async. client_event_callback shape in particular — ESP-IDF has changed this API's shape across versions) are written from documented/remembered API shape, not checked against a real ESP-IDF checkout (none available in this environment). First thing a real bring-up needs: does this even compile against the ESP-IDF version Workstream I's project targets?
2. RTL2832U enumeration over a real ESP32-P4 USB-OTG host port — does usb_host_device_open() / usb_host_get_device_descriptor() actually see the dongle at all on this specific chip's USB-OTG controller (power delivery to the dongle over USB-host mode is itself a hardware question — does the ESP32-P4 dev board supply VBUS in host mode, does the dongle draw more current than it can supply).
3. The init vendor-command sequence itself — carried over unchanged from sdr.ts, which is itself unverified. Two layers of "reasoned, not tested." A logic analyzer / USB protocol analyzer trace against a real dongle (or cross-checking against real librtlsdr -T verbose output) is needed to confirm register values, not just transfer plumbing.
4. R820T PLL frequency math and demod resample-ratio math — both are the same simplified integer-N approximation sdr.ts uses (its own comment: "real librtlsdr computes the exact sdm/vco from a 28.8MHz crystal reference. Simplified... Marked for hardware."). Likely wrong or imprecise until replaced with the real librtlsdr formula and checked against an actual received signal.
5. Bulk transfer chunk size, pipeline depth, and timeout=0 choice (RTLSDR_EXP_BULK_CHUNK_BYTES/RTLSDR_EXP_BULK_QUEUE_DEPTH Kconfig, defaults 16384B / depth 4) — these are guesses. Real ESP32-P4 USB Host Library heap/DMA limits, achievable sustained throughput at 2.048 Msps (~4.1 MB/s), and whether timeout_ms = 0 (no timeout, i.e. "block until data or disconnect") is even the right transfer mode for this endpoint all need real measurement.
6. Drop-on-full backpressure policy (iq_blocks_dropped stat) — is silently dropping IQ blocks when the forward task falls behind acceptable, or does it need real flow control (e.g. throttling the bulk read rate itself)? Depends on (5) and on real WiFi uplink bandwidth from a real device on a real seeker's home network.
7. The proposed IQ-forwarding wire format and endpoint — entirely hypothetical (see architecture section above); no backend exists to validate the framing against, and the POST-vs-WebSocket question above is unresolved.
8. Concurrent USB Host Library ownership with the rest of the firmware project. rtlsdr_exp_start()/rtlsdr_exp_stop() call usb_host_ install()/usb_host_uninstall() directly, assuming this module is the only USB Host client in the project. If Workstream I's skeleton (or anything else) also needs USB host for something, this needs to change to a shared-ownership model (install once, both modules register as clients) — impossible to resolve without seeing that code, flagged here for whoever does the merge.
9. Memory footprint. Multiple in-flight 16KB bulk transfer buffers plus a forward queue plus esp_http_client buffers, alongside whatever Workstream I's WiFi/HTTP/BMP280/RD-03E stack already needs, on a single ESP32-P4's RAM — not sized or measured against a real linker map.

Build / integration notes

This component is off by default (RTLSDR_EXP_ENABLE Kconfig option, default n) so it cannot affect Workstream I's core build unless explicitly turned on via idf.py menuconfig → "RTL-SDR Experimental Module." To include it in a real project once Workstream I's skeleton exists:

1. Add this directory's components/ to the consuming project's EXTRA_COMPONENT_DIRS in the top-level CMakeLists.txt (or copy components/rtlsdr_experimental/ into that project's own components/).
2. From wherever app_main() sets up its other sensor drivers, e.g.:

   #include "rtlsdr_experimental.h"
   
   rtlsdr_exp_config_t sdr_cfg;
   rtlsdr_exp_config_default(&sdr_cfg);
   sdr_cfg.iq_upload_url = NULL; /* leave unset until a real backend
                                     endpoint exists — see README */
   sdr_cfg.device_bearer_token = DEVICE_BEARER_TOKEN; /* from
                                     main/device_config.h, same token
                                     used for the telemetry POST loop */
   rtlsdr_exp_handle_t sdr_handle;
   if (rtlsdr_exp_init(&sdr_cfg, &sdr_handle) == ESP_OK) {
       rtlsdr_exp_start(sdr_handle); /* non-blocking; watches for a dongle */
   }
   

3. idf.py build — not run in this environment (no ESP-IDF toolchain available); this module has only been reasoned about, not compiled. Treat "compiles" as an open question for the next person with a real toolchain, not a claim made here.

Verified in this environment: none of it, against hardware or a real compiler. What has been done: careful structural translation of a real, already-partially-researched protocol (sdr.ts) to the documented shape of a real, chip-specific API (ESP32-P4's USB Host Library), with every simplification and open question called out rather than hidden.