Resolved add/add conflict in the top-level firmware README: both I and J created one (J's brief said "create it if I hasn't", and both ran in isolated worktrees with no visibility into each other). Combined them — kept I's comprehensive core-project README as the base, appended J's real RTL-SDR technical section, dropped J's now-stale "Status of this directory" preamble (written when it couldn't see I's already-completed work) and its duplicate honesty-policy note. Updated the stale components/README.md placeholder to reflect that the module now exists.
565 lines
29 KiB
Markdown
565 lines
29 KiB
Markdown
# Quantumancy ESP32-P4 Sensor Node
|
||
|
||
Firmware for the paired hardware sensor node described in
|
||
[`docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md`](../../docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md)
|
||
("Workstream I — firmware, ESP-IDF C — core sensor node"). Real ESP-IDF C
|
||
(FreeRTOS-based), not Arduino, not pseudocode. Connects to the seeker's home
|
||
Wi-Fi, samples a small set of sensors, and POSTs readings to the Quantumancy
|
||
backend's `POST /api/device/telemetry` endpoint, which feeds them into the
|
||
séance's live anomaly-detection pipeline as a sixth signal source alongside
|
||
`wire`/`evp`/`radio`/`emf`.
|
||
|
||
## Honesty policy — READ THIS FIRST
|
||
|
||
> This app's whole ethos is "real signal processing on real data, and it
|
||
> says so when something is unverified."
|
||
|
||
**Nobody working on this had physical ESP32-P4 hardware, a BME280, or an
|
||
LD2410 module to flash and test against.** Everything in this directory is
|
||
real, structurally-correct ESP-IDF C, written against ESP-IDF's documented
|
||
APIs and each sensor's public datasheet/protocol documentation, and reasoned
|
||
about carefully — but it has **never been compiled with a real ESP-IDF
|
||
toolchain, never been flashed, and never talked to real hardware.** Treat
|
||
every claim below as "should work, per the docs" rather than "confirmed
|
||
working." See [What's verified vs. not](#whats-verified-vs-not) for the
|
||
specific, itemized breakdown — this mirrors the same convention
|
||
`frontend/src/lib/sdr.ts`'s `HARDWARE PASS REQUIRED` header comment uses
|
||
elsewhere in this repo.
|
||
|
||
## Directory layout
|
||
|
||
```
|
||
firmware/esp32p4-sensor-node/
|
||
├── CMakeLists.txt top-level ESP-IDF project file
|
||
├── sdkconfig.defaults seed config (idf.py generates the real sdkconfig)
|
||
├── README.md this file
|
||
├── components/ reserved for Workstream J (RTL-SDR), empty here
|
||
└── main/
|
||
├── CMakeLists.txt component registration
|
||
├── app_main.c entry point / boot sequence
|
||
├── device_config.h.example template you copy to device_config.h
|
||
├── wifi_manager.{h,c} Wi-Fi station mode connect/reconnect
|
||
├── telemetry_client.{h,c} HTTP POST task -> /api/device/telemetry
|
||
├── sensor_driver.h the sensor_driver_t registry interface
|
||
├── sensor_registry.{h,c} the concrete list of compiled-in drivers
|
||
├── bme280.{h,c} temperature/humidity/pressure over I2C
|
||
└── ld2410.{h,c} presence/distance over UART
|
||
```
|
||
|
||
## Build instructions
|
||
|
||
Requires an ESP-IDF install (v5.3 or newer — ESP32-P4 target support landed
|
||
around v5.2/5.3; this was written without a toolchain available to pin an
|
||
exact tested version, see honesty section). With `idf.py` on your `PATH`
|
||
(e.g. after sourcing ESP-IDF's `export.sh`):
|
||
|
||
```sh
|
||
cd firmware/esp32p4-sensor-node
|
||
|
||
# 1. Fill in your Wi-Fi + pairing details (see next section) — the build
|
||
# will fail on a missing #include until you do this.
|
||
cp main/device_config.h.example main/device_config.h
|
||
$EDITOR main/device_config.h
|
||
|
||
# 2. Target and build.
|
||
idf.py set-target esp32p4
|
||
idf.py build
|
||
|
||
# 3. Flash + monitor (adjust the port for your machine).
|
||
idf.py -p /dev/ttyUSB0 flash monitor
|
||
```
|
||
|
||
## Manual configuration (no provisioning UI — by design)
|
||
|
||
A full BLE/Wi-Fi-AP provisioning flow is explicitly out of scope for this
|
||
spec (see the spec's "Explicitly out of scope" section). Instead, you
|
||
hand-edit one header before building:
|
||
|
||
1. In the Quantumancy web app, sign in and create a device from your
|
||
account (name + optional sensor-type hint). The backend shows you a
|
||
**raw pairing token exactly once** — copy it immediately, it cannot be
|
||
retrieved again (same one-time-secret convention as the site's session
|
||
tokens).
|
||
2. `cp main/device_config.h.example main/device_config.h`
|
||
3. Edit `main/device_config.h` and fill in:
|
||
- `DEVICE_WIFI_SSID` / `DEVICE_WIFI_PASSWORD` — your home Wi-Fi.
|
||
- `DEVICE_BACKEND_BASE_URL` — the backend's base URL, no trailing slash.
|
||
- `DEVICE_PAIRING_TOKEN` — the raw token from step 1.
|
||
- `DEVICE_REPORT_INTERVAL_SEC` — optional, defaults to 15s.
|
||
4. `main/device_config.h` is listed in `.gitignore` — it will never be
|
||
committed. Never put real credentials in `device_config.h.example`
|
||
itself; that file is the template everyone else copies.
|
||
|
||
There is deliberately no other config path (no NVS-based captive portal, no
|
||
BLE provisioning) in this build — see the spec's scope boundary.
|
||
|
||
## What's verified vs. not
|
||
|
||
**Structurally verified** (reasoned through carefully against ESP-IDF's
|
||
documented API surface and each sensor's public protocol docs; internally
|
||
consistent; no known syntax errors or obviously-wrong API usage):
|
||
|
||
- Project skeleton (`CMakeLists.txt` × 2, `sdkconfig.defaults`,
|
||
`idf_component_register` call) follows ESP-IDF's standard project layout.
|
||
- Wi-Fi station-mode connect/reconnect (`wifi_manager.c`) follows ESP-IDF's
|
||
documented event-driven pattern (`WIFI_EVENT`/`IP_EVENT` handlers +
|
||
`EventGroupHandle_t`), extended with an exponential-backoff reconnect
|
||
timer instead of giving up after N tries.
|
||
- HTTP client (`telemetry_client.c`) builds the exact JSON shape the spec's
|
||
contract defines and POSTs it via `esp_http_client` with
|
||
`Authorization: Bearer <token>` and `Content-Type: application/json`.
|
||
- BME280 driver (`bme280.c`): register map and the double-precision
|
||
compensation formulas are transcribed from Bosch's public BME280
|
||
datasheet (rev 1.23, §4.2.2–4.2.3) — this is well-trodden, publicly
|
||
documented territory, and the formulas are checkable line-by-line against
|
||
the datasheet. Uses ESP-IDF's newer `driver/i2c_master.h` API (the
|
||
current idiomatic choice; the older `driver/i2c.h` is being phased out).
|
||
- LD2410 driver (`ld2410.c`): UART frame envelope (header/footer magic
|
||
bytes, length-prefixed payload) follows the shape consistently reported
|
||
across public LD2410 protocol write-ups. **The exact payload byte offsets
|
||
for target state / distances / energies are the single least-certain
|
||
piece of code in this entire firmware** — see the detailed note in
|
||
`ld2410_parse_payload()`. The driver defends itself with a head/tail
|
||
marker sanity check (`0xAA`/`0x55`) and silently skips anything that
|
||
doesn't match rather than reporting garbage, but that check catches
|
||
gross corruption, not subtle off-by-one offset errors.
|
||
- Sensor driver registry (`sensor_driver.h`, `sensor_registry.c`): a
|
||
`sensor_driver_t { name, init, read }` struct, a compile-time array of
|
||
them, and generic init/collect functions that `app_main.c` and
|
||
`telemetry_client.c` call without knowing which concrete sensors exist.
|
||
|
||
**NOT verified — requires real hardware bring-up:**
|
||
|
||
- `idf.py build` has never actually been run in this environment (no
|
||
ESP-IDF toolchain installed here) — there could be a typo, a missing
|
||
include, or an API signature mismatch against whatever exact ESP-IDF
|
||
version you build with that only a real compile will surface.
|
||
- I2C timing/electricals: pull-up resistor values, bus speed headroom,
|
||
cable length — none of this has been bench-tested.
|
||
- BME280 compensation formula correctness in practice: the math is
|
||
transcribed carefully, but "matches the datasheet" and "produces a
|
||
plausible number when this exact C runs on this exact silicon" are
|
||
different claims until someone compares a real reading to a reference
|
||
thermometer/barometer.
|
||
- LD2410 frame parsing, as above — verify against a logic analyzer capture
|
||
or a known-good reference implementation (e.g. the `ncmreynolds/ld2410`
|
||
or `iavorvel/MyLD2410` Arduino libraries, cross-checked) before trusting
|
||
field values.
|
||
- Wi-Fi reconnect behavior under real-world conditions (router reboot,
|
||
weak signal, captive portals) — the backoff logic is reasoned about, not
|
||
soak-tested.
|
||
- HTTP client behavior against the real backend: TLS handshake against
|
||
its actual certificate, real latency, real error responses. The
|
||
`sdkconfig.defaults` enables mbedTLS's full certificate bundle for this,
|
||
but that's untested against the live deploy.
|
||
- Timing/power: task stack sizes (`telemetry_task`'s 8192 words, etc.) are
|
||
reasonable guesses, not measured high-water-marks from a real run.
|
||
- **The ESP32-P4-has-no-integrated-Wi-Fi caveat below** — this is a real
|
||
hardware architecture question, not just an untested detail.
|
||
|
||
## Important hardware caveat: ESP32-P4 has no integrated Wi-Fi radio
|
||
|
||
The ESP32-P4 SoC (per Espressif's own published specs) has **no built-in
|
||
2.4GHz radio**. A real deployment needs one of:
|
||
|
||
- **A companion Wi-Fi chip** (e.g. ESP32-C6) wired to the P4 via SDIO or
|
||
SPI, running Espressif's "esp-hosted" firmware/driver stack. Critically,
|
||
esp-hosted presents the *same* `esp_wifi`/`esp_netif` API this firmware
|
||
already uses — so `wifi_manager.c` should not need to change, only board
|
||
wiring and `sdkconfig` (host-side esp-hosted config) would.
|
||
- **Building this same code against a Wi-Fi-native target instead**, e.g.
|
||
`idf.py set-target esp32s3` or `esp32c6`. The application code
|
||
(`wifi_manager.c`, `telemetry_client.c`, the sensor drivers) is written
|
||
against the standard API surface and doesn't reference P4-specific
|
||
peripherals for anything except I2C/UART GPIO numbers, so it should be
|
||
largely target-portable.
|
||
|
||
This wasn't in the original spec's framing but matters enough for a real
|
||
build that it's called out here explicitly, in the honesty-policy spirit —
|
||
better to flag a real hardware-architecture gap than let someone discover
|
||
it after ordering a bare P4 dev board expecting it to just join Wi-Fi.
|
||
|
||
## Wiring / pinout
|
||
|
||
### BME280 (I2C) — temperature, humidity, pressure
|
||
|
||
Chosen as the concrete default sensor per the spec ("a common,
|
||
well-documented sensor... pick this as the concrete default since no
|
||
specific part number was given").
|
||
|
||
| BME280 pin | Connects to |
|
||
|------------|---------------------------------------|
|
||
| VCC | 3V3 |
|
||
| GND | GND |
|
||
| SDA | GPIO8 (`BME280_I2C_SDA_GPIO`) |
|
||
| SCL | GPIO9 (`BME280_I2C_SCL_GPIO`) |
|
||
| CSB | VCC (selects I2C mode, not SPI) |
|
||
| SDO | GND → I2C address `0x76` (default assumed; tie to VCC + change `BME280_I2C_ADDR` for `0x77`) |
|
||
|
||
GPIO numbers are `#define`s at the top of `bme280.h` — override them there
|
||
(or via a future `idf.py menuconfig` entry) to match your actual wiring.
|
||
100kHz I2C clock by default (`BME280_I2C_CLK_HZ`); the part supports faster
|
||
modes if your wiring/pull-ups support it.
|
||
|
||
### LD2410 (UART) — presence, distance, motion
|
||
|
||
**Chosen over a plain PIR** — see the rationale in `ld2410.h`'s header
|
||
comment: the LD2410 reports moving-target and stationary-target distance
|
||
and energy separately, not just a boolean, which is richer signal for the
|
||
anomaly pipeline and better matches this app's "believable" ethos (it can
|
||
distinguish "someone crossed the room" from "the sitter shifted in their
|
||
chair" in a way a boolean PIR cannot). The tradeoff is a materially more
|
||
complex protocol than a PIR's single GPIO pin — see the honesty note in
|
||
[What's verified vs. not](#whats-verified-vs-not) about the LD2410 frame
|
||
parser being the least-certain code in this firmware. If you'd rather start
|
||
with a boolean PIR for a faster, more certain first bring-up, it fits the
|
||
same `sensor_driver_t` interface — see
|
||
[Adding a new sensor](#adding-a-new-sensor) below.
|
||
|
||
| LD2410 pin | Connects to |
|
||
|------------|----------------------------------------|
|
||
| VCC | 5V (sensor front-end runs at 5V; confirm your board revision's UART logic level before wiring directly to a 3.3V-only UART pin) |
|
||
| GND | GND |
|
||
| TX | GPIO17 (`LD2410_UART_RX_GPIO`, ESP32 RX) |
|
||
| RX | GPIO18 (`LD2410_UART_TX_GPIO`, ESP32 TX) |
|
||
|
||
Default UART settings: 256000 baud, 8N1 (module factory default), reporting
|
||
in "basic" (non-engineering) mode. GPIO numbers and baud rate are
|
||
`#define`s at the top of `ld2410.h`.
|
||
|
||
## Sensor driver registry — the extensibility pattern
|
||
|
||
`sensor_driver.h` defines:
|
||
|
||
```c
|
||
typedef struct {
|
||
char sensor_type[SENSOR_READING_TYPE_MAXLEN];
|
||
double value;
|
||
char unit[SENSOR_READING_UNIT_MAXLEN];
|
||
cJSON *metadata; // nullable; NULL serializes as {}
|
||
} sensor_reading_t;
|
||
|
||
typedef struct sensor_driver {
|
||
const char *name;
|
||
esp_err_t (*init)(void);
|
||
esp_err_t (*read)(sensor_reading_t *out, size_t max_out, size_t *out_count);
|
||
} sensor_driver_t;
|
||
```
|
||
|
||
`sensor_registry.c` holds a compile-time array of these (currently BME280
|
||
and LD2410) and two generic functions, `sensor_registry_init_all()` and
|
||
`sensor_registry_collect()`, that `app_main.c` and `telemetry_client.c`
|
||
call without ever referencing `bme280.c`/`ld2410.c` directly. One driver
|
||
failing `init()` or `read()` is logged and skipped — it doesn't take the
|
||
whole node offline.
|
||
|
||
### Adding a new sensor
|
||
|
||
1. Write `main/my_sensor.h` / `main/my_sensor.c` implementing `init()` and
|
||
`read()` matching `sensor_driver_t`'s function pointer signatures.
|
||
2. Add `"my_sensor.c"` to the `SRCS` list in `main/CMakeLists.txt`.
|
||
3. `#include "my_sensor.h"` in `sensor_registry.c` and add one line to the
|
||
`s_drivers[]` array:
|
||
```c
|
||
{ .name = "my_sensor", .init = my_sensor_init, .read = my_sensor_read },
|
||
```
|
||
|
||
Nothing in `app_main.c`, `telemetry_client.c`, or the main reporting loop's
|
||
control flow needs to change — that's the whole point of this structure per
|
||
the spec.
|
||
|
||
## Backend contract this firmware targets
|
||
|
||
From the spec (binding, see the spec file for the authoritative version):
|
||
|
||
```json
|
||
POST /api/device/telemetry
|
||
Authorization: Bearer <raw pairing token>
|
||
Content-Type: application/json
|
||
|
||
{
|
||
"readings": [
|
||
{"sensor_type": "presence", "value": 1, "unit": "bool", "metadata": {}},
|
||
{"sensor_type": "temperature", "value": 21.4, "unit": "c", "metadata": {}},
|
||
{"sensor_type": "humidity", "value": 47.2, "unit": "pct", "metadata": {}},
|
||
{"sensor_type": "pressure", "value": 1013.2, "unit": "hpa", "metadata": {}}
|
||
]
|
||
}
|
||
```
|
||
|
||
This firmware's BME280 driver emits `temperature`/`humidity`/`pressure`
|
||
exactly as shown; its LD2410 driver emits `presence` as a `0`/`1` boolean
|
||
in `value` with the richer distance/energy data folded into `metadata`
|
||
(`moving_distance_cm`, `moving_energy`, `stationary_distance_cm`,
|
||
`stationary_energy`, `detection_distance_cm`, `target_state`).
|
||
|
||
## Out of scope here
|
||
|
||
Per the spec: thermal camera support, a full BLE/Wi-Fi-AP provisioning UX,
|
||
on-device spectrum analysis/FFT (the RTL-SDR module below forwards raw IQ
|
||
upstream rather than analyzing on-device), and anything on the
|
||
backend/frontend side (Workstreams G, H, K).
|
||
|
||
---
|
||
|
||
## Workstream J — RTL-SDR experimental module
|
||
|
||
**⚠️ This is the least certain part of the entire firmware build. Read
|
||
this whole section before touching it.**
|
||
|
||
### What this is
|
||
|
||
A clearly-separated, opt-in, experimental module exploring USB-host
|
||
communication with an RTL2832U-based SDR dongle over the ESP32-P4's
|
||
native USB-OTG host controller (USB Host Library, `usb_host.h` — a real,
|
||
documented capability of this specific chip, unlike most ESP32 variants).
|
||
The idea, per the design spec: attach a cheap RTL-SDR dongle to the
|
||
sensor node, treat "spirit radio scanning" as a hardware-backed mode
|
||
instead of only a browser WebUSB feature.
|
||
|
||
**Be honest about the real constraint (this is the spec's framing, and
|
||
it's correct):** wideband IQ sample rates and FFT processing are
|
||
demanding relative to an MCU's compute, even one with ESP32-P4's AI
|
||
accelerator. **On-device spectrum analysis is not what this module
|
||
attempts.** The realistic architecture — and the one implemented here —
|
||
is: pull raw IQ samples off the dongle via USB host, and forward them
|
||
upstream for the *backend* to FFT/analyze (the same job the browser
|
||
already does client-side today via `frontend/src/lib/fft.ts` +
|
||
`frontend/src/lib/sdr.ts`'s `SpectrumAnomalyDetector`). Even that
|
||
"just pass the bytes through" architecture needs real USB throughput
|
||
numbers to know if it's viable — see "What's unverified" below.
|
||
|
||
### Primary reference
|
||
|
||
`frontend/src/lib/sdr.ts` — this repo's existing browser-based (WebUSB)
|
||
RTL2832U + R820T driver. It's real, already-researched protocol detail
|
||
against the public librtlsdr register documentation (vendor commands,
|
||
I2C-repeater tuner access, the demod/tuner init sequence), itself marked
|
||
`HARDWARE PASS REQUIRED` since it's never been run against a real dongle
|
||
either. This firmware module is a direct **port** of that file's control-
|
||
transfer sequence from WebUSB JS calls to ESP-IDF USB Host Library C
|
||
calls — line-by-line correspondences are called out in code comments
|
||
(e.g. `rtlsdr_run_init_sequence()` mirrors `open()` in sdr.ts almost
|
||
register-for-register). It intentionally does not re-derive any register
|
||
math from scratch; wherever sdr.ts says "simplified" or "HARDWARE PASS
|
||
REQUIRED" (e.g. the R820T PLL frequency math, the demod resample-ratio
|
||
math), this module carries the exact same simplification forward with the
|
||
exact same caveat, rather than inventing new unverified math on top of
|
||
already-unverified math.
|
||
|
||
### What's implemented (structurally — see caveats below)
|
||
|
||
1. **USB Host Library lifecycle** (`rtlsdr_exp_start()` /
|
||
`rtlsdr_usb_lib_daemon_task()` / `rtlsdr_exp_client_task()`):
|
||
`usb_host_install()`, `usb_host_client_register()` with an async event
|
||
callback, and the two-task pump pattern the USB Host Library's async
|
||
model requires (one task for `usb_host_lib_handle_events()`, one for
|
||
`usb_host_client_handle_events()` — the latter also being how this
|
||
module's own control- and bulk-transfer completion callbacks get
|
||
dispatched, since the Library calls them synchronously from whichever
|
||
task is pumping events, not from a hidden thread or ISR).
|
||
2. **RTL2832U/Terratec device enumeration and vendor-ID matching**
|
||
(`rtlsdr_try_bring_up()`, `rtlsdr_vendor_id_matches()`): on a
|
||
`USB_HOST_CLIENT_EVENT_NEW_DEV` event, opens the device, reads its
|
||
device descriptor, and matches `idVendor` against `0x0BDA`
|
||
(RTL2832U) / `0x0CCD` (Terratec-rebadged) — ported directly from
|
||
`sdr.ts`'s `requestDevice()` filter list (`RTL2832U_VENDOR`,
|
||
`TERRATEC_VENDOR`). Faithfully carries over that file's specific
|
||
choice to filter by **vendor ID only**, not product ID (its comment:
|
||
"many dongles report product ids outside the handful we know, so
|
||
filtering by productId hides them from the picker") — the known
|
||
product-id list (`RTLSDR_EXP_KNOWN_PRODUCT_IDS`) is kept as an
|
||
informational log line only, never a hard filter, exactly mirroring
|
||
how `RTL2832U_PRODUCTS` is exported-but-unused-as-a-filter in
|
||
`sdr.ts`.
|
||
3. **RTL2832U init vendor-command sequence** over USB control transfers
|
||
(`rtlsdr_run_init_sequence()`, `rtlsdr_demod_write()`,
|
||
`rtlsdr_reg_write()`, `rtlsdr_i2c_write()`, `rtlsdr_exp_set_frequency()`,
|
||
`rtlsdr_exp_set_sample_rate()`): the same demod soft-reset → demod_ctl/
|
||
suspend-off block → standby-off → AGC-mode → R820T tuner power-up
|
||
(through the I2C repeater) → sample-rate program → initial tune →
|
||
streaming-endpoint reset → test-mode-off sequence as `sdr.ts`'s
|
||
`open()`, with the same register addresses/values and the same
|
||
`wValue`/`wIndex` encoding (`(block<<8)|0x10` / `(page<<8)|address`),
|
||
translated from `USBDevice.controlTransferOut()` to
|
||
`usb_host_transfer_submit_control()` with a manually-built
|
||
`usb_setup_packet_t`.
|
||
4. **A basic bulk-transfer read loop structure** for pulling IQ sample
|
||
data off the device (`rtlsdr_start_bulk_streaming()`,
|
||
`rtlsdr_bulk_xfer_cb()`): unlike `sdr.ts` (which does one-shot
|
||
`await dev.transferIn(...)` calls from inside its own async `sweep()`
|
||
loop), the USB Host Library is callback-driven, so continuous
|
||
streaming here is a small pipeline — N transfers (`bulk_read_queue_
|
||
depth`, default 4) of a fixed chunk size (`bulk_read_chunk_bytes`,
|
||
default 16384B, enforced as a multiple of 512 the same way `sdr.ts`'s
|
||
`readSamples()` comment requires) are kept perpetually in flight; each
|
||
completion callback copies the received bytes into a heap block, hands
|
||
it to a FreeRTOS queue for the forward task, and immediately resubmits
|
||
itself to keep the pipe full. **This is the part most likely to need
|
||
real-hardware tuning** — see below.
|
||
5. **A stub/structure for forwarding raw IQ data upstream**
|
||
(`rtlsdr_exp_forward_iq_block()`, `rtlsdr_forward_task()`) — see the
|
||
architecture decision writeup immediately below.
|
||
|
||
### IQ-forwarding architecture decision
|
||
|
||
The spec explicitly leaves this as a product-judgment call ("your call on
|
||
whether this warrants a separate endpoint/stream vs. reusing the sensor
|
||
telemetry shape with a binary/base64 payload"). Decision made here:
|
||
**a separate binary stream/endpoint, not a reuse of the JSON telemetry
|
||
shape.** Reasoning:
|
||
|
||
- **Raw throughput is the wrong order of magnitude for the telemetry
|
||
contract.** Even a modest 2.048 Msps capture at 8 bits/sample/channel
|
||
(RTL2832U's native ADC format) is interleaved I/Q bytes at roughly
|
||
4.1 MB/s. The design spec's `POST /api/device/telemetry` contract caps
|
||
the `readings` array at ~64 entries, caps the request body at ~16KB,
|
||
and rate-limits ingestion to roughly 1 request/second sustained per
|
||
device — sized for a handful of scalar sensor readings (temperature,
|
||
humidity, presence booleans), not a continuous multi-megabyte/second
|
||
binary stream. Forcing IQ data through that shape would mean either
|
||
violating those caps (defeating their whole purpose — bounding what a
|
||
misbehaving/malicious device can push at the backend) or chopping IQ
|
||
into thousands of tiny requests per second, which is worse for both
|
||
sides than one continuous stream.
|
||
- **Base64-in-JSON adds ~33% size overhead** on top of a payload that's
|
||
already too large for the telemetry shape, for no benefit — there's no
|
||
reason to pay text-encoding tax on a payload nothing needs to
|
||
eyeball as text.
|
||
- **It's a fundamentally different kind of data with different backend
|
||
handling needs.** Telemetry readings feed the anomaly-detection
|
||
baseline/threshold pipeline directly and cheaply (a few floats per
|
||
sensor). IQ data needs FFT processing before it's useful for anything
|
||
— an entirely different backend code path (closer to how the browser's
|
||
own `powerSpectrumDb()` + `SpectrumAnomalyDetector` work today,
|
||
server-side instead of client-side). Conflating the two request shapes
|
||
would couple two things that should scale, rate-limit, and fail
|
||
independently.
|
||
|
||
Given that, this module's stub (`rtlsdr_exp_forward_iq_block()`) targets
|
||
a **separate, currently-hypothetical** endpoint (`iq_upload_url` in
|
||
`rtlsdr_exp_config_t`, suggested path `/api/device/iq-stream` in code
|
||
comments) carrying a small fixed binary header (magic, sequence number,
|
||
center frequency, sample rate, payload length — see
|
||
`rtlsdr_iq_chunk_header_t`) followed by the raw IQ bytes, POSTed as
|
||
`application/octet-stream` with the same `Authorization: Bearer <device
|
||
token>` auth as the regular telemetry loop. **This endpoint does not
|
||
exist anywhere in this repo.** Building it is explicitly out of scope for
|
||
this workstream (it's backend work, not firmware, and the spec doesn't
|
||
assign a backend workstream to receive IQ data at all — only to receive
|
||
scalar telemetry). The stub is inert by default (`rtlsdr_exp_forward_
|
||
iq_block()` returns `ESP_OK` immediately unless `iq_upload_url` is
|
||
configured) specifically so this module can be compiled/enabled for its
|
||
USB-host/bring-up behavior without requiring a backend that doesn't
|
||
exist.
|
||
|
||
A real product decision here — genuinely open, not resolved by this
|
||
workstream — is whether a POST-per-chunk model is even right versus a
|
||
persistent WebSocket stream (the spec's `/ws/device-feed` design already
|
||
establishes a WS pub/sub pattern on the backend for telemetry; a
|
||
`/ws/device-iq` sibling might fit that architecture better than repeated
|
||
HTTP POSTs, especially if backpressure/flow-control matters, which for a
|
||
continuous stream it very much does). That's flagged here rather than
|
||
silently decided, because it depends on backend design judgment as much
|
||
as firmware judgment.
|
||
|
||
### What's unverified — an honest, specific list for real hardware bring-up
|
||
|
||
Nothing below has run against real hardware. In rough order of "most
|
||
likely to break first":
|
||
|
||
1. **USB Host Library API surface.** Function names, struct field names,
|
||
and callback signatures (`usb_host_client_config_t`'s `.async.
|
||
client_event_callback` shape in particular — ESP-IDF has changed this
|
||
API's shape across versions) are written from documented/remembered
|
||
API shape, not checked against a real ESP-IDF checkout (none available
|
||
in this environment). **First thing a real bring-up needs: does this
|
||
even compile against the ESP-IDF version Workstream I's project
|
||
targets?**
|
||
2. **RTL2832U enumeration over a real ESP32-P4 USB-OTG host port** — does
|
||
`usb_host_device_open()` / `usb_host_get_device_descriptor()` actually
|
||
see the dongle at all on this specific chip's USB-OTG controller (power
|
||
delivery to the dongle over USB-host mode is itself a hardware
|
||
question — does the ESP32-P4 dev board supply VBUS in host mode, does
|
||
the dongle draw more current than it can supply).
|
||
3. **The init vendor-command sequence itself** — carried over unchanged
|
||
from `sdr.ts`, which is *itself* unverified. Two layers of "reasoned,
|
||
not tested." A logic analyzer / USB protocol analyzer trace against a
|
||
real dongle (or cross-checking against real librtlsdr `-T` verbose
|
||
output) is needed to confirm register values, not just transfer
|
||
plumbing.
|
||
4. **R820T PLL frequency math and demod resample-ratio math** — both are
|
||
the same simplified integer-N approximation `sdr.ts` uses (its own
|
||
comment: "real librtlsdr computes the exact sdm/vco from a 28.8MHz
|
||
crystal reference. Simplified... Marked for hardware."). Likely wrong
|
||
or imprecise until replaced with the real librtlsdr formula and
|
||
checked against an actual received signal.
|
||
5. **Bulk transfer chunk size, pipeline depth, and timeout=0 choice**
|
||
(`RTLSDR_EXP_BULK_CHUNK_BYTES`/`RTLSDR_EXP_BULK_QUEUE_DEPTH` Kconfig,
|
||
defaults 16384B / depth 4) — these are guesses. Real ESP32-P4 USB
|
||
Host Library heap/DMA limits, achievable sustained throughput at
|
||
2.048 Msps (~4.1 MB/s), and whether `timeout_ms = 0` (no timeout, i.e.
|
||
"block until data or disconnect") is even the right transfer mode for
|
||
this endpoint all need real measurement.
|
||
6. **Drop-on-full backpressure policy** (`iq_blocks_dropped` stat) — is
|
||
silently dropping IQ blocks when the forward task falls behind
|
||
acceptable, or does it need real flow control (e.g. throttling the
|
||
bulk read rate itself)? Depends on (5) and on real WiFi uplink
|
||
bandwidth from a real device on a real seeker's home network.
|
||
7. **The proposed IQ-forwarding wire format and endpoint** — entirely
|
||
hypothetical (see architecture section above); no backend exists to
|
||
validate the framing against, and the POST-vs-WebSocket question above
|
||
is unresolved.
|
||
8. **Concurrent USB Host Library ownership with the rest of the firmware
|
||
project.** `rtlsdr_exp_start()`/`rtlsdr_exp_stop()` call `usb_host_
|
||
install()`/`usb_host_uninstall()` directly, assuming this module is the
|
||
*only* USB Host client in the project. If Workstream I's skeleton (or
|
||
anything else) also needs USB host for something, this needs to change
|
||
to a shared-ownership model (install once, both modules register as
|
||
clients) — impossible to resolve without seeing that code, flagged
|
||
here for whoever does the merge.
|
||
9. **Memory footprint.** Multiple in-flight 16KB bulk transfer buffers
|
||
plus a forward queue plus `esp_http_client` buffers, alongside whatever
|
||
Workstream I's WiFi/HTTP/BME280/presence stack already needs, on a
|
||
single ESP32-P4's RAM — not sized or measured against a real linker
|
||
map.
|
||
|
||
### Build / integration notes
|
||
|
||
This component is **off by default** (`RTLSDR_EXP_ENABLE` Kconfig option,
|
||
default `n`) so it cannot affect Workstream I's core build unless
|
||
explicitly turned on via `idf.py menuconfig` → "RTL-SDR Experimental
|
||
Module." To include it in a real project once Workstream I's skeleton
|
||
exists:
|
||
|
||
1. Add this directory's `components/` to the consuming project's
|
||
`EXTRA_COMPONENT_DIRS` in the top-level `CMakeLists.txt` (or copy
|
||
`components/rtlsdr_experimental/` into that project's own
|
||
`components/`).
|
||
2. From wherever `app_main()` sets up its other sensor drivers, e.g.:
|
||
```c
|
||
#include "rtlsdr_experimental.h"
|
||
|
||
rtlsdr_exp_config_t sdr_cfg;
|
||
rtlsdr_exp_config_default(&sdr_cfg);
|
||
sdr_cfg.iq_upload_url = NULL; /* leave unset until a real backend
|
||
endpoint exists — see README */
|
||
sdr_cfg.device_bearer_token = DEVICE_BEARER_TOKEN; /* from
|
||
main/device_config.h, same token
|
||
used for the telemetry POST loop */
|
||
rtlsdr_exp_handle_t sdr_handle;
|
||
if (rtlsdr_exp_init(&sdr_cfg, &sdr_handle) == ESP_OK) {
|
||
rtlsdr_exp_start(sdr_handle); /* non-blocking; watches for a dongle */
|
||
}
|
||
```
|
||
3. `idf.py build` — **not run in this environment** (no ESP-IDF
|
||
toolchain available); this module has only been reasoned about, not
|
||
compiled. Treat "compiles" as an open question for the next person
|
||
with a real toolchain, not a claim made here.
|
||
|
||
Verified in this environment: none of it, against hardware or a real
|
||
compiler. What *has* been done: careful structural translation of a
|
||
real, already-partially-researched protocol (`sdr.ts`) to the documented
|
||
shape of a real, chip-specific API (ESP32-P4's USB Host Library), with
|
||
every simplification and open question called out rather than hidden.
|