qtalker---/firmware/esp32p4-sensor-node/README.md

# ESP32-P4 Sensor Node — firmware

Firmware for the physical ESP32-P4 "Ultimate Quantum Box" sensor node
(see the ESP32-P4 Sensor Node design spec:
`docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md`). Joins
the seeker's home WiFi and streams sensor readings to `POST
/api/device/telemetry` on their paired account, which feeds the séance's
live anomaly-detection pipeline exactly like the browser-based Wire
Ghost/EVP/Spirit Radio/EMF modes already do.

## Honesty-policy note (binding on this whole directory)

This app's whole ethos is "real signal processing on real data, and it
says so when something is unverified" (see the frontend's Spirit Radio
`HARDWARE PASS REQUIRED` convention in `frontend/src/lib/sdr.ts`). Nobody
working on this firmware has physical ESP32-P4 hardware to flash and test
against, and this development environment has no ESP-IDF toolchain to
even compile it. **Every file under this directory is written to be
structurally correct ESP-IDF C against the documented API shape, and
reasoned about carefully — not verified against real hardware or even a
real build.** Anything that needs a real board to actually confirm is
called out explicitly, in code comments and in this README, rather than
glossed over. Treat "compiles cleanly / structurally sound" and "verified
on-device" as two entirely different claims — this repo only makes the
first one for anything under `firmware/`.

## Status of this directory

This copy of the repo currently contains **only Workstream J** (the
RTL-SDR experimental module, documented in full below). Workstream I
— the core firmware project skeleton (`CMakeLists.txt`,
`sdkconfig.defaults`, the `main/` component with WiFi station-mode
connect, the `/api/device/telemetry` HTTP client task, `main/
device_config.h`, and the BME280 + presence-sensor drivers) — was built
in a separate, isolated worktree that this workstream cannot see or
depend on. **A real build of this project needs Workstream I's skeleton
merged in alongside what's here.** Nothing in this directory currently
compiles standalone into a flashable image; the `components/
rtlsdr_experimental/` module is a self-contained ESP-IDF *component*,
deliberately structured so it can be dropped into Workstream I's project
via `EXTRA_COMPONENT_DIRS` (or copied under that project's own
`components/`) without editing any file that workstream owns. Once
merged, this README should be extended with Workstream I's own build
steps, wiring/pinout notes for the BME280 (I2C) and presence sensor
(UART/GPIO), and its own verified-vs-not accounting — that content
doesn't exist here yet because this workstream never had visibility into
that code.

## Directory layout (this workstream's contribution)

```
firmware/esp32p4-sensor-node/
├── README.md                              (this file)
└── components/
    └── rtlsdr_experimental/                Self-contained ESP-IDF component.
        ├── CMakeLists.txt
        ├── Kconfig                         Off by default (RTLSDR_EXP_ENABLE=n).
        ├── include/
        │   └── rtlsdr_experimental.h        Public API + full disclaimer header.
        └── rtlsdr_experimental.c            Implementation.
```

---

## Workstream J — RTL-SDR experimental module (this workstream)

**⚠️ This is the least certain part of the entire firmware build. Read
this whole section before touching it.**

### What this is

A clearly-separated, opt-in, experimental module exploring USB-host
communication with an RTL2832U-based SDR dongle over the ESP32-P4's
native USB-OTG host controller (USB Host Library, `usb_host.h` — a real,
documented capability of this specific chip, unlike most ESP32 variants).
The idea, per the design spec: attach a cheap RTL-SDR dongle to the
sensor node, treat "spirit radio scanning" as a hardware-backed mode
instead of only a browser WebUSB feature.

**Be honest about the real constraint (this is the spec's framing, and
it's correct):** wideband IQ sample rates and FFT processing are
demanding relative to an MCU's compute, even one with ESP32-P4's AI
accelerator. **On-device spectrum analysis is not what this module
attempts.** The realistic architecture — and the one implemented here —
is: pull raw IQ samples off the dongle via USB host, and forward them
upstream for the *backend* to FFT/analyze (the same job the browser
already does client-side today via `frontend/src/lib/fft.ts` +
`frontend/src/lib/sdr.ts`'s `SpectrumAnomalyDetector`). Even that
"just pass the bytes through" architecture needs real USB throughput
numbers to know if it's viable — see "What's unverified" below.

### Primary reference

`frontend/src/lib/sdr.ts` — this repo's existing browser-based (WebUSB)
RTL2832U + R820T driver. It's real, already-researched protocol detail
against the public librtlsdr register documentation (vendor commands,
I2C-repeater tuner access, the demod/tuner init sequence), itself marked
`HARDWARE PASS REQUIRED` since it's never been run against a real dongle
either. This firmware module is a direct **port** of that file's control-
transfer sequence from WebUSB JS calls to ESP-IDF USB Host Library C
calls — line-by-line correspondences are called out in code comments
(e.g. `rtlsdr_run_init_sequence()` mirrors `open()` in sdr.ts almost
register-for-register). It intentionally does not re-derive any register
math from scratch; wherever sdr.ts says "simplified" or "HARDWARE PASS
REQUIRED" (e.g. the R820T PLL frequency math, the demod resample-ratio
math), this module carries the exact same simplification forward with the
exact same caveat, rather than inventing new unverified math on top of
already-unverified math.

### What's implemented (structurally — see caveats below)

1. **USB Host Library lifecycle** (`rtlsdr_exp_start()` /
   `rtlsdr_usb_lib_daemon_task()` / `rtlsdr_exp_client_task()`):
   `usb_host_install()`, `usb_host_client_register()` with an async event
   callback, and the two-task pump pattern the USB Host Library's async
   model requires (one task for `usb_host_lib_handle_events()`, one for
   `usb_host_client_handle_events()` — the latter also being how this
   module's own control- and bulk-transfer completion callbacks get
   dispatched, since the Library calls them synchronously from whichever
   task is pumping events, not from a hidden thread or ISR).
2. **RTL2832U/Terratec device enumeration and vendor-ID matching**
   (`rtlsdr_try_bring_up()`, `rtlsdr_vendor_id_matches()`): on a
   `USB_HOST_CLIENT_EVENT_NEW_DEV` event, opens the device, reads its
   device descriptor, and matches `idVendor` against `0x0BDA`
   (RTL2832U) / `0x0CCD` (Terratec-rebadged) — ported directly from
   `sdr.ts`'s `requestDevice()` filter list (`RTL2832U_VENDOR`,
   `TERRATEC_VENDOR`). Faithfully carries over that file's specific
   choice to filter by **vendor ID only**, not product ID (its comment:
   "many dongles report product ids outside the handful we know, so
   filtering by productId hides them from the picker") — the known
   product-id list (`RTLSDR_EXP_KNOWN_PRODUCT_IDS`) is kept as an
   informational log line only, never a hard filter, exactly mirroring
   how `RTL2832U_PRODUCTS` is exported-but-unused-as-a-filter in
   `sdr.ts`.
3. **RTL2832U init vendor-command sequence** over USB control transfers
   (`rtlsdr_run_init_sequence()`, `rtlsdr_demod_write()`,
   `rtlsdr_reg_write()`, `rtlsdr_i2c_write()`, `rtlsdr_exp_set_frequency()`,
   `rtlsdr_exp_set_sample_rate()`): the same demod soft-reset → demod_ctl/
   suspend-off block → standby-off → AGC-mode → R820T tuner power-up
   (through the I2C repeater) → sample-rate program → initial tune →
   streaming-endpoint reset → test-mode-off sequence as `sdr.ts`'s
   `open()`, with the same register addresses/values and the same
   `wValue`/`wIndex` encoding (`(block<<8)|0x10` / `(page<<8)|address`),
   translated from `USBDevice.controlTransferOut()` to
   `usb_host_transfer_submit_control()` with a manually-built
   `usb_setup_packet_t`.
4. **A basic bulk-transfer read loop structure** for pulling IQ sample
   data off the device (`rtlsdr_start_bulk_streaming()`,
   `rtlsdr_bulk_xfer_cb()`): unlike `sdr.ts` (which does one-shot
   `await dev.transferIn(...)` calls from inside its own async `sweep()`
   loop), the USB Host Library is callback-driven, so continuous
   streaming here is a small pipeline — N transfers (`bulk_read_queue_
   depth`, default 4) of a fixed chunk size (`bulk_read_chunk_bytes`,
   default 16384B, enforced as a multiple of 512 the same way `sdr.ts`'s
   `readSamples()` comment requires) are kept perpetually in flight; each
   completion callback copies the received bytes into a heap block, hands
   it to a FreeRTOS queue for the forward task, and immediately resubmits
   itself to keep the pipe full. **This is the part most likely to need
   real-hardware tuning** — see below.
5. **A stub/structure for forwarding raw IQ data upstream**
   (`rtlsdr_exp_forward_iq_block()`, `rtlsdr_forward_task()`) — see the
   architecture decision writeup immediately below.

### IQ-forwarding architecture decision

The spec explicitly leaves this as a product-judgment call ("your call on
whether this warrants a separate endpoint/stream vs. reusing the sensor
telemetry shape with a binary/base64 payload"). Decision made here:
**a separate binary stream/endpoint, not a reuse of the JSON telemetry
shape.** Reasoning:

- **Raw throughput is the wrong order of magnitude for the telemetry
  contract.** Even a modest 2.048 Msps capture at 8 bits/sample/channel
  (RTL2832U's native ADC format) is interleaved I/Q bytes at roughly
  4.1 MB/s. The design spec's `POST /api/device/telemetry` contract caps
  the `readings` array at ~64 entries, caps the request body at ~16KB,
  and rate-limits ingestion to roughly 1 request/second sustained per
  device — sized for a handful of scalar sensor readings (temperature,
  humidity, presence booleans), not a continuous multi-megabyte/second
  binary stream. Forcing IQ data through that shape would mean either
  violating those caps (defeating their whole purpose — bounding what a
  misbehaving/malicious device can push at the backend) or chopping IQ
  into thousands of tiny requests per second, which is worse for both
  sides than one continuous stream.
- **Base64-in-JSON adds ~33% size overhead** on top of a payload that's
  already too large for the telemetry shape, for no benefit — there's no
  reason to pay text-encoding tax on a payload nothing needs to
  eyeball as text.
- **It's a fundamentally different kind of data with different backend
  handling needs.** Telemetry readings feed the anomaly-detection
  baseline/threshold pipeline directly and cheaply (a few floats per
  sensor). IQ data needs FFT processing before it's useful for anything
  — an entirely different backend code path (closer to how the browser's
  own `powerSpectrumDb()` + `SpectrumAnomalyDetector` work today,
  server-side instead of client-side). Conflating the two request shapes
  would couple two things that should scale, rate-limit, and fail
  independently.

Given that, this module's stub (`rtlsdr_exp_forward_iq_block()`) targets
a **separate, currently-hypothetical** endpoint (`iq_upload_url` in
`rtlsdr_exp_config_t`, suggested path `/api/device/iq-stream` in code
comments) carrying a small fixed binary header (magic, sequence number,
center frequency, sample rate, payload length — see
`rtlsdr_iq_chunk_header_t`) followed by the raw IQ bytes, POSTed as
`application/octet-stream` with the same `Authorization: Bearer <device
token>` auth as the regular telemetry loop. **This endpoint does not
exist anywhere in this repo.** Building it is explicitly out of scope for
this workstream (it's backend work, not firmware, and the spec doesn't
assign a backend workstream to receive IQ data at all — only to receive
scalar telemetry). The stub is inert by default (`rtlsdr_exp_forward_
iq_block()` returns `ESP_OK` immediately unless `iq_upload_url` is
configured) specifically so this module can be compiled/enabled for its
USB-host/bring-up behavior without requiring a backend that doesn't
exist.

A real product decision here — genuinely open, not resolved by this
workstream — is whether a POST-per-chunk model is even right versus a
persistent WebSocket stream (the spec's `/ws/device-feed` design already
establishes a WS pub/sub pattern on the backend for telemetry; a
`/ws/device-iq` sibling might fit that architecture better than repeated
HTTP POSTs, especially if backpressure/flow-control matters, which for a
continuous stream it very much does). That's flagged here rather than
silently decided, because it depends on backend design judgment as much
as firmware judgment.

### What's unverified — an honest, specific list for real hardware bring-up

Nothing below has run against real hardware. In rough order of "most
likely to break first":

1. **USB Host Library API surface.** Function names, struct field names,
   and callback signatures (`usb_host_client_config_t`'s `.async.
   client_event_callback` shape in particular — ESP-IDF has changed this
   API's shape across versions) are written from documented/remembered
   API shape, not checked against a real ESP-IDF checkout (none available
   in this environment). **First thing a real bring-up needs: does this
   even compile against the ESP-IDF version Workstream I's project
   targets?**
2. **RTL2832U enumeration over a real ESP32-P4 USB-OTG host port** — does
   `usb_host_device_open()` / `usb_host_get_device_descriptor()` actually
   see the dongle at all on this specific chip's USB-OTG controller (power
   delivery to the dongle over USB-host mode is itself a hardware
   question — does the ESP32-P4 dev board supply VBUS in host mode, does
   the dongle draw more current than it can supply).
3. **The init vendor-command sequence itself** — carried over unchanged
   from `sdr.ts`, which is *itself* unverified. Two layers of "reasoned,
   not tested." A logic analyzer / USB protocol analyzer trace against a
   real dongle (or cross-checking against real librtlsdr `-T` verbose
   output) is needed to confirm register values, not just transfer
   plumbing.
4. **R820T PLL frequency math and demod resample-ratio math** — both are
   the same simplified integer-N approximation `sdr.ts` uses (its own
   comment: "real librtlsdr computes the exact sdm/vco from a 28.8MHz
   crystal reference. Simplified... Marked for hardware."). Likely wrong
   or imprecise until replaced with the real librtlsdr formula and
   checked against an actual received signal.
5. **Bulk transfer chunk size, pipeline depth, and timeout=0 choice**
   (`RTLSDR_EXP_BULK_CHUNK_BYTES`/`RTLSDR_EXP_BULK_QUEUE_DEPTH` Kconfig,
   defaults 16384B / depth 4) — these are guesses. Real ESP32-P4 USB
   Host Library heap/DMA limits, achievable sustained throughput at
   2.048 Msps (~4.1 MB/s), and whether `timeout_ms = 0` (no timeout, i.e.
   "block until data or disconnect") is even the right transfer mode for
   this endpoint all need real measurement.
6. **Drop-on-full backpressure policy** (`iq_blocks_dropped` stat) — is
   silently dropping IQ blocks when the forward task falls behind
   acceptable, or does it need real flow control (e.g. throttling the
   bulk read rate itself)? Depends on (5) and on real WiFi uplink
   bandwidth from a real device on a real seeker's home network.
7. **The proposed IQ-forwarding wire format and endpoint** — entirely
   hypothetical (see architecture section above); no backend exists to
   validate the framing against, and the POST-vs-WebSocket question above
   is unresolved.
8. **Concurrent USB Host Library ownership with the rest of the firmware
   project.** `rtlsdr_exp_start()`/`rtlsdr_exp_stop()` call `usb_host_
   install()`/`usb_host_uninstall()` directly, assuming this module is the
   *only* USB Host client in the project. If Workstream I's skeleton (or
   anything else) also needs USB host for something, this needs to change
   to a shared-ownership model (install once, both modules register as
   clients) — impossible to resolve without seeing that code, flagged
   here for whoever does the merge.
9. **Memory footprint.** Multiple in-flight 16KB bulk transfer buffers
   plus a forward queue plus `esp_http_client` buffers, alongside whatever
   Workstream I's WiFi/HTTP/BME280/presence stack already needs, on a
   single ESP32-P4's RAM — not sized or measured against a real linker
   map.

### Build / integration notes

This component is **off by default** (`RTLSDR_EXP_ENABLE` Kconfig option,
default `n`) so it cannot affect Workstream I's core build unless
explicitly turned on via `idf.py menuconfig` → "RTL-SDR Experimental
Module." To include it in a real project once Workstream I's skeleton
exists:

1. Add this directory's `components/` to the consuming project's
   `EXTRA_COMPONENT_DIRS` in the top-level `CMakeLists.txt` (or copy
   `components/rtlsdr_experimental/` into that project's own
   `components/`).
2. From wherever `app_main()` sets up its other sensor drivers, e.g.:
   ```c
   #include "rtlsdr_experimental.h"

   rtlsdr_exp_config_t sdr_cfg;
   rtlsdr_exp_config_default(&sdr_cfg);
   sdr_cfg.iq_upload_url = NULL; /* leave unset until a real backend
                                     endpoint exists — see README */
   sdr_cfg.device_bearer_token = DEVICE_BEARER_TOKEN; /* from
                                     main/device_config.h, same token
                                     used for the telemetry POST loop */
   rtlsdr_exp_handle_t sdr_handle;
   if (rtlsdr_exp_init(&sdr_cfg, &sdr_handle) == ESP_OK) {
       rtlsdr_exp_start(sdr_handle); /* non-blocking; watches for a dongle */
   }
   ```
3. `idf.py build` — **not run in this environment** (no ESP-IDF
   toolchain available); this module has only been reasoned about, not
   compiled. Treat "compiles" as an open question for the next person
   with a real toolchain, not a claim made here.

Verified in this environment: none of it, against hardware or a real
compiler. What *has* been done: careful structural translation of a
real, already-partially-researched protocol (`sdr.ts`) to the documented
shape of a real, chip-specific API (ESP32-P4's USB Host Library), with
every simplification and open question called out rather than hidden.