qtalker---/firmware/esp32p4-sensor-node/README.md

ESP32-P4 Sensor Node — firmware

Firmware for the physical ESP32-P4 "Ultimate Quantum Box" sensor node (see the ESP32-P4 Sensor Node design spec: docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md). Joins the seeker's home WiFi and streams sensor readings to POST /api/device/telemetry on their paired account, which feeds the séance's live anomaly-detection pipeline exactly like the browser-based Wire Ghost/EVP/Spirit Radio/EMF modes already do.

Honesty-policy note (binding on this whole directory)

This app's whole ethos is "real signal processing on real data, and it says so when something is unverified" (see the frontend's Spirit Radio HARDWARE PASS REQUIRED convention in frontend/src/lib/sdr.ts). Nobody working on this firmware has physical ESP32-P4 hardware to flash and test against, and this development environment has no ESP-IDF toolchain to even compile it. Every file under this directory is written to be structurally correct ESP-IDF C against the documented API shape, and reasoned about carefully — not verified against real hardware or even a real build. Anything that needs a real board to actually confirm is called out explicitly, in code comments and in this README, rather than glossed over. Treat "compiles cleanly / structurally sound" and "verified on-device" as two entirely different claims — this repo only makes the first one for anything under firmware/.

Status of this directory

This copy of the repo currently contains only Workstream J (the RTL-SDR experimental module, documented in full below). Workstream I — the core firmware project skeleton (CMakeLists.txt, sdkconfig.defaults, the main/ component with WiFi station-mode connect, the /api/device/telemetry HTTP client task, main/ device_config.h, and the BME280 + presence-sensor drivers) — was built in a separate, isolated worktree that this workstream cannot see or depend on. A real build of this project needs Workstream I's skeleton merged in alongside what's here. Nothing in this directory currently compiles standalone into a flashable image; the components/ rtlsdr_experimental/ module is a self-contained ESP-IDF component, deliberately structured so it can be dropped into Workstream I's project via EXTRA_COMPONENT_DIRS (or copied under that project's own components/) without editing any file that workstream owns. Once merged, this README should be extended with Workstream I's own build steps, wiring/pinout notes for the BME280 (I2C) and presence sensor (UART/GPIO), and its own verified-vs-not accounting — that content doesn't exist here yet because this workstream never had visibility into that code.

Directory layout (this workstream's contribution)

firmware/esp32p4-sensor-node/
├── README.md                              (this file)
└── components/
    └── rtlsdr_experimental/                Self-contained ESP-IDF component.
        ├── CMakeLists.txt
        ├── Kconfig                         Off by default (RTLSDR_EXP_ENABLE=n).
        ├── include/
        │   └── rtlsdr_experimental.h        Public API + full disclaimer header.
        └── rtlsdr_experimental.c            Implementation.

---

Workstream J — RTL-SDR experimental module (this workstream)

⚠️ This is the least certain part of the entire firmware build. Read this whole section before touching it.

What this is

A clearly-separated, opt-in, experimental module exploring USB-host communication with an RTL2832U-based SDR dongle over the ESP32-P4's native USB-OTG host controller (USB Host Library, usb_host.h — a real, documented capability of this specific chip, unlike most ESP32 variants). The idea, per the design spec: attach a cheap RTL-SDR dongle to the sensor node, treat "spirit radio scanning" as a hardware-backed mode instead of only a browser WebUSB feature.

Be honest about the real constraint (this is the spec's framing, and it's correct): wideband IQ sample rates and FFT processing are demanding relative to an MCU's compute, even one with ESP32-P4's AI accelerator. On-device spectrum analysis is not what this module attempts. The realistic architecture — and the one implemented here — is: pull raw IQ samples off the dongle via USB host, and forward them upstream for the backend to FFT/analyze (the same job the browser already does client-side today via frontend/src/lib/fft.ts + frontend/src/lib/sdr.ts's SpectrumAnomalyDetector). Even that "just pass the bytes through" architecture needs real USB throughput numbers to know if it's viable — see "What's unverified" below.

Primary reference

frontend/src/lib/sdr.ts — this repo's existing browser-based (WebUSB) RTL2832U + R820T driver. It's real, already-researched protocol detail against the public librtlsdr register documentation (vendor commands, I2C-repeater tuner access, the demod/tuner init sequence), itself marked HARDWARE PASS REQUIRED since it's never been run against a real dongle either. This firmware module is a direct port of that file's control- transfer sequence from WebUSB JS calls to ESP-IDF USB Host Library C calls — line-by-line correspondences are called out in code comments (e.g. rtlsdr_run_init_sequence() mirrors open() in sdr.ts almost register-for-register). It intentionally does not re-derive any register math from scratch; wherever sdr.ts says "simplified" or "HARDWARE PASS REQUIRED" (e.g. the R820T PLL frequency math, the demod resample-ratio math), this module carries the exact same simplification forward with the exact same caveat, rather than inventing new unverified math on top of already-unverified math.

What's implemented (structurally — see caveats below)

1. USB Host Library lifecycle (rtlsdr_exp_start() / rtlsdr_usb_lib_daemon_task() / rtlsdr_exp_client_task()): usb_host_install(), usb_host_client_register() with an async event callback, and the two-task pump pattern the USB Host Library's async model requires (one task for usb_host_lib_handle_events(), one for usb_host_client_handle_events() — the latter also being how this module's own control- and bulk-transfer completion callbacks get dispatched, since the Library calls them synchronously from whichever task is pumping events, not from a hidden thread or ISR).
2. RTL2832U/Terratec device enumeration and vendor-ID matching (rtlsdr_try_bring_up(), rtlsdr_vendor_id_matches()): on a USB_HOST_CLIENT_EVENT_NEW_DEV event, opens the device, reads its device descriptor, and matches idVendor against 0x0BDA (RTL2832U) / 0x0CCD (Terratec-rebadged) — ported directly from sdr.ts's requestDevice() filter list (RTL2832U_VENDOR, TERRATEC_VENDOR). Faithfully carries over that file's specific choice to filter by vendor ID only, not product ID (its comment: "many dongles report product ids outside the handful we know, so filtering by productId hides them from the picker") — the known product-id list (RTLSDR_EXP_KNOWN_PRODUCT_IDS) is kept as an informational log line only, never a hard filter, exactly mirroring how RTL2832U_PRODUCTS is exported-but-unused-as-a-filter in sdr.ts.
3. RTL2832U init vendor-command sequence over USB control transfers (rtlsdr_run_init_sequence(), rtlsdr_demod_write(), rtlsdr_reg_write(), rtlsdr_i2c_write(), rtlsdr_exp_set_frequency(), rtlsdr_exp_set_sample_rate()): the same demod soft-reset → demod_ctl/ suspend-off block → standby-off → AGC-mode → R820T tuner power-up (through the I2C repeater) → sample-rate program → initial tune → streaming-endpoint reset → test-mode-off sequence as sdr.ts's open(), with the same register addresses/values and the same wValue/wIndex encoding ((block<<8)|0x10 / (page<<8)|address), translated from USBDevice.controlTransferOut() to usb_host_transfer_submit_control() with a manually-built usb_setup_packet_t.
4. A basic bulk-transfer read loop structure for pulling IQ sample data off the device (rtlsdr_start_bulk_streaming(), rtlsdr_bulk_xfer_cb()): unlike sdr.ts (which does one-shot await dev.transferIn(...) calls from inside its own async sweep() loop), the USB Host Library is callback-driven, so continuous streaming here is a small pipeline — N transfers (bulk_read_queue_ depth, default 4) of a fixed chunk size (bulk_read_chunk_bytes, default 16384B, enforced as a multiple of 512 the same way sdr.ts's readSamples() comment requires) are kept perpetually in flight; each completion callback copies the received bytes into a heap block, hands it to a FreeRTOS queue for the forward task, and immediately resubmits itself to keep the pipe full. This is the part most likely to need real-hardware tuning — see below.
5. A stub/structure for forwarding raw IQ data upstream (rtlsdr_exp_forward_iq_block(), rtlsdr_forward_task()) — see the architecture decision writeup immediately below.

IQ-forwarding architecture decision

The spec explicitly leaves this as a product-judgment call ("your call on whether this warrants a separate endpoint/stream vs. reusing the sensor telemetry shape with a binary/base64 payload"). Decision made here: a separate binary stream/endpoint, not a reuse of the JSON telemetry shape. Reasoning:

• Raw throughput is the wrong order of magnitude for the telemetry contract. Even a modest 2.048 Msps capture at 8 bits/sample/channel (RTL2832U's native ADC format) is interleaved I/Q bytes at roughly 4.1 MB/s. The design spec's POST /api/device/telemetry contract caps the readings array at ~64 entries, caps the request body at ~16KB, and rate-limits ingestion to roughly 1 request/second sustained per device — sized for a handful of scalar sensor readings (temperature, humidity, presence booleans), not a continuous multi-megabyte/second binary stream. Forcing IQ data through that shape would mean either violating those caps (defeating their whole purpose — bounding what a misbehaving/malicious device can push at the backend) or chopping IQ into thousands of tiny requests per second, which is worse for both sides than one continuous stream.
• Base64-in-JSON adds ~33% size overhead on top of a payload that's already too large for the telemetry shape, for no benefit — there's no reason to pay text-encoding tax on a payload nothing needs to eyeball as text.
• It's a fundamentally different kind of data with different backend handling needs. Telemetry readings feed the anomaly-detection baseline/threshold pipeline directly and cheaply (a few floats per sensor). IQ data needs FFT processing before it's useful for anything — an entirely different backend code path (closer to how the browser's own powerSpectrumDb() + SpectrumAnomalyDetector work today, server-side instead of client-side). Conflating the two request shapes would couple two things that should scale, rate-limit, and fail independently.

Given that, this module's stub (rtlsdr_exp_forward_iq_block()) targets a separate, currently-hypothetical endpoint (iq_upload_url in rtlsdr_exp_config_t, suggested path /api/device/iq-stream in code comments) carrying a small fixed binary header (magic, sequence number, center frequency, sample rate, payload length — see rtlsdr_iq_chunk_header_t) followed by the raw IQ bytes, POSTed as application/octet-stream with the same Authorization: Bearer <device token> auth as the regular telemetry loop. This endpoint does not exist anywhere in this repo. Building it is explicitly out of scope for this workstream (it's backend work, not firmware, and the spec doesn't assign a backend workstream to receive IQ data at all — only to receive scalar telemetry). The stub is inert by default (rtlsdr_exp_forward_ iq_block() returns ESP_OK immediately unless iq_upload_url is configured) specifically so this module can be compiled/enabled for its USB-host/bring-up behavior without requiring a backend that doesn't exist.

A real product decision here — genuinely open, not resolved by this workstream — is whether a POST-per-chunk model is even right versus a persistent WebSocket stream (the spec's /ws/device-feed design already establishes a WS pub/sub pattern on the backend for telemetry; a /ws/device-iq sibling might fit that architecture better than repeated HTTP POSTs, especially if backpressure/flow-control matters, which for a continuous stream it very much does). That's flagged here rather than silently decided, because it depends on backend design judgment as much as firmware judgment.

What's unverified — an honest, specific list for real hardware bring-up

Nothing below has run against real hardware. In rough order of "most likely to break first":

1. USB Host Library API surface. Function names, struct field names, and callback signatures (usb_host_client_config_t's .async. client_event_callback shape in particular — ESP-IDF has changed this API's shape across versions) are written from documented/remembered API shape, not checked against a real ESP-IDF checkout (none available in this environment). First thing a real bring-up needs: does this even compile against the ESP-IDF version Workstream I's project targets?
2. RTL2832U enumeration over a real ESP32-P4 USB-OTG host port — does usb_host_device_open() / usb_host_get_device_descriptor() actually see the dongle at all on this specific chip's USB-OTG controller (power delivery to the dongle over USB-host mode is itself a hardware question — does the ESP32-P4 dev board supply VBUS in host mode, does the dongle draw more current than it can supply).
3. The init vendor-command sequence itself — carried over unchanged from sdr.ts, which is itself unverified. Two layers of "reasoned, not tested." A logic analyzer / USB protocol analyzer trace against a real dongle (or cross-checking against real librtlsdr -T verbose output) is needed to confirm register values, not just transfer plumbing.
4. R820T PLL frequency math and demod resample-ratio math — both are the same simplified integer-N approximation sdr.ts uses (its own comment: "real librtlsdr computes the exact sdm/vco from a 28.8MHz crystal reference. Simplified... Marked for hardware."). Likely wrong or imprecise until replaced with the real librtlsdr formula and checked against an actual received signal.
5. Bulk transfer chunk size, pipeline depth, and timeout=0 choice (RTLSDR_EXP_BULK_CHUNK_BYTES/RTLSDR_EXP_BULK_QUEUE_DEPTH Kconfig, defaults 16384B / depth 4) — these are guesses. Real ESP32-P4 USB Host Library heap/DMA limits, achievable sustained throughput at 2.048 Msps (~4.1 MB/s), and whether timeout_ms = 0 (no timeout, i.e. "block until data or disconnect") is even the right transfer mode for this endpoint all need real measurement.
6. Drop-on-full backpressure policy (iq_blocks_dropped stat) — is silently dropping IQ blocks when the forward task falls behind acceptable, or does it need real flow control (e.g. throttling the bulk read rate itself)? Depends on (5) and on real WiFi uplink bandwidth from a real device on a real seeker's home network.
7. The proposed IQ-forwarding wire format and endpoint — entirely hypothetical (see architecture section above); no backend exists to validate the framing against, and the POST-vs-WebSocket question above is unresolved.
8. Concurrent USB Host Library ownership with the rest of the firmware project. rtlsdr_exp_start()/rtlsdr_exp_stop() call usb_host_ install()/usb_host_uninstall() directly, assuming this module is the only USB Host client in the project. If Workstream I's skeleton (or anything else) also needs USB host for something, this needs to change to a shared-ownership model (install once, both modules register as clients) — impossible to resolve without seeing that code, flagged here for whoever does the merge.
9. Memory footprint. Multiple in-flight 16KB bulk transfer buffers plus a forward queue plus esp_http_client buffers, alongside whatever Workstream I's WiFi/HTTP/BME280/presence stack already needs, on a single ESP32-P4's RAM — not sized or measured against a real linker map.

Build / integration notes

This component is off by default (RTLSDR_EXP_ENABLE Kconfig option, default n) so it cannot affect Workstream I's core build unless explicitly turned on via idf.py menuconfig → "RTL-SDR Experimental Module." To include it in a real project once Workstream I's skeleton exists:

1. Add this directory's components/ to the consuming project's EXTRA_COMPONENT_DIRS in the top-level CMakeLists.txt (or copy components/rtlsdr_experimental/ into that project's own components/).
2. From wherever app_main() sets up its other sensor drivers, e.g.:

   #include "rtlsdr_experimental.h"
   
   rtlsdr_exp_config_t sdr_cfg;
   rtlsdr_exp_config_default(&sdr_cfg);
   sdr_cfg.iq_upload_url = NULL; /* leave unset until a real backend
                                     endpoint exists — see README */
   sdr_cfg.device_bearer_token = DEVICE_BEARER_TOKEN; /* from
                                     main/device_config.h, same token
                                     used for the telemetry POST loop */
   rtlsdr_exp_handle_t sdr_handle;
   if (rtlsdr_exp_init(&sdr_cfg, &sdr_handle) == ESP_OK) {
       rtlsdr_exp_start(sdr_handle); /* non-blocking; watches for a dongle */
   }
   

3. idf.py build — not run in this environment (no ESP-IDF toolchain available); this module has only been reasoned about, not compiled. Treat "compiles" as an open question for the next person with a real toolchain, not a claim made here.

Verified in this environment: none of it, against hardware or a real compiler. What has been done: careful structural translation of a real, already-partially-researched protocol (sdr.ts) to the documented shape of a real, chip-specific API (ESP32-P4's USB Host Library), with every simplification and open question called out rather than hidden.