diff --git a/firmware/esp32p4-sensor-node/README.md b/firmware/esp32p4-sensor-node/README.md new file mode 100644 index 0000000..b23a540 --- /dev/null +++ b/firmware/esp32p4-sensor-node/README.md @@ -0,0 +1,324 @@ +# ESP32-P4 Sensor Node — firmware + +Firmware for the physical ESP32-P4 "Ultimate Quantum Box" sensor node +(see the ESP32-P4 Sensor Node design spec: +`docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md`). Joins +the seeker's home WiFi and streams sensor readings to `POST +/api/device/telemetry` on their paired account, which feeds the séance's +live anomaly-detection pipeline exactly like the browser-based Wire +Ghost/EVP/Spirit Radio/EMF modes already do. + +## Honesty-policy note (binding on this whole directory) + +This app's whole ethos is "real signal processing on real data, and it +says so when something is unverified" (see the frontend's Spirit Radio +`HARDWARE PASS REQUIRED` convention in `frontend/src/lib/sdr.ts`). Nobody +working on this firmware has physical ESP32-P4 hardware to flash and test +against, and this development environment has no ESP-IDF toolchain to +even compile it. **Every file under this directory is written to be +structurally correct ESP-IDF C against the documented API shape, and +reasoned about carefully — not verified against real hardware or even a +real build.** Anything that needs a real board to actually confirm is +called out explicitly, in code comments and in this README, rather than +glossed over. Treat "compiles cleanly / structurally sound" and "verified +on-device" as two entirely different claims — this repo only makes the +first one for anything under `firmware/`. + +## Status of this directory + +This copy of the repo currently contains **only Workstream J** (the +RTL-SDR experimental module, documented in full below). Workstream I +— the core firmware project skeleton (`CMakeLists.txt`, +`sdkconfig.defaults`, the `main/` component with WiFi station-mode +connect, the `/api/device/telemetry` HTTP client task, `main/ +device_config.h`, and the BME280 + presence-sensor drivers) — was built +in a separate, isolated worktree that this workstream cannot see or +depend on. **A real build of this project needs Workstream I's skeleton +merged in alongside what's here.** Nothing in this directory currently +compiles standalone into a flashable image; the `components/ +rtlsdr_experimental/` module is a self-contained ESP-IDF *component*, +deliberately structured so it can be dropped into Workstream I's project +via `EXTRA_COMPONENT_DIRS` (or copied under that project's own +`components/`) without editing any file that workstream owns. Once +merged, this README should be extended with Workstream I's own build +steps, wiring/pinout notes for the BME280 (I2C) and presence sensor +(UART/GPIO), and its own verified-vs-not accounting — that content +doesn't exist here yet because this workstream never had visibility into +that code. + +## Directory layout (this workstream's contribution) + +``` +firmware/esp32p4-sensor-node/ +├── README.md (this file) +└── components/ + └── rtlsdr_experimental/ Self-contained ESP-IDF component. + ├── CMakeLists.txt + ├── Kconfig Off by default (RTLSDR_EXP_ENABLE=n). + ├── include/ + │ └── rtlsdr_experimental.h Public API + full disclaimer header. + └── rtlsdr_experimental.c Implementation. +``` + +--- + +## Workstream J — RTL-SDR experimental module (this workstream) + +**⚠️ This is the least certain part of the entire firmware build. Read +this whole section before touching it.** + +### What this is + +A clearly-separated, opt-in, experimental module exploring USB-host +communication with an RTL2832U-based SDR dongle over the ESP32-P4's +native USB-OTG host controller (USB Host Library, `usb_host.h` — a real, +documented capability of this specific chip, unlike most ESP32 variants). +The idea, per the design spec: attach a cheap RTL-SDR dongle to the +sensor node, treat "spirit radio scanning" as a hardware-backed mode +instead of only a browser WebUSB feature. + +**Be honest about the real constraint (this is the spec's framing, and +it's correct):** wideband IQ sample rates and FFT processing are +demanding relative to an MCU's compute, even one with ESP32-P4's AI +accelerator. **On-device spectrum analysis is not what this module +attempts.** The realistic architecture — and the one implemented here — +is: pull raw IQ samples off the dongle via USB host, and forward them +upstream for the *backend* to FFT/analyze (the same job the browser +already does client-side today via `frontend/src/lib/fft.ts` + +`frontend/src/lib/sdr.ts`'s `SpectrumAnomalyDetector`). Even that +"just pass the bytes through" architecture needs real USB throughput +numbers to know if it's viable — see "What's unverified" below. + +### Primary reference + +`frontend/src/lib/sdr.ts` — this repo's existing browser-based (WebUSB) +RTL2832U + R820T driver. It's real, already-researched protocol detail +against the public librtlsdr register documentation (vendor commands, +I2C-repeater tuner access, the demod/tuner init sequence), itself marked +`HARDWARE PASS REQUIRED` since it's never been run against a real dongle +either. This firmware module is a direct **port** of that file's control- +transfer sequence from WebUSB JS calls to ESP-IDF USB Host Library C +calls — line-by-line correspondences are called out in code comments +(e.g. `rtlsdr_run_init_sequence()` mirrors `open()` in sdr.ts almost +register-for-register). It intentionally does not re-derive any register +math from scratch; wherever sdr.ts says "simplified" or "HARDWARE PASS +REQUIRED" (e.g. the R820T PLL frequency math, the demod resample-ratio +math), this module carries the exact same simplification forward with the +exact same caveat, rather than inventing new unverified math on top of +already-unverified math. + +### What's implemented (structurally — see caveats below) + +1. **USB Host Library lifecycle** (`rtlsdr_exp_start()` / + `rtlsdr_usb_lib_daemon_task()` / `rtlsdr_exp_client_task()`): + `usb_host_install()`, `usb_host_client_register()` with an async event + callback, and the two-task pump pattern the USB Host Library's async + model requires (one task for `usb_host_lib_handle_events()`, one for + `usb_host_client_handle_events()` — the latter also being how this + module's own control- and bulk-transfer completion callbacks get + dispatched, since the Library calls them synchronously from whichever + task is pumping events, not from a hidden thread or ISR). +2. **RTL2832U/Terratec device enumeration and vendor-ID matching** + (`rtlsdr_try_bring_up()`, `rtlsdr_vendor_id_matches()`): on a + `USB_HOST_CLIENT_EVENT_NEW_DEV` event, opens the device, reads its + device descriptor, and matches `idVendor` against `0x0BDA` + (RTL2832U) / `0x0CCD` (Terratec-rebadged) — ported directly from + `sdr.ts`'s `requestDevice()` filter list (`RTL2832U_VENDOR`, + `TERRATEC_VENDOR`). Faithfully carries over that file's specific + choice to filter by **vendor ID only**, not product ID (its comment: + "many dongles report product ids outside the handful we know, so + filtering by productId hides them from the picker") — the known + product-id list (`RTLSDR_EXP_KNOWN_PRODUCT_IDS`) is kept as an + informational log line only, never a hard filter, exactly mirroring + how `RTL2832U_PRODUCTS` is exported-but-unused-as-a-filter in + `sdr.ts`. +3. **RTL2832U init vendor-command sequence** over USB control transfers + (`rtlsdr_run_init_sequence()`, `rtlsdr_demod_write()`, + `rtlsdr_reg_write()`, `rtlsdr_i2c_write()`, `rtlsdr_exp_set_frequency()`, + `rtlsdr_exp_set_sample_rate()`): the same demod soft-reset → demod_ctl/ + suspend-off block → standby-off → AGC-mode → R820T tuner power-up + (through the I2C repeater) → sample-rate program → initial tune → + streaming-endpoint reset → test-mode-off sequence as `sdr.ts`'s + `open()`, with the same register addresses/values and the same + `wValue`/`wIndex` encoding (`(block<<8)|0x10` / `(page<<8)|address`), + translated from `USBDevice.controlTransferOut()` to + `usb_host_transfer_submit_control()` with a manually-built + `usb_setup_packet_t`. +4. **A basic bulk-transfer read loop structure** for pulling IQ sample + data off the device (`rtlsdr_start_bulk_streaming()`, + `rtlsdr_bulk_xfer_cb()`): unlike `sdr.ts` (which does one-shot + `await dev.transferIn(...)` calls from inside its own async `sweep()` + loop), the USB Host Library is callback-driven, so continuous + streaming here is a small pipeline — N transfers (`bulk_read_queue_ + depth`, default 4) of a fixed chunk size (`bulk_read_chunk_bytes`, + default 16384B, enforced as a multiple of 512 the same way `sdr.ts`'s + `readSamples()` comment requires) are kept perpetually in flight; each + completion callback copies the received bytes into a heap block, hands + it to a FreeRTOS queue for the forward task, and immediately resubmits + itself to keep the pipe full. **This is the part most likely to need + real-hardware tuning** — see below. +5. **A stub/structure for forwarding raw IQ data upstream** + (`rtlsdr_exp_forward_iq_block()`, `rtlsdr_forward_task()`) — see the + architecture decision writeup immediately below. + +### IQ-forwarding architecture decision + +The spec explicitly leaves this as a product-judgment call ("your call on +whether this warrants a separate endpoint/stream vs. reusing the sensor +telemetry shape with a binary/base64 payload"). Decision made here: +**a separate binary stream/endpoint, not a reuse of the JSON telemetry +shape.** Reasoning: + +- **Raw throughput is the wrong order of magnitude for the telemetry + contract.** Even a modest 2.048 Msps capture at 8 bits/sample/channel + (RTL2832U's native ADC format) is interleaved I/Q bytes at roughly + 4.1 MB/s. The design spec's `POST /api/device/telemetry` contract caps + the `readings` array at ~64 entries, caps the request body at ~16KB, + and rate-limits ingestion to roughly 1 request/second sustained per + device — sized for a handful of scalar sensor readings (temperature, + humidity, presence booleans), not a continuous multi-megabyte/second + binary stream. Forcing IQ data through that shape would mean either + violating those caps (defeating their whole purpose — bounding what a + misbehaving/malicious device can push at the backend) or chopping IQ + into thousands of tiny requests per second, which is worse for both + sides than one continuous stream. +- **Base64-in-JSON adds ~33% size overhead** on top of a payload that's + already too large for the telemetry shape, for no benefit — there's no + reason to pay text-encoding tax on a payload nothing needs to + eyeball as text. +- **It's a fundamentally different kind of data with different backend + handling needs.** Telemetry readings feed the anomaly-detection + baseline/threshold pipeline directly and cheaply (a few floats per + sensor). IQ data needs FFT processing before it's useful for anything + — an entirely different backend code path (closer to how the browser's + own `powerSpectrumDb()` + `SpectrumAnomalyDetector` work today, + server-side instead of client-side). Conflating the two request shapes + would couple two things that should scale, rate-limit, and fail + independently. + +Given that, this module's stub (`rtlsdr_exp_forward_iq_block()`) targets +a **separate, currently-hypothetical** endpoint (`iq_upload_url` in +`rtlsdr_exp_config_t`, suggested path `/api/device/iq-stream` in code +comments) carrying a small fixed binary header (magic, sequence number, +center frequency, sample rate, payload length — see +`rtlsdr_iq_chunk_header_t`) followed by the raw IQ bytes, POSTed as +`application/octet-stream` with the same `Authorization: Bearer ` auth as the regular telemetry loop. **This endpoint does not +exist anywhere in this repo.** Building it is explicitly out of scope for +this workstream (it's backend work, not firmware, and the spec doesn't +assign a backend workstream to receive IQ data at all — only to receive +scalar telemetry). The stub is inert by default (`rtlsdr_exp_forward_ +iq_block()` returns `ESP_OK` immediately unless `iq_upload_url` is +configured) specifically so this module can be compiled/enabled for its +USB-host/bring-up behavior without requiring a backend that doesn't +exist. + +A real product decision here — genuinely open, not resolved by this +workstream — is whether a POST-per-chunk model is even right versus a +persistent WebSocket stream (the spec's `/ws/device-feed` design already +establishes a WS pub/sub pattern on the backend for telemetry; a +`/ws/device-iq` sibling might fit that architecture better than repeated +HTTP POSTs, especially if backpressure/flow-control matters, which for a +continuous stream it very much does). That's flagged here rather than +silently decided, because it depends on backend design judgment as much +as firmware judgment. + +### What's unverified — an honest, specific list for real hardware bring-up + +Nothing below has run against real hardware. In rough order of "most +likely to break first": + +1. **USB Host Library API surface.** Function names, struct field names, + and callback signatures (`usb_host_client_config_t`'s `.async. + client_event_callback` shape in particular — ESP-IDF has changed this + API's shape across versions) are written from documented/remembered + API shape, not checked against a real ESP-IDF checkout (none available + in this environment). **First thing a real bring-up needs: does this + even compile against the ESP-IDF version Workstream I's project + targets?** +2. **RTL2832U enumeration over a real ESP32-P4 USB-OTG host port** — does + `usb_host_device_open()` / `usb_host_get_device_descriptor()` actually + see the dongle at all on this specific chip's USB-OTG controller (power + delivery to the dongle over USB-host mode is itself a hardware + question — does the ESP32-P4 dev board supply VBUS in host mode, does + the dongle draw more current than it can supply). +3. **The init vendor-command sequence itself** — carried over unchanged + from `sdr.ts`, which is *itself* unverified. Two layers of "reasoned, + not tested." A logic analyzer / USB protocol analyzer trace against a + real dongle (or cross-checking against real librtlsdr `-T` verbose + output) is needed to confirm register values, not just transfer + plumbing. +4. **R820T PLL frequency math and demod resample-ratio math** — both are + the same simplified integer-N approximation `sdr.ts` uses (its own + comment: "real librtlsdr computes the exact sdm/vco from a 28.8MHz + crystal reference. Simplified... Marked for hardware."). Likely wrong + or imprecise until replaced with the real librtlsdr formula and + checked against an actual received signal. +5. **Bulk transfer chunk size, pipeline depth, and timeout=0 choice** + (`RTLSDR_EXP_BULK_CHUNK_BYTES`/`RTLSDR_EXP_BULK_QUEUE_DEPTH` Kconfig, + defaults 16384B / depth 4) — these are guesses. Real ESP32-P4 USB + Host Library heap/DMA limits, achievable sustained throughput at + 2.048 Msps (~4.1 MB/s), and whether `timeout_ms = 0` (no timeout, i.e. + "block until data or disconnect") is even the right transfer mode for + this endpoint all need real measurement. +6. **Drop-on-full backpressure policy** (`iq_blocks_dropped` stat) — is + silently dropping IQ blocks when the forward task falls behind + acceptable, or does it need real flow control (e.g. throttling the + bulk read rate itself)? Depends on (5) and on real WiFi uplink + bandwidth from a real device on a real seeker's home network. +7. **The proposed IQ-forwarding wire format and endpoint** — entirely + hypothetical (see architecture section above); no backend exists to + validate the framing against, and the POST-vs-WebSocket question above + is unresolved. +8. **Concurrent USB Host Library ownership with the rest of the firmware + project.** `rtlsdr_exp_start()`/`rtlsdr_exp_stop()` call `usb_host_ + install()`/`usb_host_uninstall()` directly, assuming this module is the + *only* USB Host client in the project. If Workstream I's skeleton (or + anything else) also needs USB host for something, this needs to change + to a shared-ownership model (install once, both modules register as + clients) — impossible to resolve without seeing that code, flagged + here for whoever does the merge. +9. **Memory footprint.** Multiple in-flight 16KB bulk transfer buffers + plus a forward queue plus `esp_http_client` buffers, alongside whatever + Workstream I's WiFi/HTTP/BME280/presence stack already needs, on a + single ESP32-P4's RAM — not sized or measured against a real linker + map. + +### Build / integration notes + +This component is **off by default** (`RTLSDR_EXP_ENABLE` Kconfig option, +default `n`) so it cannot affect Workstream I's core build unless +explicitly turned on via `idf.py menuconfig` → "RTL-SDR Experimental +Module." To include it in a real project once Workstream I's skeleton +exists: + +1. Add this directory's `components/` to the consuming project's + `EXTRA_COMPONENT_DIRS` in the top-level `CMakeLists.txt` (or copy + `components/rtlsdr_experimental/` into that project's own + `components/`). +2. From wherever `app_main()` sets up its other sensor drivers, e.g.: + ```c + #include "rtlsdr_experimental.h" + + rtlsdr_exp_config_t sdr_cfg; + rtlsdr_exp_config_default(&sdr_cfg); + sdr_cfg.iq_upload_url = NULL; /* leave unset until a real backend + endpoint exists — see README */ + sdr_cfg.device_bearer_token = DEVICE_BEARER_TOKEN; /* from + main/device_config.h, same token + used for the telemetry POST loop */ + rtlsdr_exp_handle_t sdr_handle; + if (rtlsdr_exp_init(&sdr_cfg, &sdr_handle) == ESP_OK) { + rtlsdr_exp_start(sdr_handle); /* non-blocking; watches for a dongle */ + } + ``` +3. `idf.py build` — **not run in this environment** (no ESP-IDF + toolchain available); this module has only been reasoned about, not + compiled. Treat "compiles" as an open question for the next person + with a real toolchain, not a claim made here. + +Verified in this environment: none of it, against hardware or a real +compiler. What *has* been done: careful structural translation of a +real, already-partially-researched protocol (`sdr.ts`) to the documented +shape of a real, chip-specific API (ESP32-P4's USB Host Library), with +every simplification and open question called out rather than hidden. diff --git a/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/CMakeLists.txt b/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/CMakeLists.txt new file mode 100644 index 0000000..0dfa2cb --- /dev/null +++ b/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/CMakeLists.txt @@ -0,0 +1,17 @@ +# rtlsdr_experimental — EXPERIMENTAL, UNVERIFIED AGAINST REAL HARDWARE. +# See include/rtlsdr_experimental.h and ../../README.md ("Workstream J") +# for the full honesty write-up. +# +# This is a self-contained ESP-IDF component so it can be dropped into +# Workstream I's core firmware project (firmware/esp32p4-sensor-node/) via +# EXTRA_COMPONENT_DIRS, or by copying this directory under that project's +# own components/, without editing any file that workstream owns. +# +# `usb` = ESP-IDF's USB Host Library (usb_host.h) — native to ESP32-P4's +# USB-OTG host controller. `esp_http_client` = the upstream IQ-forwarding +# stub's transport (see rtlsdr_experimental.c "Upstream IQ forwarding"). +idf_component_register( + SRCS "rtlsdr_experimental.c" + INCLUDE_DIRS "include" + REQUIRES usb esp_http_client +) diff --git a/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/Kconfig b/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/Kconfig new file mode 100644 index 0000000..e3fd9c7 --- /dev/null +++ b/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/Kconfig @@ -0,0 +1,35 @@ +menu "RTL-SDR Experimental Module (Workstream J — UNVERIFIED)" + + config RTLSDR_EXP_ENABLE + bool "Enable RTL-SDR experimental USB-host module" + default n + help + Attempts to initialize the ESP32-P4 USB Host Library and probe + for an attached RTL2832U-based dongle. This module is + experimental and UNVERIFIED against real hardware — see + firmware/esp32p4-sensor-node/README.md, "Workstream J" section, + before enabling on a device you care about. Off by default so + it can never interfere with the core sensor node build unless + explicitly turned on. + + config RTLSDR_EXP_BULK_CHUNK_BYTES + int "Bulk IQ read chunk size (bytes, must be a multiple of 512)" + default 16384 + depends on RTLSDR_EXP_ENABLE + help + UNVERIFIED default. Real achievable USB bulk throughput on + ESP32-P4 (and the right tradeoff against available heap/DMA + memory) is one of the concrete things a real hardware bring-up + session needs to measure and tune. + + config RTLSDR_EXP_BULK_QUEUE_DEPTH + int "In-flight bulk transfer pipeline depth" + default 4 + range 1 8 + depends on RTLSDR_EXP_ENABLE + help + Number of bulk-IN transfers kept perpetually submitted so the + USB pipe doesn't idle. UNVERIFIED — needs tuning against real + measured throughput vs. available RAM. + +endmenu diff --git a/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/include/rtlsdr_experimental.h b/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/include/rtlsdr_experimental.h new file mode 100644 index 0000000..1265674 --- /dev/null +++ b/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/include/rtlsdr_experimental.h @@ -0,0 +1,151 @@ +/* + * rtlsdr_experimental.h — EXPERIMENTAL RTL2832U-over-USB-Host module for the + * ESP32-P4 sensor node (Workstream J, ESP32-P4 Sensor Node spec). + * + * ============================================================================ + * HARDWARE PASS REQUIRED — UNVERIFIED AGAINST REAL HARDWARE + * ============================================================================ + * This entire module (this header + rtlsdr_experimental.c) has been written + * from documented ESP-IDF USB Host Library API shape and the RTL2832U/R820T + * register sequence already researched for this project in + * frontend/src/lib/sdr.ts (a WebUSB driver, real and documented against + * librtlsdr's public register map). It has NOT been compiled against a real + * ESP-IDF toolchain (none is available in this environment) and has NOT run + * against real ESP32-P4 + RTL2832U dongle hardware. Treat every register + * value, timing constant, buffer size, and API call signature here as + * "structurally reasoned, not verified." See the "Workstream J" section of + * ../../README.md for the full honesty write-up, including exactly what a + * real hardware bring-up session needs to check before this is trusted. + * + * Scope reminder: this is a self-contained ESP-IDF *component*, deliberately + * kept out of `main/` so it can be dropped into Workstream I's firmware + * project skeleton (WiFi/HTTP/BME280/presence-sensor core) without touching + * any file that workstream owns. Integration is one line in the consuming + * project: add this directory to EXTRA_COMPONENT_DIRS (or copy it under that + * project's own components/) and call rtlsdr_exp_init()/rtlsdr_exp_start() + * from wherever app_main() sets up its other sensor drivers. + * ============================================================================ + */ +#pragma once + +#include +#include +#include + +#include "esp_err.h" + +#ifdef __cplusplus +extern "C" { +#endif + +/* --------------------------------------------------------------------------- + * USB identification — ported 1:1 from frontend/src/lib/sdr.ts + * (RTL2832U_VENDOR / RTL2832U_PRODUCTS / TERRATEC_VENDOR, lines ~30-34). + * + * sdr.ts deliberately filters WebUSB's device picker by *vendor* id only + * (see its requestDevice() comment: "many dongles report product ids + * outside the handful we know, so filtering by productId hides them from + * the picker"). We mirror that: RTLSDR_EXP_KNOWN_PRODUCT_IDS below is + * informational only (used for a startup log line), never a hard match + * filter. Any device presenting vendor id RTL2832U or TERRATEC is treated + * as a candidate and bring-up is attempted. + * ------------------------------------------------------------------------ */ +#define RTLSDR_EXP_VENDOR_RTL2832U 0x0BDAu +#define RTLSDR_EXP_VENDOR_TERRATEC 0x0CCDu + +/* Known RTL2832U product ids (informational/logging only — see above). */ +extern const uint16_t RTLSDR_EXP_KNOWN_PRODUCT_IDS[4]; +#define RTLSDR_EXP_NUM_KNOWN_PRODUCT_IDS 4 + +/* --------------------------------------------------------------------------- + * Configuration + * ------------------------------------------------------------------------ */ +typedef struct { + /* Tuning defaults, applied once bring-up completes. Matches sdr.ts's + * open(sampleRateHz = 2_048_000) default and its FM-band test tune of + * 98 MHz — pick whatever the product actually wants to listen to. */ + uint32_t center_freq_hz; /* e.g. 98_000_000 */ + uint32_t sample_rate_hz; /* e.g. 2_048_000 */ + + /* Bulk-IN read tuning. bulk_read_chunk_bytes MUST be a multiple of 512 + * (USB high-speed bulk max packet size) — this mirrors sdr.ts's own + * assumption ("Length must be multiple of 512" on readSamples()). + * UNVERIFIED: real chunk size vs. USB Host Library heap/DMA limits and + * actual achievable throughput needs a real device. See README. */ + size_t bulk_read_chunk_bytes; /* default suggestion: 16384 */ + size_t bulk_read_queue_depth; /* in-flight pipelined transfers, e.g. 4 */ + + /* Upstream IQ forwarding (see rtlsdr_exp_forward_iq_block() and the + * "IQ forwarding architecture" section of the README for the reasoning + * behind a *separate* raw-binary stream rather than reusing the JSON + * sensor telemetry endpoint). NOTE: the backend endpoint referenced + * here does not exist yet anywhere in this repo as of this workstream — + * this is a client-side stub against a *proposed* shape, not a wired + * integration. */ + const char *iq_upload_url; /* e.g. "https://host/api/device/iq-stream" */ + const char *device_bearer_token; /* same raw pairing token used for + * the regular telemetry POST loop */ +} rtlsdr_exp_config_t; + +/* Fills in documented, conservative defaults. Caller still MUST set + * iq_upload_url / device_bearer_token before starting. */ +void rtlsdr_exp_config_default(rtlsdr_exp_config_t *config); + +typedef struct rtlsdr_exp_handle_s *rtlsdr_exp_handle_t; + +/* --------------------------------------------------------------------------- + * Lifecycle + * ------------------------------------------------------------------------ */ + +/* Allocates internal state. Does NOT touch USB hardware yet. */ +esp_err_t rtlsdr_exp_init(const rtlsdr_exp_config_t *config, rtlsdr_exp_handle_t *out_handle); +esp_err_t rtlsdr_exp_deinit(rtlsdr_exp_handle_t handle); + +/* Installs the USB Host Library, registers a client, and spawns the + * background tasks that watch for a matching RTL2832U device, run the + * init vendor-command sequence on attach, and (once bring-up succeeds) + * start the bulk-IN read/forward loop. Non-blocking — returns once the + * tasks are created, not once a device is found. + * + * UNVERIFIED beyond "this is the documented shape of the ESP-IDF USB Host + * Library API" — see README for exactly what needs a real board to check. */ +esp_err_t rtlsdr_exp_start(rtlsdr_exp_handle_t handle); + +/* Stops streaming, releases the USB interface/device if held, deregisters + * the USB Host client, and tears down the background tasks. Does NOT call + * usb_host_uninstall() if other USB Host clients might still be active in + * the wider firmware project (Workstream I owns whether anything else in + * the project also uses USB host) — see README integration notes. */ +esp_err_t rtlsdr_exp_stop(rtlsdr_exp_handle_t handle); + +/* Best-effort retune while streaming — same caveats as sdr.ts's + * setFrequency()/setSampleRate() (simplified PLL/resampler math, marked + * HARDWARE PASS REQUIRED there too). */ +esp_err_t rtlsdr_exp_set_frequency(rtlsdr_exp_handle_t handle, uint32_t hz); +esp_err_t rtlsdr_exp_set_sample_rate(rtlsdr_exp_handle_t handle, uint32_t hz); + +/* --------------------------------------------------------------------------- + * Observability — deliberately exposed so this experimental module can + * report its own health (e.g. folded into the regular sensor_type registry + * as a synthetic "sdr_status" reading) rather than fail silently. + * ------------------------------------------------------------------------ */ +typedef struct { + bool device_present; /* a matching-vendor USB device is enumerated */ + bool bring_up_ok; /* init vendor-command sequence completed without + * a control-transfer error (does NOT mean the + * tuner/demod are actually producing valid IQ — + * only real hardware + a spectrum check can + * confirm that) */ + bool streaming; + uint32_t bulk_reads_ok; + uint32_t bulk_reads_failed; + uint32_t iq_blocks_dropped; /* forward queue was full — see README */ + uint32_t bytes_forwarded_upstream; + uint32_t forward_failures; +} rtlsdr_exp_stats_t; + +void rtlsdr_exp_get_stats(rtlsdr_exp_handle_t handle, rtlsdr_exp_stats_t *out_stats); + +#ifdef __cplusplus +} +#endif diff --git a/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/rtlsdr_experimental.c b/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/rtlsdr_experimental.c new file mode 100644 index 0000000..c65ccdb --- /dev/null +++ b/firmware/esp32p4-sensor-node/components/rtlsdr_experimental/rtlsdr_experimental.c @@ -0,0 +1,913 @@ +/* + * rtlsdr_experimental.c — EXPERIMENTAL RTL2832U-over-USB-Host module. + * + * ============================================================================ + * HARDWARE PASS REQUIRED — UNVERIFIED AGAINST REAL HARDWARE + * ============================================================================ + * See rtlsdr_experimental.h for the full disclaimer and ../../README.md's + * "Workstream J" section for the complete honesty write-up. Short version: + * nobody working on this had physical ESP32-P4 or RTL2832U hardware, and + * this environment has no ESP-IDF toolchain to even compile against. Every + * function below is written to the *documented shape* of: + * (a) the ESP-IDF USB Host Library API (usb_host.h) — install/client/ + * transfer lifecycle, callback dispatch model, control vs. bulk + * transfer submission, config/interface descriptor parsing helpers; + * (b) the RTL2832U + R820T vendor-command sequence already researched + * and documented in frontend/src/lib/sdr.ts (WebUSB driver for this + * same chipset, against public librtlsdr register docs). + * It has not been built or run. Register pokes, timing, and buffer sizing + * are carried over from sdr.ts's own "HARDWARE PASS REQUIRED" markings + * with no changes beyond translating WebUSB JS calls to USB Host C calls. + * ============================================================================ + * + * Architecture (see README for the fuller picture): + * + * usb_host_lib_daemon_task — pumps usb_host_lib_handle_events() forever. + * Required by the USB Host Library regardless + * of how many clients exist. + * rtlsdr_exp_client_task — registers as the one USB Host *client* this + * module needs, pumps usb_host_client_handle_ + * events() (which is also how *this task's* + * control- and bulk-transfer completion + * callbacks get dispatched — the ESP-IDF USB + * Host Library calls transfer callbacks + * synchronously from inside whichever task + * called *_handle_events(), not from an ISR + * or a hidden thread). On NEW_DEV it attempts + * bring-up; on DEV_GONE it tears down. + * rtlsdr_exp_forward_task — drains the IQ block queue that the bulk + * transfer callback feeds and hands each + * block to rtlsdr_exp_forward_iq_block() + * (the upstream-forwarding STUB). + * + * Bulk IQ reads are pipelined: `bulk_read_queue_depth` transfers are kept + * perpetually in flight (each callback re-submits itself), so the USB pipe + * doesn't idle waiting for the forward task to catch up. Completed buffers + * are handed off via a FreeRTOS queue; if the forward task falls behind, + * blocks are DROPPED (counted in stats.iq_blocks_dropped) rather than + * blocking the USB callback — losing samples is preferable to stalling the + * USB Host Library's event dispatch, which would also stall the client's + * disconnect detection. Whether this is an acceptable loss policy for the + * real product is exactly the kind of thing that needs real-hardware/real- + * throughput testing — see README. + */ + +#include +#include + +#include "freertos/FreeRTOS.h" +#include "freertos/task.h" +#include "freertos/queue.h" + +#include "esp_log.h" +#include "esp_check.h" + +#include "usb/usb_host.h" +#include "usb/usb_helpers.h" +#include "usb/usb_types_ch9.h" + +#include "esp_http_client.h" + +#include "rtlsdr_experimental.h" + +static const char *TAG = "rtlsdr_exp"; + +const uint16_t RTLSDR_EXP_KNOWN_PRODUCT_IDS[4] = { 0x2832, 0x2834, 0x2838, 0x2837 }; + +/* --------------------------------------------------------------------------- + * Vendor-command constants — ported verbatim from frontend/src/lib/sdr.ts + * (CTRL_IN / CTRL_OUT / DEMOD / USB_EPA / SYS / PAGE_USB, lines ~37-42). + * ------------------------------------------------------------------------ */ +#define RTLSDR_CTRL_IN 0xC0u /* bmRequestType: IN | vendor | device */ +#define RTLSDR_CTRL_OUT 0x40u /* bmRequestType: OUT | vendor | device */ +#define RTLSDR_BLOCK_DEMOD 0x03u +#define RTLSDR_BLOCK_USB_EPA 0x02u +#define RTLSDR_BLOCK_SYS 0x09u +#define RTLSDR_PAGE_USB 0x01u + +/* Default tuning + read-loop parameters (rtlsdr_exp_config_default). These + * mirror sdr.ts's open(sampleRateHz = 2_048_000) and its 98 MHz test tune; + * bulk_read_chunk_bytes/queue_depth are new (sdr.ts's readSamples() takes + * an explicit byte count per call from its sweep() caller, there is no + * fixed default there to inherit) and are UNVERIFIED guesses pending real + * throughput measurement. */ +#define RTLSDR_EXP_DEFAULT_CENTER_HZ 98000000u +#define RTLSDR_EXP_DEFAULT_SAMPLE_RATE 2048000u +#define RTLSDR_EXP_DEFAULT_CHUNK_BYTES 16384u /* 32 * 512, USB HS multiple */ +#define RTLSDR_EXP_DEFAULT_QUEUE_DEPTH 4u +#define RTLSDR_EXP_FORWARD_QUEUE_LEN 8u /* IQ blocks buffered for forward task */ +#define RTLSDR_EXP_CTRL_XFER_TIMEOUT_MS 1000u +#define RTLSDR_EXP_USB_INTERFACE_NUM 0u + +struct rtlsdr_exp_handle_s { + rtlsdr_exp_config_t cfg; + + usb_host_client_handle_t client_hdl; + usb_device_handle_t dev_hdl; + bool device_open; + bool interface_claimed; + uint8_t ep_in_addr; /* bulk IN endpoint address, once found */ + + /* pending work handed from the client-event callback (kept short, per + * ESP-IDF USB Host guidance) to the client task's main loop */ + volatile uint8_t pending_new_dev_addr; /* 0 == none pending */ + volatile bool pending_dev_gone; + + volatile bool running; /* set false by rtlsdr_exp_stop() */ + volatile bool streaming; /* bulk read loop active */ + + TaskHandle_t lib_task; + TaskHandle_t client_task; + TaskHandle_t forward_task; + + QueueHandle_t iq_block_queue; /* holds rtlsdr_iq_block_t* */ + + /* control-transfer synchronous-wait bookkeeping. Safe as plain fields + * (not a semaphore) because control transfers issued by this module are + * only ever awaited from the same task that also pumps + * usb_host_client_handle_events() — see file header. */ + volatile bool ctrl_xfer_pending; + volatile bool ctrl_xfer_ok; + + usb_transfer_t *bulk_transfers[8]; /* sized to the max we allow bulk_read_queue_depth to be */ + + rtlsdr_exp_stats_t stats; +}; + +typedef struct { + uint8_t *data; + size_t len; +} rtlsdr_iq_block_t; + +/* --------------------------------------------------------------------------- + * Public: config defaults + * ------------------------------------------------------------------------ */ +void rtlsdr_exp_config_default(rtlsdr_exp_config_t *config) +{ + if (!config) return; + memset(config, 0, sizeof(*config)); + config->center_freq_hz = RTLSDR_EXP_DEFAULT_CENTER_HZ; + config->sample_rate_hz = RTLSDR_EXP_DEFAULT_SAMPLE_RATE; + config->bulk_read_chunk_bytes = RTLSDR_EXP_DEFAULT_CHUNK_BYTES; + config->bulk_read_queue_depth = RTLSDR_EXP_DEFAULT_QUEUE_DEPTH; + config->iq_upload_url = NULL; /* caller must set */ + config->device_bearer_token = NULL; /* caller must set */ +} + +/* --------------------------------------------------------------------------- + * Low-level control-transfer helpers. + * Direct port of sdr.ts's private writeReg()/demodWrite()/i2cWrite() + * (lines ~252-303). Same wValue/wIndex encoding, same register addresses, + * same repeater on/off dance for tuner I2C access. The WebUSB calls + * + * dev.controlTransferOut({ requestType: 'vendor', recipient: 'device', + * request: 0, value: ..., index: ... }, data) + * + * become a manually-built usb_setup_packet_t with bmRequestType = + * RTLSDR_CTRL_OUT (0x40 — OUT | vendor | device, matching WebUSB's + * 'vendor'/'device' fields), bRequest = 0, wValue/wIndex as below, wLength + * = payload length, submitted via usb_host_transfer_submit_control(). + * ------------------------------------------------------------------------ */ + +/* Synchronously wait for a previously-submitted control transfer to + * complete by pumping usb_host_client_handle_events() from THIS task (see + * file header for why that's safe/required here). Returns ESP_OK if the + * transfer completed successfully within the timeout. */ +static esp_err_t rtlsdr_wait_ctrl_xfer(rtlsdr_exp_handle_t h) +{ + TickType_t start = xTaskGetTickCount(); + TickType_t timeout_ticks = pdMS_TO_TICKS(RTLSDR_EXP_CTRL_XFER_TIMEOUT_MS); + while (h->ctrl_xfer_pending) { + usb_host_client_handle_events(h->client_hdl, pdMS_TO_TICKS(10)); + if ((xTaskGetTickCount() - start) > timeout_ticks) { + ESP_LOGW(TAG, "control transfer timed out"); + h->ctrl_xfer_pending = false; + return ESP_ERR_TIMEOUT; + } + } + return h->ctrl_xfer_ok ? ESP_OK : ESP_FAIL; +} + +static void rtlsdr_ctrl_xfer_cb(usb_transfer_t *transfer) +{ + rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)transfer->context; + h->ctrl_xfer_ok = (transfer->status == USB_TRANSFER_STATUS_COMPLETED); + h->ctrl_xfer_pending = false; + usb_host_transfer_free(transfer); +} + +/* value/index encoding matches sdr.ts's writeReg()/demodWrite() exactly: + * writeReg: wValue = (block<<8)|0x10, wIndex = address + * demodWrite: wValue = (DEMOD<<8)|0x10, wIndex = (page<<8)|address + * `length` is 1 or 2 bytes of little-endian `value`, same as sdr.ts. */ +static esp_err_t rtlsdr_ctrl_write_raw(rtlsdr_exp_handle_t h, uint16_t wValue, + uint16_t wIndex, uint16_t value, uint8_t length) +{ + if (!h->dev_hdl) return ESP_ERR_INVALID_STATE; + + usb_transfer_t *transfer = NULL; + esp_err_t err = usb_host_transfer_alloc(sizeof(usb_setup_packet_t) + length, 0, &transfer); + if (err != ESP_OK) return err; + + usb_setup_packet_t *setup = (usb_setup_packet_t *)transfer->data_buffer; + setup->bmRequestType = RTLSDR_CTRL_OUT; + setup->bRequest = 0; + setup->wValue = wValue; + setup->wIndex = wIndex; + setup->wLength = length; + + uint8_t *payload = transfer->data_buffer + sizeof(usb_setup_packet_t); + payload[0] = (uint8_t)(value & 0xFF); + if (length > 1) payload[1] = (uint8_t)((value >> 8) & 0xFF); + + transfer->device_handle = h->dev_hdl; + transfer->bEndpointAddress = 0; /* control endpoint */ + transfer->num_bytes = sizeof(usb_setup_packet_t) + length; + transfer->callback = rtlsdr_ctrl_xfer_cb; + transfer->context = h; + transfer->timeout_ms = RTLSDR_EXP_CTRL_XFER_TIMEOUT_MS; + + h->ctrl_xfer_pending = true; + h->ctrl_xfer_ok = false; + err = usb_host_transfer_submit_control(h->client_hdl, transfer); + if (err != ESP_OK) { + h->ctrl_xfer_pending = false; + usb_host_transfer_free(transfer); + return err; + } + return rtlsdr_wait_ctrl_xfer(h); +} + +/* writeReg() equivalent — sdr.ts private writeReg(block, address, value, length) */ +static esp_err_t rtlsdr_reg_write(rtlsdr_exp_handle_t h, uint8_t block, uint16_t address, + uint16_t value, uint8_t length) +{ + uint16_t wValue = (uint16_t)((block << 8) | 0x10); + return rtlsdr_ctrl_write_raw(h, wValue, address, value, length); +} + +/* demodWrite() equivalent — sdr.ts private demodWrite(page, address, value, length) */ +static esp_err_t rtlsdr_demod_write(rtlsdr_exp_handle_t h, uint8_t page, uint16_t address, + uint16_t value, uint8_t length) +{ + uint16_t wValue = (uint16_t)((RTLSDR_BLOCK_DEMOD << 8) | 0x10); + uint16_t wIndex = (uint16_t)((page << 8) | address); + return rtlsdr_ctrl_write_raw(h, wValue, wIndex, value, length); +} + +/* i2cWrite() equivalent — sdr.ts private i2cWrite(i2cAddr, reg, value). + * Tunnels a single-byte tuner register write through the demod's I2C + * repeater: repeater on (demod 1/0x02 = 0x41) -> vendor write addressed to + * (i2cAddr<<8)|reg -> repeater off (demod 1/0x02 = 0x01). */ +static esp_err_t rtlsdr_i2c_write(rtlsdr_exp_handle_t h, uint8_t i2c_addr, uint8_t reg, uint8_t value) +{ + esp_err_t err = rtlsdr_demod_write(h, 1, 0x02, 0x41, 1); /* repeater on */ + if (err != ESP_OK) return err; + + uint16_t wValue = (uint16_t)((0x02 << 8) | 0x10); + uint16_t wIndex = (uint16_t)((i2c_addr << 8) | reg); + err = rtlsdr_ctrl_write_raw(h, wValue, wIndex, value, 1); + + esp_err_t err2 = rtlsdr_demod_write(h, 1, 0x02, 0x01, 1); /* repeater off, always attempted */ + return (err != ESP_OK) ? err : err2; +} + +/* --------------------------------------------------------------------------- + * Tuning — port of sdr.ts's setFrequency()/setSampleRate() (lines + * ~156-180). Same simplifications, same "HARDWARE PASS REQUIRED" caveat: + * real librtlsdr computes exact sdm/vco from the crystal reference; this + * is the same reduced integer-N approximation sdr.ts uses, carried over + * unchanged rather than re-derived (see this project's Honesty-policy note + * — the browser driver's math is the primary reference, not a place to + * invent new, unverified math on top of already-unverified math). + * ------------------------------------------------------------------------ */ +esp_err_t rtlsdr_exp_set_frequency(rtlsdr_exp_handle_t h, uint32_t hz) +{ + if (!h || !h->dev_hdl) return ESP_ERR_INVALID_STATE; + + uint32_t lo_hz = hz + 3570000u; /* R820T IF offset */ + uint32_t ref = 28800000u; + uint32_t mix_div = 2u; + uint32_t nint = lo_hz / (ref * mix_div); + uint32_t vco = lo_hz % (ref * mix_div); + uint32_t sdm = (uint32_t)(((uint64_t)vco * 65536u) / (ref * mix_div)); + if (sdm > 0xFFFFu) sdm = 0xFFFFu; + uint8_t reg = (uint8_t)(nint & 0x3F); + + esp_err_t err; + ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1A, 0x10, reg)), TAG, "pll nint"); + ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1A, 0x11, (sdm >> 8) & 0xFF)), TAG, "pll sdm hi"); + ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1A, 0x12, sdm & 0xFF)), TAG, "pll sdm lo"); + h->cfg.center_freq_hz = hz; + return ESP_OK; +} + +esp_err_t rtlsdr_exp_set_sample_rate(rtlsdr_exp_handle_t h, uint32_t hz) +{ + if (!h || !h->dev_hdl) return ESP_ERR_INVALID_STATE; + + uint32_t crystal = 28800000u; + uint32_t rsamp_ratio = (uint32_t)((((uint64_t)crystal << 22) / hz)) & 0x0FFFFFFCu; + + esp_err_t err; + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x9f, (rsamp_ratio >> 16) & 0xFFFF, 2)), TAG, "rsamp hi"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0xa1, rsamp_ratio & 0xFFFF, 2)), TAG, "rsamp lo"); + h->cfg.sample_rate_hz = hz; + return ESP_OK; +} + +/* --------------------------------------------------------------------------- + * Init sequence — port of sdr.ts's open() body (lines ~110-140): demod + * soft reset -> demod_ctl/suspend-off register block -> standby off -> AGC + * mode -> R820T tuner power-up over the I2C repeater -> sample rate -> + * initial tune -> streaming endpoint reset -> test-mode off. Same order, + * same register addresses/values, unchanged. + * ------------------------------------------------------------------------ */ +static esp_err_t rtlsdr_run_init_sequence(rtlsdr_exp_handle_t h) +{ + esp_err_t err; + + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x01, 0x14, 1)), TAG, "soft reset"); /* soft reset */ + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x01, 0x10, 1)), TAG, "reset clear"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 0, 0x01, 0x08, 2)), TAG, "demod_ctl"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 0, 0x06, 0x80, 1)), TAG, "demod_ctl2"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x15, 0x00, 1)), TAG, "suspend off 0x15"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x16, 0x00, 1)), TAG, "suspend off 0x16"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x17, 0x00, 1)), TAG, "suspend off 0x17"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x18, 0x00, 1)), TAG, "suspend off 0x18"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x19, 0x00, 1)), TAG, "suspend off 0x19"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x1a, 0x00, 1)), TAG, "suspend off 0x1a"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x1b, 0x00, 1)), TAG, "suspend off 0x1b"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x1c, 0x00, 1)), TAG, "suspend off 0x1c"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x0d, 0x83, 1)), TAG, "standby off"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x0b, 0x1b, 1)), TAG, "AGC mode"); + + /* R820T power-up through I2C repeater (addr 0x1a) */ + ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1a, 0x05, 0x8f)), TAG, "LNA power"); + ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1a, 0x08, 0x80)), TAG, "mixer"); + ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1a, 0x0a, 0x10)), TAG, "IF filter"); + + ESP_RETURN_ON_ERROR((err = rtlsdr_exp_set_sample_rate(h, h->cfg.sample_rate_hz)), TAG, "sample rate"); + ESP_RETURN_ON_ERROR((err = rtlsdr_exp_set_frequency(h, h->cfg.center_freq_hz)), TAG, "frequency"); + + /* Reset streaming endpoint before bulk reads begin. */ + ESP_RETURN_ON_ERROR((err = rtlsdr_reg_write(h, RTLSDR_BLOCK_USB_EPA, 0x0001, 0xFFFF, 2)), TAG, "ep reset"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x02, 0x00, 1)), TAG, "streaming pre"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 0, 0x02, 0x40, 2)), TAG, "streaming enable"); + ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x02, 0x00, 1)), TAG, "test mode off"); + + return ESP_OK; +} + +/* --------------------------------------------------------------------------- + * Bulk-IN read loop — NOT present in sdr.ts in this pipelined form (the + * WebUSB driver does one-shot `await dev.transferIn(...)` calls from + * inside its own async sweep loop; ESP-IDF's USB Host Library is callback- + * driven and async by design, so continuous streaming needs an explicit + * resubmit-on-completion pipeline instead). This is the least-verified + * part of this whole module — see README "What's unverified" — because + * correct chunk size, queue depth, and drop policy all depend on real + * measured USB throughput on real ESP32-P4 silicon, which nobody involved + * in this workstream has access to. + * ------------------------------------------------------------------------ */ +static void rtlsdr_bulk_xfer_cb(usb_transfer_t *transfer) +{ + rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)transfer->context; + + if (transfer->status == USB_TRANSFER_STATUS_COMPLETED && transfer->actual_num_bytes > 0) { + h->stats.bulk_reads_ok++; + + rtlsdr_iq_block_t *block = malloc(sizeof(rtlsdr_iq_block_t)); + uint8_t *copy = block ? malloc(transfer->actual_num_bytes) : NULL; + if (block && copy) { + memcpy(copy, transfer->data_buffer, transfer->actual_num_bytes); + block->data = copy; + block->len = (size_t)transfer->actual_num_bytes; + /* Non-blocking send: dropping is preferred to stalling the USB + * callback path (see file header). Drop is counted, not silent. */ + if (h->iq_block_queue && xQueueSend(h->iq_block_queue, &block, 0) != pdTRUE) { + h->stats.iq_blocks_dropped++; + free(copy); + free(block); + } + } else { + h->stats.iq_blocks_dropped++; + free(copy); + free(block); + } + } else if (transfer->status != USB_TRANSFER_STATUS_COMPLETED) { + h->stats.bulk_reads_failed++; + ESP_LOGW(TAG, "bulk IN transfer failed, status=%d", transfer->status); + } + + /* Keep the pipe full: resubmit immediately unless we're stopping. */ + if (h->streaming && h->running) { + esp_err_t err = usb_host_transfer_submit(transfer); + if (err != ESP_OK) { + ESP_LOGW(TAG, "failed to resubmit bulk transfer: %d", err); + h->stats.bulk_reads_failed++; + } + } +} + +static esp_err_t rtlsdr_start_bulk_streaming(rtlsdr_exp_handle_t h) +{ + size_t depth = h->cfg.bulk_read_queue_depth; + if (depth == 0 || depth > (sizeof(h->bulk_transfers) / sizeof(h->bulk_transfers[0]))) { + depth = RTLSDR_EXP_DEFAULT_QUEUE_DEPTH; + } + if (h->cfg.bulk_read_chunk_bytes == 0 || (h->cfg.bulk_read_chunk_bytes % 512) != 0) { + ESP_LOGW(TAG, "bulk_read_chunk_bytes not a multiple of 512, forcing default"); + h->cfg.bulk_read_chunk_bytes = RTLSDR_EXP_DEFAULT_CHUNK_BYTES; + } + + for (size_t i = 0; i < depth; i++) { + usb_transfer_t *t = NULL; + esp_err_t err = usb_host_transfer_alloc(h->cfg.bulk_read_chunk_bytes, 0, &t); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to allocate bulk transfer %zu: %d", i, err); + return err; + } + t->device_handle = h->dev_hdl; + t->bEndpointAddress = h->ep_in_addr; + t->num_bytes = h->cfg.bulk_read_chunk_bytes; + t->callback = rtlsdr_bulk_xfer_cb; + t->context = h; + t->timeout_ms = 0; /* no timeout: this endpoint is expected to be + * continuously producing once streaming is on; + * UNVERIFIED whether 0 (no timeout) is the + * right choice vs. a bounded timeout + retry — + * needs real-hardware behavior to decide. */ + h->bulk_transfers[i] = t; + } + + h->streaming = true; + for (size_t i = 0; i < depth; i++) { + esp_err_t err = usb_host_transfer_submit(h->bulk_transfers[i]); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to submit initial bulk transfer %zu: %d", i, err); + h->streaming = false; + return err; + } + } + + ESP_LOGI(TAG, "bulk IQ streaming started: chunk=%zuB depth=%zu", + h->cfg.bulk_read_chunk_bytes, depth); + return ESP_OK; +} + +static void rtlsdr_stop_bulk_streaming(rtlsdr_exp_handle_t h) +{ + h->streaming = false; + for (size_t i = 0; i < sizeof(h->bulk_transfers) / sizeof(h->bulk_transfers[0]); i++) { + if (h->bulk_transfers[i]) { + usb_host_transfer_free(h->bulk_transfers[i]); + h->bulk_transfers[i] = NULL; + } + } +} + +/* --------------------------------------------------------------------------- + * Upstream IQ forwarding — STUB. See README "IQ forwarding architecture + * decision" for the full reasoning. Summary: raw IQ at even a modest + * 2.048 Msps / 8-bit-per-sample-per-channel is ~4.1 MB/s, which is wildly + * incompatible with the regular `POST /api/device/telemetry` shape (per + * the ESP32-P4 Sensor Node spec's contract: readings array capped at 64 + * entries, request body capped at 16KB, rate-limited to ~1 req/sec + * sustained per device) — that endpoint is sized for a handful of scalar + * sensor readings, not a continuous binary stream. Rather than distort + * that endpoint's shape (e.g. base64-stuffing IQ bytes into a JSON + * "reading" — which would also add ~33% overhead on top of an already + * too-large payload), this stub targets a SEPARATE, not-yet-implemented + * endpoint carrying raw binary chunks, authenticated the same way (device + * bearer token) but outside the JSON telemetry contract entirely. + * + * This function is intentionally inert by default (returns early unless + * iq_upload_url is set) because: + * 1. No backend endpoint for this exists anywhere in this repo yet — + * building one is explicitly out of scope for this workstream. + * 2. Even the *shape* proposed here (raw octet-stream POST per block, + * small fixed binary header) is a guess pending: (a) real measured + * USB throughput off actual hardware, (b) a product decision on + * whether continuous IQ upload is even desired given typical + * home-WiFi-uplink bandwidth, and (c) whether a WebSocket stream + * would suit the backend's existing async pub/sub model (see the + * spec's /ws/device-feed design) better than repeated POSTs. + * ------------------------------------------------------------------------ */ + +/* Minimal fixed binary header prepended to each forwarded chunk, so the + * (not-yet-existing) backend endpoint can frame chunks without needing + * JSON/base64 parsing on a hot path. All fields little-endian. + * UNVERIFIED / PROPOSED — not agreed with any backend implementation. */ +typedef struct __attribute__((packed)) { + uint32_t magic; /* 'QMIQ' = 0x51 0x4D 0x49 0x51 */ + uint32_t seq; + uint32_t center_hz; + uint32_t sample_rate_hz; + uint32_t payload_len; +} rtlsdr_iq_chunk_header_t; + +#define RTLSDR_IQ_CHUNK_MAGIC 0x51494D51u /* "QMIQ" */ + +static esp_err_t rtlsdr_exp_forward_iq_block(rtlsdr_exp_handle_t h, const uint8_t *data, size_t len) +{ + if (!h->cfg.iq_upload_url || !h->cfg.device_bearer_token) { + /* No upload target configured — this is the expected state until a + * real backend endpoint exists and a caller opts in. Not an error. */ + return ESP_OK; + } + + static uint32_t s_seq = 0; + rtlsdr_iq_chunk_header_t hdr = { + .magic = RTLSDR_IQ_CHUNK_MAGIC, + .seq = s_seq++, + .center_hz = h->cfg.center_freq_hz, + .sample_rate_hz = h->cfg.sample_rate_hz, + .payload_len = (uint32_t)len, + }; + + esp_http_client_config_t http_cfg = { + .url = h->cfg.iq_upload_url, + .method = HTTP_METHOD_POST, + .timeout_ms = 5000, + }; + esp_http_client_handle_t client = esp_http_client_init(&http_cfg); + if (!client) return ESP_FAIL; + + char auth_header[512]; + snprintf(auth_header, sizeof(auth_header), "Bearer %s", h->cfg.device_bearer_token); + esp_http_client_set_header(client, "Authorization", auth_header); + esp_http_client_set_header(client, "Content-Type", "application/octet-stream"); + + esp_err_t err = esp_http_client_open(client, (int)(sizeof(hdr) + len)); + if (err == ESP_OK) { + int written = esp_http_client_write(client, (const char *)&hdr, sizeof(hdr)); + if (written == (int)sizeof(hdr)) { + written = esp_http_client_write(client, (const char *)data, (int)len); + } + if (written < 0) err = ESP_FAIL; + esp_http_client_fetch_headers(client); + int status = esp_http_client_get_status_code(client); + if (status < 200 || status >= 300) { + ESP_LOGW(TAG, "IQ upload got HTTP %d (endpoint likely doesn't exist yet)", status); + err = ESP_FAIL; + } + } + esp_http_client_close(client); + esp_http_client_cleanup(client); + + if (err == ESP_OK) { + h->stats.bytes_forwarded_upstream += (uint32_t)len; + } else { + h->stats.forward_failures++; + } + return err; +} + +static void rtlsdr_forward_task(void *arg) +{ + rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)arg; + rtlsdr_iq_block_t *block = NULL; + + while (h->running) { + if (xQueueReceive(h->iq_block_queue, &block, pdMS_TO_TICKS(500)) == pdTRUE) { + rtlsdr_exp_forward_iq_block(h, block->data, block->len); + free(block->data); + free(block); + block = NULL; + } + } + + /* drain remaining queued blocks on shutdown without forwarding */ + while (xQueueReceive(h->iq_block_queue, &block, 0) == pdTRUE) { + free(block->data); + free(block); + } + vTaskDelete(NULL); +} + +/* --------------------------------------------------------------------------- + * Device bring-up + * ------------------------------------------------------------------------ */ +static bool rtlsdr_vendor_id_matches(uint16_t vid) +{ + return vid == RTLSDR_EXP_VENDOR_RTL2832U || vid == RTLSDR_EXP_VENDOR_TERRATEC; +} + +static void rtlsdr_log_known_product_hint(uint16_t vid, uint16_t pid) +{ + if (vid != RTLSDR_EXP_VENDOR_RTL2832U) return; + for (size_t i = 0; i < RTLSDR_EXP_NUM_KNOWN_PRODUCT_IDS; i++) { + if (RTLSDR_EXP_KNOWN_PRODUCT_IDS[i] == pid) return; + } + ESP_LOGI(TAG, "product id 0x%04x not in the known RTL2832U list — " + "attempting bring-up anyway (vendor-only match, same policy " + "as frontend/src/lib/sdr.ts's WebUSB device filter)", pid); +} + +/* Finds the first bulk-IN endpoint on the device's active configuration's + * first interface. Mirrors sdr.ts's: + * iface = dev.configuration.interfaces[0]; alt = iface.alternates[0]; + * ep = alt.endpoints.find(e => e.direction==='in' && e.type==='bulk') */ +static esp_err_t rtlsdr_find_bulk_in_endpoint(usb_device_handle_t dev_hdl, uint8_t *out_ep) +{ + const usb_config_desc_t *config_desc = NULL; + esp_err_t err = usb_host_get_active_config_descriptor(dev_hdl, &config_desc); + if (err != ESP_OK || !config_desc) return ESP_FAIL; + + int offset = 0; + const usb_intf_desc_t *intf = usb_parse_interface_descriptor(config_desc, 0, 0, &offset); + if (!intf) return ESP_FAIL; + + for (int i = 0; i < intf->bNumEndpoints; i++) { + int ep_offset = offset; + const usb_ep_desc_t *ep = usb_parse_endpoint_descriptor_by_index( + intf, i, config_desc->wTotalLength, &ep_offset); + if (!ep) continue; + bool is_in = (ep->bEndpointAddress & USB_B_ENDPOINT_ADDRESS_EP_DIR_MASK) != 0; + bool is_bulk = (ep->bmAttributes & USB_BM_ATTRIBUTES_XFERTYPE_MASK) == USB_BM_ATTRIBUTES_XFER_BULK; + if (is_in && is_bulk) { + *out_ep = ep->bEndpointAddress; + return ESP_OK; + } + } + return ESP_ERR_NOT_FOUND; +} + +static void rtlsdr_teardown_device(rtlsdr_exp_handle_t h) +{ + rtlsdr_stop_bulk_streaming(h); + if (h->interface_claimed) { + usb_host_interface_release(h->client_hdl, h->dev_hdl, RTLSDR_EXP_USB_INTERFACE_NUM); + h->interface_claimed = false; + } + if (h->device_open) { + usb_host_device_close(h->client_hdl, h->dev_hdl); + h->device_open = false; + } + h->dev_hdl = NULL; + h->stats.device_present = false; + h->stats.bring_up_ok = false; + h->stats.streaming = false; +} + +static void rtlsdr_try_bring_up(rtlsdr_exp_handle_t h, uint8_t dev_addr) +{ + esp_err_t err = usb_host_device_open(h->client_hdl, dev_addr, &h->dev_hdl); + if (err != ESP_OK) { + ESP_LOGW(TAG, "usb_host_device_open failed: %d", err); + return; + } + h->device_open = true; + + const usb_device_desc_t *dev_desc = NULL; + err = usb_host_get_device_descriptor(h->dev_hdl, &dev_desc); + if (err != ESP_OK || !dev_desc) { + ESP_LOGW(TAG, "could not read device descriptor: %d", err); + rtlsdr_teardown_device(h); + return; + } + + if (!rtlsdr_vendor_id_matches(dev_desc->idVendor)) { + ESP_LOGD(TAG, "vendor 0x%04x is not RTL2832U/Terratec, ignoring", dev_desc->idVendor); + rtlsdr_teardown_device(h); + return; + } + rtlsdr_log_known_product_hint(dev_desc->idVendor, dev_desc->idProduct); + ESP_LOGI(TAG, "candidate RTL2832U-class device: vid=0x%04x pid=0x%04x", + dev_desc->idVendor, dev_desc->idProduct); + + err = usb_host_interface_claim(h->client_hdl, h->dev_hdl, RTLSDR_EXP_USB_INTERFACE_NUM, 0); + if (err != ESP_OK) { + ESP_LOGW(TAG, "could not claim interface %d: %d — device may be claimed by " + "another driver, or this isn't the RTL2832U bulk interface", + RTLSDR_EXP_USB_INTERFACE_NUM, err); + rtlsdr_teardown_device(h); + return; + } + h->interface_claimed = true; + + err = rtlsdr_find_bulk_in_endpoint(h->dev_hdl, &h->ep_in_addr); + if (err != ESP_OK) { + ESP_LOGW(TAG, "no bulk IN endpoint found on interface %d", RTLSDR_EXP_USB_INTERFACE_NUM); + rtlsdr_teardown_device(h); + return; + } + + h->stats.device_present = true; + + err = rtlsdr_run_init_sequence(h); + if (err != ESP_OK) { + ESP_LOGE(TAG, "RTL2832U init vendor-command sequence failed at some step: %d " + "(UNVERIFIED sequence — see README, this is exactly the kind of " + "failure real hardware bring-up is expected to need to debug)", err); + rtlsdr_teardown_device(h); + return; + } + h->stats.bring_up_ok = true; + ESP_LOGI(TAG, "RTL2832U init sequence completed without a control-transfer error " + "(this does NOT confirm valid IQ is being produced — only a real " + "spectrum/logic-analyzer check against hardware can confirm that)"); + + err = rtlsdr_start_bulk_streaming(h); + if (err != ESP_OK) { + ESP_LOGE(TAG, "failed to start bulk IQ streaming: %d", err); + rtlsdr_teardown_device(h); + return; + } + h->stats.streaming = true; +} + +/* --------------------------------------------------------------------------- + * USB Host Library plumbing: daemon task + client task + client event cb + * ------------------------------------------------------------------------ */ +static void rtlsdr_usb_lib_daemon_task(void *arg) +{ + rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)arg; + while (h->running) { + uint32_t event_flags = 0; + usb_host_lib_handle_events(pdMS_TO_TICKS(1000), &event_flags); + if (event_flags & USB_HOST_LIB_EVENT_FLAGS_NO_CLIENTS) { + ESP_LOGD(TAG, "USB host lib: no clients"); + } + if (event_flags & USB_HOST_LIB_EVENT_FLAGS_ALL_FREE) { + ESP_LOGD(TAG, "USB host lib: all devices free"); + } + } + vTaskDelete(NULL); +} + +static void rtlsdr_client_event_cb(const usb_host_client_event_msg_t *event_msg, void *arg) +{ + rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)arg; + /* Deliberately kept short — per ESP-IDF USB Host guidance, heavy work + * (device open, control transfers) is done back in the client task's + * main loop, not inside this callback. */ + switch (event_msg->event) { + case USB_HOST_CLIENT_EVENT_NEW_DEV: + h->pending_new_dev_addr = event_msg->new_dev.address; + break; + case USB_HOST_CLIENT_EVENT_DEV_GONE: + h->pending_dev_gone = true; + break; + default: + break; + } +} + +static void rtlsdr_exp_client_task(void *arg) +{ + rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)arg; + + usb_host_client_config_t client_config = { + .is_synchronous = false, + .max_num_event_msg = 5, + .async = { + .client_event_callback = rtlsdr_client_event_cb, + .callback_arg = h, + }, + }; + esp_err_t err = usb_host_client_register(&client_config, &h->client_hdl); + if (err != ESP_OK) { + ESP_LOGE(TAG, "usb_host_client_register failed: %d", err); + h->running = false; + vTaskDelete(NULL); + return; + } + + while (h->running) { + /* This call is also what dispatches completion callbacks for any + * control/bulk transfers this client has outstanding — see file + * header. Timeout keeps the loop responsive to h->running. */ + usb_host_client_handle_events(h->client_hdl, pdMS_TO_TICKS(200)); + + if (h->pending_new_dev_addr != 0) { + uint8_t addr = h->pending_new_dev_addr; + h->pending_new_dev_addr = 0; + if (!h->dev_hdl) { /* only attempt bring-up if we don't already hold a device */ + rtlsdr_try_bring_up(h, addr); + } + } + if (h->pending_dev_gone) { + h->pending_dev_gone = false; + ESP_LOGW(TAG, "RTL2832U device disconnected"); + rtlsdr_teardown_device(h); + } + } + + rtlsdr_teardown_device(h); + usb_host_client_deregister(h->client_hdl); + h->client_hdl = NULL; + vTaskDelete(NULL); +} + +/* --------------------------------------------------------------------------- + * Public lifecycle API + * ------------------------------------------------------------------------ */ +esp_err_t rtlsdr_exp_init(const rtlsdr_exp_config_t *config, rtlsdr_exp_handle_t *out_handle) +{ + if (!config || !out_handle) return ESP_ERR_INVALID_ARG; + + rtlsdr_exp_handle_t h = calloc(1, sizeof(struct rtlsdr_exp_handle_s)); + if (!h) return ESP_ERR_NO_MEM; + + h->cfg = *config; + if (h->cfg.center_freq_hz == 0) h->cfg.center_freq_hz = RTLSDR_EXP_DEFAULT_CENTER_HZ; + if (h->cfg.sample_rate_hz == 0) h->cfg.sample_rate_hz = RTLSDR_EXP_DEFAULT_SAMPLE_RATE; + if (h->cfg.bulk_read_chunk_bytes == 0) h->cfg.bulk_read_chunk_bytes = RTLSDR_EXP_DEFAULT_CHUNK_BYTES; + if (h->cfg.bulk_read_queue_depth == 0) h->cfg.bulk_read_queue_depth = RTLSDR_EXP_DEFAULT_QUEUE_DEPTH; + + h->iq_block_queue = xQueueCreate(RTLSDR_EXP_FORWARD_QUEUE_LEN, sizeof(rtlsdr_iq_block_t *)); + if (!h->iq_block_queue) { + free(h); + return ESP_ERR_NO_MEM; + } + + *out_handle = h; + return ESP_OK; +} + +esp_err_t rtlsdr_exp_deinit(rtlsdr_exp_handle_t h) +{ + if (!h) return ESP_ERR_INVALID_ARG; + if (h->running) { + rtlsdr_exp_stop(h); + } + if (h->iq_block_queue) vQueueDelete(h->iq_block_queue); + free(h); + return ESP_OK; +} + +esp_err_t rtlsdr_exp_start(rtlsdr_exp_handle_t h) +{ + if (!h) return ESP_ERR_INVALID_ARG; + if (h->running) return ESP_ERR_INVALID_STATE; + + /* NOTE: usb_host_install() installs a library instance shared by the + * whole firmware process. If Workstream I's core skeleton (or any + * future module) also needs USB host for something else, installing + * it twice is an error — this experimental module assumes it is the + * ONLY USB Host client in the project. Document/resolve this at merge + * time; see README integration notes. */ + usb_host_config_t host_config = { + .skip_phy_setup = false, + .intr_flags = ESP_INTR_FLAG_LEVEL1, + }; + esp_err_t err = usb_host_install(&host_config); + if (err != ESP_OK) { + ESP_LOGE(TAG, "usb_host_install failed: %d", err); + return err; + } + + h->running = true; + + BaseType_t ok = xTaskCreate(rtlsdr_usb_lib_daemon_task, "rtlsdr_usb_lib", 4096, h, 5, &h->lib_task); + if (ok != pdPASS) { + h->running = false; + usb_host_uninstall(); + return ESP_ERR_NO_MEM; + } + + ok = xTaskCreate(rtlsdr_exp_client_task, "rtlsdr_client", 6144, h, 5, &h->client_task); + if (ok != pdPASS) { + h->running = false; + vTaskDelete(h->lib_task); + usb_host_uninstall(); + return ESP_ERR_NO_MEM; + } + + ok = xTaskCreate(rtlsdr_forward_task, "rtlsdr_forward", 4096, h, 4, &h->forward_task); + if (ok != pdPASS) { + h->running = false; + vTaskDelete(h->client_task); + vTaskDelete(h->lib_task); + usb_host_uninstall(); + return ESP_ERR_NO_MEM; + } + + ESP_LOGI(TAG, "rtlsdr_experimental started (UNVERIFIED against real hardware — " + "see firmware/esp32p4-sensor-node/README.md)"); + return ESP_OK; +} + +esp_err_t rtlsdr_exp_stop(rtlsdr_exp_handle_t h) +{ + if (!h) return ESP_ERR_INVALID_ARG; + if (!h->running) return ESP_OK; + + h->running = false; + /* Tasks self-delete once they observe h->running == false; give them a + * moment. A production version should use task-completion notification + * instead of a fixed delay — left as a TODO, not hardware-dependent but + * still unverified/untuned. */ + vTaskDelay(pdMS_TO_TICKS(500)); + + usb_host_uninstall(); /* see rtlsdr_exp_start() note re: sole USB client assumption */ + return ESP_OK; +} + +void rtlsdr_exp_get_stats(rtlsdr_exp_handle_t h, rtlsdr_exp_stats_t *out_stats) +{ + if (!h || !out_stats) return; + *out_stats = h->stats; +}