drjones/qtalker---
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(firmware): add experimental RTL-SDR USB-host module (Workstream J)

Browse Source 
Self-contained ESP-IDF component (firmware/esp32p4-sensor-node/components/
rtlsdr_experimental/) exploring RTL2832U-over-USB-host on the ESP32-P4,
ported from frontend/src/lib/sdr.ts's researched WebUSB protocol sequence
(vendor commands, I2C-repeater tuner init) to the ESP-IDF USB Host Library.

Implements: USB Host Library install/client lifecycle, RTL2832U/Terratec
vendor-ID device matching, the demod+R820T init vendor-command sequence
over control transfers, a pipelined bulk-IN read loop for raw IQ, and an
inert-by-default upstream IQ-forwarding stub targeting a proposed separate
binary endpoint (not the JSON telemetry shape — reasoning documented in
the README) since no such backend endpoint exists yet.

Off by default (RTLSDR_EXP_ENABLE Kconfig, default n). Unverified against
real hardware and never compiled (no ESP-IDF toolchain in this
environment) — marked as such in every source file and in a dedicated
"Workstream J" section of firmware/esp32p4-sensor-node/README.md, which
this commit also creates since Workstream I's core skeleton (owned by a
separate, unmerged worktree) hadn't created one yet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Indiana
2026-07-24 01:10:21 +00:00
parent cf817e5241
commit abd7174c9c
5 changed files with 1440 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

324
firmware/esp32p4-sensor-node/README.md Normal file
View File

@@ -0,0 +1,324 @@
# ESP32-P4 Sensor Node — firmware
Firmware for the physical ESP32-P4 "Ultimate Quantum Box" sensor node
(see the ESP32-P4 Sensor Node design spec:
`docs/superpowers/specs/2026-07-23-esp32-sensor-node-design.md`). Joins
the seeker's home WiFi and streams sensor readings to `POST
/api/device/telemetry` on their paired account, which feeds the séance's
live anomaly-detection pipeline exactly like the browser-based Wire
Ghost/EVP/Spirit Radio/EMF modes already do.
## Honesty-policy note (binding on this whole directory)
This app's whole ethos is "real signal processing on real data, and it
says so when something is unverified" (see the frontend's Spirit Radio
`HARDWARE PASS REQUIRED` convention in `frontend/src/lib/sdr.ts`). Nobody
working on this firmware has physical ESP32-P4 hardware to flash and test
against, and this development environment has no ESP-IDF toolchain to
even compile it. **Every file under this directory is written to be
structurally correct ESP-IDF C against the documented API shape, and
reasoned about carefully — not verified against real hardware or even a
real build.** Anything that needs a real board to actually confirm is
called out explicitly, in code comments and in this README, rather than
glossed over. Treat "compiles cleanly / structurally sound" and "verified
on-device" as two entirely different claims — this repo only makes the
first one for anything under `firmware/`.
## Status of this directory
This copy of the repo currently contains **only Workstream J** (the
RTL-SDR experimental module, documented in full below). Workstream I
— the core firmware project skeleton (`CMakeLists.txt`,
`sdkconfig.defaults`, the `main/` component with WiFi station-mode
connect, the `/api/device/telemetry` HTTP client task, `main/
device_config.h`, and the BME280 + presence-sensor drivers) — was built
in a separate, isolated worktree that this workstream cannot see or
depend on. **A real build of this project needs Workstream I's skeleton
merged in alongside what's here.** Nothing in this directory currently
compiles standalone into a flashable image; the `components/
rtlsdr_experimental/` module is a self-contained ESP-IDF *component*,
deliberately structured so it can be dropped into Workstream I's project
via `EXTRA_COMPONENT_DIRS` (or copied under that project's own
`components/`) without editing any file that workstream owns. Once
merged, this README should be extended with Workstream I's own build
steps, wiring/pinout notes for the BME280 (I2C) and presence sensor
(UART/GPIO), and its own verified-vs-not accounting — that content
doesn't exist here yet because this workstream never had visibility into
that code.
## Directory layout (this workstream's contribution)
```
firmware/esp32p4-sensor-node/
├── README.md (this file)
└── components/
└── rtlsdr_experimental/ Self-contained ESP-IDF component.
├── CMakeLists.txt
├── Kconfig Off by default (RTLSDR_EXP_ENABLE=n).
├── include/
│ └── rtlsdr_experimental.h Public API + full disclaimer header.
└── rtlsdr_experimental.c Implementation.
```
---
## Workstream J — RTL-SDR experimental module (this workstream)
**⚠️ This is the least certain part of the entire firmware build. Read
this whole section before touching it.**
### What this is
A clearly-separated, opt-in, experimental module exploring USB-host
communication with an RTL2832U-based SDR dongle over the ESP32-P4's
native USB-OTG host controller (USB Host Library, `usb_host.h` — a real,
documented capability of this specific chip, unlike most ESP32 variants).
The idea, per the design spec: attach a cheap RTL-SDR dongle to the
sensor node, treat "spirit radio scanning" as a hardware-backed mode
instead of only a browser WebUSB feature.
**Be honest about the real constraint (this is the spec's framing, and
it's correct):** wideband IQ sample rates and FFT processing are
demanding relative to an MCU's compute, even one with ESP32-P4's AI
accelerator. **On-device spectrum analysis is not what this module
attempts.** The realistic architecture — and the one implemented here —
is: pull raw IQ samples off the dongle via USB host, and forward them
upstream for the *backend* to FFT/analyze (the same job the browser
already does client-side today via `frontend/src/lib/fft.ts` +
`frontend/src/lib/sdr.ts`'s `SpectrumAnomalyDetector`). Even that
"just pass the bytes through" architecture needs real USB throughput
numbers to know if it's viable — see "What's unverified" below.
### Primary reference
`frontend/src/lib/sdr.ts` — this repo's existing browser-based (WebUSB)
RTL2832U + R820T driver. It's real, already-researched protocol detail
against the public librtlsdr register documentation (vendor commands,
I2C-repeater tuner access, the demod/tuner init sequence), itself marked
`HARDWARE PASS REQUIRED` since it's never been run against a real dongle
either. This firmware module is a direct **port** of that file's control-
transfer sequence from WebUSB JS calls to ESP-IDF USB Host Library C
calls — line-by-line correspondences are called out in code comments
(e.g. `rtlsdr_run_init_sequence()` mirrors `open()` in sdr.ts almost
register-for-register). It intentionally does not re-derive any register
math from scratch; wherever sdr.ts says "simplified" or "HARDWARE PASS
REQUIRED" (e.g. the R820T PLL frequency math, the demod resample-ratio
math), this module carries the exact same simplification forward with the
exact same caveat, rather than inventing new unverified math on top of
already-unverified math.
### What's implemented (structurally — see caveats below)
1. **USB Host Library lifecycle** (`rtlsdr_exp_start()` /
`rtlsdr_usb_lib_daemon_task()` / `rtlsdr_exp_client_task()`):
`usb_host_install()`, `usb_host_client_register()` with an async event
callback, and the two-task pump pattern the USB Host Library's async
model requires (one task for `usb_host_lib_handle_events()`, one for
`usb_host_client_handle_events()` — the latter also being how this
module's own control- and bulk-transfer completion callbacks get
dispatched, since the Library calls them synchronously from whichever
task is pumping events, not from a hidden thread or ISR).
2. **RTL2832U/Terratec device enumeration and vendor-ID matching**
(`rtlsdr_try_bring_up()`, `rtlsdr_vendor_id_matches()`): on a
`USB_HOST_CLIENT_EVENT_NEW_DEV` event, opens the device, reads its
device descriptor, and matches `idVendor` against `0x0BDA`
(RTL2832U) / `0x0CCD` (Terratec-rebadged) — ported directly from
`sdr.ts`'s `requestDevice()` filter list (`RTL2832U_VENDOR`,
`TERRATEC_VENDOR`). Faithfully carries over that file's specific
choice to filter by **vendor ID only**, not product ID (its comment:
"many dongles report product ids outside the handful we know, so
filtering by productId hides them from the picker") — the known
product-id list (`RTLSDR_EXP_KNOWN_PRODUCT_IDS`) is kept as an
informational log line only, never a hard filter, exactly mirroring
how `RTL2832U_PRODUCTS` is exported-but-unused-as-a-filter in
`sdr.ts`.
3. **RTL2832U init vendor-command sequence** over USB control transfers
(`rtlsdr_run_init_sequence()`, `rtlsdr_demod_write()`,
`rtlsdr_reg_write()`, `rtlsdr_i2c_write()`, `rtlsdr_exp_set_frequency()`,
`rtlsdr_exp_set_sample_rate()`): the same demod soft-reset → demod_ctl/
suspend-off block → standby-off → AGC-mode → R820T tuner power-up
(through the I2C repeater) → sample-rate program → initial tune →
streaming-endpoint reset → test-mode-off sequence as `sdr.ts`'s
`open()`, with the same register addresses/values and the same
`wValue`/`wIndex` encoding (`(block<<8)|0x10` / `(page<<8)|address`),
translated from `USBDevice.controlTransferOut()` to
`usb_host_transfer_submit_control()` with a manually-built
`usb_setup_packet_t`.
4. **A basic bulk-transfer read loop structure** for pulling IQ sample
data off the device (`rtlsdr_start_bulk_streaming()`,
`rtlsdr_bulk_xfer_cb()`): unlike `sdr.ts` (which does one-shot
`await dev.transferIn(...)` calls from inside its own async `sweep()`
loop), the USB Host Library is callback-driven, so continuous
streaming here is a small pipeline — N transfers (`bulk_read_queue_
depth`, default 4) of a fixed chunk size (`bulk_read_chunk_bytes`,
default 16384B, enforced as a multiple of 512 the same way `sdr.ts`'s
`readSamples()` comment requires) are kept perpetually in flight; each
completion callback copies the received bytes into a heap block, hands
it to a FreeRTOS queue for the forward task, and immediately resubmits
itself to keep the pipe full. **This is the part most likely to need
real-hardware tuning** — see below.
5. **A stub/structure for forwarding raw IQ data upstream**
(`rtlsdr_exp_forward_iq_block()`, `rtlsdr_forward_task()`) — see the
architecture decision writeup immediately below.
### IQ-forwarding architecture decision
The spec explicitly leaves this as a product-judgment call ("your call on
whether this warrants a separate endpoint/stream vs. reusing the sensor
telemetry shape with a binary/base64 payload"). Decision made here:
**a separate binary stream/endpoint, not a reuse of the JSON telemetry
shape.** Reasoning:
- **Raw throughput is the wrong order of magnitude for the telemetry
contract.** Even a modest 2.048 Msps capture at 8 bits/sample/channel
(RTL2832U's native ADC format) is interleaved I/Q bytes at roughly
4.1 MB/s. The design spec's `POST /api/device/telemetry` contract caps
the `readings` array at ~64 entries, caps the request body at ~16KB,
and rate-limits ingestion to roughly 1 request/second sustained per
device — sized for a handful of scalar sensor readings (temperature,
humidity, presence booleans), not a continuous multi-megabyte/second
binary stream. Forcing IQ data through that shape would mean either
violating those caps (defeating their whole purpose — bounding what a
misbehaving/malicious device can push at the backend) or chopping IQ
into thousands of tiny requests per second, which is worse for both
sides than one continuous stream.
- **Base64-in-JSON adds ~33% size overhead** on top of a payload that's
already too large for the telemetry shape, for no benefit — there's no
reason to pay text-encoding tax on a payload nothing needs to
eyeball as text.
- **It's a fundamentally different kind of data with different backend
handling needs.** Telemetry readings feed the anomaly-detection
baseline/threshold pipeline directly and cheaply (a few floats per
sensor). IQ data needs FFT processing before it's useful for anything
— an entirely different backend code path (closer to how the browser's
own `powerSpectrumDb()` + `SpectrumAnomalyDetector` work today,
server-side instead of client-side). Conflating the two request shapes
would couple two things that should scale, rate-limit, and fail
independently.
Given that, this module's stub (`rtlsdr_exp_forward_iq_block()`) targets
a **separate, currently-hypothetical** endpoint (`iq_upload_url` in
`rtlsdr_exp_config_t`, suggested path `/api/device/iq-stream` in code
comments) carrying a small fixed binary header (magic, sequence number,
center frequency, sample rate, payload length — see
`rtlsdr_iq_chunk_header_t`) followed by the raw IQ bytes, POSTed as
`application/octet-stream` with the same `Authorization: Bearer <device
token>` auth as the regular telemetry loop. **This endpoint does not
exist anywhere in this repo.** Building it is explicitly out of scope for
this workstream (it's backend work, not firmware, and the spec doesn't
assign a backend workstream to receive IQ data at all — only to receive
scalar telemetry). The stub is inert by default (`rtlsdr_exp_forward_
iq_block()` returns `ESP_OK` immediately unless `iq_upload_url` is
configured) specifically so this module can be compiled/enabled for its
USB-host/bring-up behavior without requiring a backend that doesn't
exist.
A real product decision here — genuinely open, not resolved by this
workstream — is whether a POST-per-chunk model is even right versus a
persistent WebSocket stream (the spec's `/ws/device-feed` design already
establishes a WS pub/sub pattern on the backend for telemetry; a
`/ws/device-iq` sibling might fit that architecture better than repeated
HTTP POSTs, especially if backpressure/flow-control matters, which for a
continuous stream it very much does). That's flagged here rather than
silently decided, because it depends on backend design judgment as much
as firmware judgment.
### What's unverified — an honest, specific list for real hardware bring-up
Nothing below has run against real hardware. In rough order of "most
likely to break first":
1. **USB Host Library API surface.** Function names, struct field names,
and callback signatures (`usb_host_client_config_t`'s `.async.
client_event_callback` shape in particular — ESP-IDF has changed this
API's shape across versions) are written from documented/remembered
API shape, not checked against a real ESP-IDF checkout (none available
in this environment). **First thing a real bring-up needs: does this
even compile against the ESP-IDF version Workstream I's project
targets?**
2. **RTL2832U enumeration over a real ESP32-P4 USB-OTG host port** — does
`usb_host_device_open()` / `usb_host_get_device_descriptor()` actually
see the dongle at all on this specific chip's USB-OTG controller (power
delivery to the dongle over USB-host mode is itself a hardware
question — does the ESP32-P4 dev board supply VBUS in host mode, does
the dongle draw more current than it can supply).
3. **The init vendor-command sequence itself** — carried over unchanged
from `sdr.ts`, which is *itself* unverified. Two layers of "reasoned,
not tested." A logic analyzer / USB protocol analyzer trace against a
real dongle (or cross-checking against real librtlsdr `-T` verbose
output) is needed to confirm register values, not just transfer
plumbing.
4. **R820T PLL frequency math and demod resample-ratio math** — both are
the same simplified integer-N approximation `sdr.ts` uses (its own
comment: "real librtlsdr computes the exact sdm/vco from a 28.8MHz
crystal reference. Simplified... Marked for hardware."). Likely wrong
or imprecise until replaced with the real librtlsdr formula and
checked against an actual received signal.
5. **Bulk transfer chunk size, pipeline depth, and timeout=0 choice**
(`RTLSDR_EXP_BULK_CHUNK_BYTES`/`RTLSDR_EXP_BULK_QUEUE_DEPTH` Kconfig,
defaults 16384B / depth 4) — these are guesses. Real ESP32-P4 USB
Host Library heap/DMA limits, achievable sustained throughput at
2.048 Msps (~4.1 MB/s), and whether `timeout_ms = 0` (no timeout, i.e.
"block until data or disconnect") is even the right transfer mode for
this endpoint all need real measurement.
6. **Drop-on-full backpressure policy** (`iq_blocks_dropped` stat) — is
silently dropping IQ blocks when the forward task falls behind
acceptable, or does it need real flow control (e.g. throttling the
bulk read rate itself)? Depends on (5) and on real WiFi uplink
bandwidth from a real device on a real seeker's home network.
7. **The proposed IQ-forwarding wire format and endpoint** — entirely
hypothetical (see architecture section above); no backend exists to
validate the framing against, and the POST-vs-WebSocket question above
is unresolved.
8. **Concurrent USB Host Library ownership with the rest of the firmware
project.** `rtlsdr_exp_start()`/`rtlsdr_exp_stop()` call `usb_host_
install()`/`usb_host_uninstall()` directly, assuming this module is the
*only* USB Host client in the project. If Workstream I's skeleton (or
anything else) also needs USB host for something, this needs to change
to a shared-ownership model (install once, both modules register as
clients) — impossible to resolve without seeing that code, flagged
here for whoever does the merge.
9. **Memory footprint.** Multiple in-flight 16KB bulk transfer buffers
plus a forward queue plus `esp_http_client` buffers, alongside whatever
Workstream I's WiFi/HTTP/BME280/presence stack already needs, on a
single ESP32-P4's RAM — not sized or measured against a real linker
map.
### Build / integration notes
This component is **off by default** (`RTLSDR_EXP_ENABLE` Kconfig option,
default `n`) so it cannot affect Workstream I's core build unless
explicitly turned on via `idf.py menuconfig` → "RTL-SDR Experimental
Module." To include it in a real project once Workstream I's skeleton
exists:
1. Add this directory's `components/` to the consuming project's
`EXTRA_COMPONENT_DIRS` in the top-level `CMakeLists.txt` (or copy
`components/rtlsdr_experimental/` into that project's own
`components/`).
2. From wherever `app_main()` sets up its other sensor drivers, e.g.:
```c
#include "rtlsdr_experimental.h"
rtlsdr_exp_config_t sdr_cfg;
rtlsdr_exp_config_default(&sdr_cfg);
sdr_cfg.iq_upload_url = NULL; /* leave unset until a real backend
endpoint exists — see README */
sdr_cfg.device_bearer_token = DEVICE_BEARER_TOKEN; /* from
main/device_config.h, same token
used for the telemetry POST loop */
rtlsdr_exp_handle_t sdr_handle;
if (rtlsdr_exp_init(&sdr_cfg, &sdr_handle) == ESP_OK) {
rtlsdr_exp_start(sdr_handle); /* non-blocking; watches for a dongle */
}
```
3. `idf.py build` — **not run in this environment** (no ESP-IDF
toolchain available); this module has only been reasoned about, not
compiled. Treat "compiles" as an open question for the next person
with a real toolchain, not a claim made here.
Verified in this environment: none of it, against hardware or a real
compiler. What *has* been done: careful structural translation of a
real, already-partially-researched protocol (`sdr.ts`) to the documented
shape of a real, chip-specific API (ESP32-P4's USB Host Library), with
every simplification and open question called out rather than hidden.

17
firmware/esp32p4-sensor-node/components/rtlsdr_experimental/CMakeLists.txt Normal file
View File

@@ -0,0 +1,17 @@
# rtlsdr_experimental — EXPERIMENTAL, UNVERIFIED AGAINST REAL HARDWARE.
# See include/rtlsdr_experimental.h and ../../README.md ("Workstream J")
# for the full honesty write-up.
#
# This is a self-contained ESP-IDF component so it can be dropped into
# Workstream I's core firmware project (firmware/esp32p4-sensor-node/) via
# EXTRA_COMPONENT_DIRS, or by copying this directory under that project's
# own components/, without editing any file that workstream owns.
#
# `usb` = ESP-IDF's USB Host Library (usb_host.h) — native to ESP32-P4's
# USB-OTG host controller. `esp_http_client` = the upstream IQ-forwarding
# stub's transport (see rtlsdr_experimental.c "Upstream IQ forwarding").
idf_component_register(
SRCS "rtlsdr_experimental.c"
INCLUDE_DIRS "include"
REQUIRES usb esp_http_client
)

35
firmware/esp32p4-sensor-node/components/rtlsdr_experimental/Kconfig Normal file
View File

@@ -0,0 +1,35 @@
menu "RTL-SDR Experimental Module (Workstream J — UNVERIFIED)"
config RTLSDR_EXP_ENABLE
bool "Enable RTL-SDR experimental USB-host module"
default n
help
Attempts to initialize the ESP32-P4 USB Host Library and probe
for an attached RTL2832U-based dongle. This module is
experimental and UNVERIFIED against real hardware — see
firmware/esp32p4-sensor-node/README.md, "Workstream J" section,
before enabling on a device you care about. Off by default so
it can never interfere with the core sensor node build unless
explicitly turned on.
config RTLSDR_EXP_BULK_CHUNK_BYTES
int "Bulk IQ read chunk size (bytes, must be a multiple of 512)"
default 16384
depends on RTLSDR_EXP_ENABLE
help
UNVERIFIED default. Real achievable USB bulk throughput on
ESP32-P4 (and the right tradeoff against available heap/DMA
memory) is one of the concrete things a real hardware bring-up
session needs to measure and tune.
config RTLSDR_EXP_BULK_QUEUE_DEPTH
int "In-flight bulk transfer pipeline depth"
default 4
range 1 8
depends on RTLSDR_EXP_ENABLE
help
Number of bulk-IN transfers kept perpetually submitted so the
USB pipe doesn't idle. UNVERIFIED — needs tuning against real
measured throughput vs. available RAM.
endmenu

151
firmware/esp32p4-sensor-node/components/rtlsdr_experimental/include/rtlsdr_experimental.h Normal file
View File

@@ -0,0 +1,151 @@
/*
* rtlsdr_experimental.h — EXPERIMENTAL RTL2832U-over-USB-Host module for the
* ESP32-P4 sensor node (Workstream J, ESP32-P4 Sensor Node spec).
*
* ============================================================================
* HARDWARE PASS REQUIRED — UNVERIFIED AGAINST REAL HARDWARE
* ============================================================================
* This entire module (this header + rtlsdr_experimental.c) has been written
* from documented ESP-IDF USB Host Library API shape and the RTL2832U/R820T
* register sequence already researched for this project in
* frontend/src/lib/sdr.ts (a WebUSB driver, real and documented against
* librtlsdr's public register map). It has NOT been compiled against a real
* ESP-IDF toolchain (none is available in this environment) and has NOT run
* against real ESP32-P4 + RTL2832U dongle hardware. Treat every register
* value, timing constant, buffer size, and API call signature here as
* "structurally reasoned, not verified." See the "Workstream J" section of
* ../../README.md for the full honesty write-up, including exactly what a
* real hardware bring-up session needs to check before this is trusted.
*
* Scope reminder: this is a self-contained ESP-IDF *component*, deliberately
* kept out of `main/` so it can be dropped into Workstream I's firmware
* project skeleton (WiFi/HTTP/BME280/presence-sensor core) without touching
* any file that workstream owns. Integration is one line in the consuming
* project: add this directory to EXTRA_COMPONENT_DIRS (or copy it under that
* project's own components/) and call rtlsdr_exp_init()/rtlsdr_exp_start()
* from wherever app_main() sets up its other sensor drivers.
* ============================================================================
*/
#pragma once
#include <stdint.h>
#include <stdbool.h>
#include <stddef.h>
#include "esp_err.h"
#ifdef __cplusplus
extern "C" {
#endif
/* ---------------------------------------------------------------------------
* USB identification — ported 1:1 from frontend/src/lib/sdr.ts
* (RTL2832U_VENDOR / RTL2832U_PRODUCTS / TERRATEC_VENDOR, lines ~30-34).
*
* sdr.ts deliberately filters WebUSB's device picker by *vendor* id only
* (see its requestDevice() comment: "many dongles report product ids
* outside the handful we know, so filtering by productId hides them from
* the picker"). We mirror that: RTLSDR_EXP_KNOWN_PRODUCT_IDS below is
* informational only (used for a startup log line), never a hard match
* filter. Any device presenting vendor id RTL2832U or TERRATEC is treated
* as a candidate and bring-up is attempted.
* ------------------------------------------------------------------------ */
#define RTLSDR_EXP_VENDOR_RTL2832U 0x0BDAu
#define RTLSDR_EXP_VENDOR_TERRATEC 0x0CCDu
/* Known RTL2832U product ids (informational/logging only — see above). */
extern const uint16_t RTLSDR_EXP_KNOWN_PRODUCT_IDS[4];
#define RTLSDR_EXP_NUM_KNOWN_PRODUCT_IDS 4
/* ---------------------------------------------------------------------------
* Configuration
* ------------------------------------------------------------------------ */
typedef struct {
/* Tuning defaults, applied once bring-up completes. Matches sdr.ts's
* open(sampleRateHz = 2_048_000) default and its FM-band test tune of
* 98 MHz — pick whatever the product actually wants to listen to. */
uint32_t center_freq_hz; /* e.g. 98_000_000 */
uint32_t sample_rate_hz; /* e.g. 2_048_000 */
/* Bulk-IN read tuning. bulk_read_chunk_bytes MUST be a multiple of 512
* (USB high-speed bulk max packet size) — this mirrors sdr.ts's own
* assumption ("Length must be multiple of 512" on readSamples()).
* UNVERIFIED: real chunk size vs. USB Host Library heap/DMA limits and
* actual achievable throughput needs a real device. See README. */
size_t bulk_read_chunk_bytes; /* default suggestion: 16384 */
size_t bulk_read_queue_depth; /* in-flight pipelined transfers, e.g. 4 */
/* Upstream IQ forwarding (see rtlsdr_exp_forward_iq_block() and the
* "IQ forwarding architecture" section of the README for the reasoning
* behind a *separate* raw-binary stream rather than reusing the JSON
* sensor telemetry endpoint). NOTE: the backend endpoint referenced
* here does not exist yet anywhere in this repo as of this workstream —
* this is a client-side stub against a *proposed* shape, not a wired
* integration. */
const char *iq_upload_url; /* e.g. "https://host/api/device/iq-stream" */
const char *device_bearer_token; /* same raw pairing token used for
* the regular telemetry POST loop */
} rtlsdr_exp_config_t;
/* Fills in documented, conservative defaults. Caller still MUST set
* iq_upload_url / device_bearer_token before starting. */
void rtlsdr_exp_config_default(rtlsdr_exp_config_t *config);
typedef struct rtlsdr_exp_handle_s *rtlsdr_exp_handle_t;
/* ---------------------------------------------------------------------------
* Lifecycle
* ------------------------------------------------------------------------ */
/* Allocates internal state. Does NOT touch USB hardware yet. */
esp_err_t rtlsdr_exp_init(const rtlsdr_exp_config_t *config, rtlsdr_exp_handle_t *out_handle);
esp_err_t rtlsdr_exp_deinit(rtlsdr_exp_handle_t handle);
/* Installs the USB Host Library, registers a client, and spawns the
* background tasks that watch for a matching RTL2832U device, run the
* init vendor-command sequence on attach, and (once bring-up succeeds)
* start the bulk-IN read/forward loop. Non-blocking — returns once the
* tasks are created, not once a device is found.
*
* UNVERIFIED beyond "this is the documented shape of the ESP-IDF USB Host
* Library API" — see README for exactly what needs a real board to check. */
esp_err_t rtlsdr_exp_start(rtlsdr_exp_handle_t handle);
/* Stops streaming, releases the USB interface/device if held, deregisters
* the USB Host client, and tears down the background tasks. Does NOT call
* usb_host_uninstall() if other USB Host clients might still be active in
* the wider firmware project (Workstream I owns whether anything else in
* the project also uses USB host) — see README integration notes. */
esp_err_t rtlsdr_exp_stop(rtlsdr_exp_handle_t handle);
/* Best-effort retune while streaming — same caveats as sdr.ts's
* setFrequency()/setSampleRate() (simplified PLL/resampler math, marked
* HARDWARE PASS REQUIRED there too). */
esp_err_t rtlsdr_exp_set_frequency(rtlsdr_exp_handle_t handle, uint32_t hz);
esp_err_t rtlsdr_exp_set_sample_rate(rtlsdr_exp_handle_t handle, uint32_t hz);
/* ---------------------------------------------------------------------------
* Observability — deliberately exposed so this experimental module can
* report its own health (e.g. folded into the regular sensor_type registry
* as a synthetic "sdr_status" reading) rather than fail silently.
* ------------------------------------------------------------------------ */
typedef struct {
bool device_present; /* a matching-vendor USB device is enumerated */
bool bring_up_ok; /* init vendor-command sequence completed without
* a control-transfer error (does NOT mean the
* tuner/demod are actually producing valid IQ —
* only real hardware + a spectrum check can
* confirm that) */
bool streaming;
uint32_t bulk_reads_ok;
uint32_t bulk_reads_failed;
uint32_t iq_blocks_dropped; /* forward queue was full — see README */
uint32_t bytes_forwarded_upstream;
uint32_t forward_failures;
} rtlsdr_exp_stats_t;
void rtlsdr_exp_get_stats(rtlsdr_exp_handle_t handle, rtlsdr_exp_stats_t *out_stats);
#ifdef __cplusplus
}
#endif

913
firmware/esp32p4-sensor-node/components/rtlsdr_experimental/rtlsdr_experimental.c Normal file
View File

@@ -0,0 +1,913 @@
/*
* rtlsdr_experimental.c — EXPERIMENTAL RTL2832U-over-USB-Host module.
*
* ============================================================================
* HARDWARE PASS REQUIRED — UNVERIFIED AGAINST REAL HARDWARE
* ============================================================================
* See rtlsdr_experimental.h for the full disclaimer and ../../README.md's
* "Workstream J" section for the complete honesty write-up. Short version:
* nobody working on this had physical ESP32-P4 or RTL2832U hardware, and
* this environment has no ESP-IDF toolchain to even compile against. Every
* function below is written to the *documented shape* of:
* (a) the ESP-IDF USB Host Library API (usb_host.h) — install/client/
* transfer lifecycle, callback dispatch model, control vs. bulk
* transfer submission, config/interface descriptor parsing helpers;
* (b) the RTL2832U + R820T vendor-command sequence already researched
* and documented in frontend/src/lib/sdr.ts (WebUSB driver for this
* same chipset, against public librtlsdr register docs).
* It has not been built or run. Register pokes, timing, and buffer sizing
* are carried over from sdr.ts's own "HARDWARE PASS REQUIRED" markings
* with no changes beyond translating WebUSB JS calls to USB Host C calls.
* ============================================================================
*
* Architecture (see README for the fuller picture):
*
* usb_host_lib_daemon_task — pumps usb_host_lib_handle_events() forever.
* Required by the USB Host Library regardless
* of how many clients exist.
* rtlsdr_exp_client_task — registers as the one USB Host *client* this
* module needs, pumps usb_host_client_handle_
* events() (which is also how *this task's*
* control- and bulk-transfer completion
* callbacks get dispatched — the ESP-IDF USB
* Host Library calls transfer callbacks
* synchronously from inside whichever task
* called *_handle_events(), not from an ISR
* or a hidden thread). On NEW_DEV it attempts
* bring-up; on DEV_GONE it tears down.
* rtlsdr_exp_forward_task — drains the IQ block queue that the bulk
* transfer callback feeds and hands each
* block to rtlsdr_exp_forward_iq_block()
* (the upstream-forwarding STUB).
*
* Bulk IQ reads are pipelined: `bulk_read_queue_depth` transfers are kept
* perpetually in flight (each callback re-submits itself), so the USB pipe
* doesn't idle waiting for the forward task to catch up. Completed buffers
* are handed off via a FreeRTOS queue; if the forward task falls behind,
* blocks are DROPPED (counted in stats.iq_blocks_dropped) rather than
* blocking the USB callback — losing samples is preferable to stalling the
* USB Host Library's event dispatch, which would also stall the client's
* disconnect detection. Whether this is an acceptable loss policy for the
* real product is exactly the kind of thing that needs real-hardware/real-
* throughput testing — see README.
*/
#include <string.h>
#include <stdlib.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/queue.h"
#include "esp_log.h"
#include "esp_check.h"
#include "usb/usb_host.h"
#include "usb/usb_helpers.h"
#include "usb/usb_types_ch9.h"
#include "esp_http_client.h"
#include "rtlsdr_experimental.h"
static const char *TAG = "rtlsdr_exp";
const uint16_t RTLSDR_EXP_KNOWN_PRODUCT_IDS[4] = { 0x2832, 0x2834, 0x2838, 0x2837 };
/* ---------------------------------------------------------------------------
* Vendor-command constants — ported verbatim from frontend/src/lib/sdr.ts
* (CTRL_IN / CTRL_OUT / DEMOD / USB_EPA / SYS / PAGE_USB, lines ~37-42).
* ------------------------------------------------------------------------ */
#define RTLSDR_CTRL_IN 0xC0u /* bmRequestType: IN | vendor | device */
#define RTLSDR_CTRL_OUT 0x40u /* bmRequestType: OUT | vendor | device */
#define RTLSDR_BLOCK_DEMOD 0x03u
#define RTLSDR_BLOCK_USB_EPA 0x02u
#define RTLSDR_BLOCK_SYS 0x09u
#define RTLSDR_PAGE_USB 0x01u
/* Default tuning + read-loop parameters (rtlsdr_exp_config_default). These
* mirror sdr.ts's open(sampleRateHz = 2_048_000) and its 98 MHz test tune;
* bulk_read_chunk_bytes/queue_depth are new (sdr.ts's readSamples() takes
* an explicit byte count per call from its sweep() caller, there is no
* fixed default there to inherit) and are UNVERIFIED guesses pending real
* throughput measurement. */
#define RTLSDR_EXP_DEFAULT_CENTER_HZ 98000000u
#define RTLSDR_EXP_DEFAULT_SAMPLE_RATE 2048000u
#define RTLSDR_EXP_DEFAULT_CHUNK_BYTES 16384u /* 32 * 512, USB HS multiple */
#define RTLSDR_EXP_DEFAULT_QUEUE_DEPTH 4u
#define RTLSDR_EXP_FORWARD_QUEUE_LEN 8u /* IQ blocks buffered for forward task */
#define RTLSDR_EXP_CTRL_XFER_TIMEOUT_MS 1000u
#define RTLSDR_EXP_USB_INTERFACE_NUM 0u
struct rtlsdr_exp_handle_s {
rtlsdr_exp_config_t cfg;
usb_host_client_handle_t client_hdl;
usb_device_handle_t dev_hdl;
bool device_open;
bool interface_claimed;
uint8_t ep_in_addr; /* bulk IN endpoint address, once found */
/* pending work handed from the client-event callback (kept short, per
* ESP-IDF USB Host guidance) to the client task's main loop */
volatile uint8_t pending_new_dev_addr; /* 0 == none pending */
volatile bool pending_dev_gone;
volatile bool running; /* set false by rtlsdr_exp_stop() */
volatile bool streaming; /* bulk read loop active */
TaskHandle_t lib_task;
TaskHandle_t client_task;
TaskHandle_t forward_task;
QueueHandle_t iq_block_queue; /* holds rtlsdr_iq_block_t* */
/* control-transfer synchronous-wait bookkeeping. Safe as plain fields
* (not a semaphore) because control transfers issued by this module are
* only ever awaited from the same task that also pumps
* usb_host_client_handle_events() — see file header. */
volatile bool ctrl_xfer_pending;
volatile bool ctrl_xfer_ok;
usb_transfer_t *bulk_transfers[8]; /* sized to the max we allow bulk_read_queue_depth to be */
rtlsdr_exp_stats_t stats;
};
typedef struct {
uint8_t *data;
size_t len;
} rtlsdr_iq_block_t;
/* ---------------------------------------------------------------------------
* Public: config defaults
* ------------------------------------------------------------------------ */
void rtlsdr_exp_config_default(rtlsdr_exp_config_t *config)
{
if (!config) return;
memset(config, 0, sizeof(*config));
config->center_freq_hz = RTLSDR_EXP_DEFAULT_CENTER_HZ;
config->sample_rate_hz = RTLSDR_EXP_DEFAULT_SAMPLE_RATE;
config->bulk_read_chunk_bytes = RTLSDR_EXP_DEFAULT_CHUNK_BYTES;
config->bulk_read_queue_depth = RTLSDR_EXP_DEFAULT_QUEUE_DEPTH;
config->iq_upload_url = NULL; /* caller must set */
config->device_bearer_token = NULL; /* caller must set */
}
/* ---------------------------------------------------------------------------
* Low-level control-transfer helpers.
* Direct port of sdr.ts's private writeReg()/demodWrite()/i2cWrite()
* (lines ~252-303). Same wValue/wIndex encoding, same register addresses,
* same repeater on/off dance for tuner I2C access. The WebUSB calls
*
* dev.controlTransferOut({ requestType: 'vendor', recipient: 'device',
* request: 0, value: ..., index: ... }, data)
*
* become a manually-built usb_setup_packet_t with bmRequestType =
* RTLSDR_CTRL_OUT (0x40 — OUT | vendor | device, matching WebUSB's
* 'vendor'/'device' fields), bRequest = 0, wValue/wIndex as below, wLength
* = payload length, submitted via usb_host_transfer_submit_control().
* ------------------------------------------------------------------------ */
/* Synchronously wait for a previously-submitted control transfer to
* complete by pumping usb_host_client_handle_events() from THIS task (see
* file header for why that's safe/required here). Returns ESP_OK if the
* transfer completed successfully within the timeout. */
static esp_err_t rtlsdr_wait_ctrl_xfer(rtlsdr_exp_handle_t h)
{
TickType_t start = xTaskGetTickCount();
TickType_t timeout_ticks = pdMS_TO_TICKS(RTLSDR_EXP_CTRL_XFER_TIMEOUT_MS);
while (h->ctrl_xfer_pending) {
usb_host_client_handle_events(h->client_hdl, pdMS_TO_TICKS(10));
if ((xTaskGetTickCount() - start) > timeout_ticks) {
ESP_LOGW(TAG, "control transfer timed out");
h->ctrl_xfer_pending = false;
return ESP_ERR_TIMEOUT;
}
}
return h->ctrl_xfer_ok ? ESP_OK : ESP_FAIL;
}
static void rtlsdr_ctrl_xfer_cb(usb_transfer_t *transfer)
{
rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)transfer->context;
h->ctrl_xfer_ok = (transfer->status == USB_TRANSFER_STATUS_COMPLETED);
h->ctrl_xfer_pending = false;
usb_host_transfer_free(transfer);
}
/* value/index encoding matches sdr.ts's writeReg()/demodWrite() exactly:
* writeReg: wValue = (block<<8)|0x10, wIndex = address
* demodWrite: wValue = (DEMOD<<8)|0x10, wIndex = (page<<8)|address
* `length` is 1 or 2 bytes of little-endian `value`, same as sdr.ts. */
static esp_err_t rtlsdr_ctrl_write_raw(rtlsdr_exp_handle_t h, uint16_t wValue,
uint16_t wIndex, uint16_t value, uint8_t length)
{
if (!h->dev_hdl) return ESP_ERR_INVALID_STATE;
usb_transfer_t *transfer = NULL;
esp_err_t err = usb_host_transfer_alloc(sizeof(usb_setup_packet_t) + length, 0, &transfer);
if (err != ESP_OK) return err;
usb_setup_packet_t *setup = (usb_setup_packet_t *)transfer->data_buffer;
setup->bmRequestType = RTLSDR_CTRL_OUT;
setup->bRequest = 0;
setup->wValue = wValue;
setup->wIndex = wIndex;
setup->wLength = length;
uint8_t *payload = transfer->data_buffer + sizeof(usb_setup_packet_t);
payload[0] = (uint8_t)(value & 0xFF);
if (length > 1) payload[1] = (uint8_t)((value >> 8) & 0xFF);
transfer->device_handle = h->dev_hdl;
transfer->bEndpointAddress = 0; /* control endpoint */
transfer->num_bytes = sizeof(usb_setup_packet_t) + length;
transfer->callback = rtlsdr_ctrl_xfer_cb;
transfer->context = h;
transfer->timeout_ms = RTLSDR_EXP_CTRL_XFER_TIMEOUT_MS;
h->ctrl_xfer_pending = true;
h->ctrl_xfer_ok = false;
err = usb_host_transfer_submit_control(h->client_hdl, transfer);
if (err != ESP_OK) {
h->ctrl_xfer_pending = false;
usb_host_transfer_free(transfer);
return err;
}
return rtlsdr_wait_ctrl_xfer(h);
}
/* writeReg() equivalent — sdr.ts private writeReg(block, address, value, length) */
static esp_err_t rtlsdr_reg_write(rtlsdr_exp_handle_t h, uint8_t block, uint16_t address,
uint16_t value, uint8_t length)
{
uint16_t wValue = (uint16_t)((block << 8) | 0x10);
return rtlsdr_ctrl_write_raw(h, wValue, address, value, length);
}
/* demodWrite() equivalent — sdr.ts private demodWrite(page, address, value, length) */
static esp_err_t rtlsdr_demod_write(rtlsdr_exp_handle_t h, uint8_t page, uint16_t address,
uint16_t value, uint8_t length)
{
uint16_t wValue = (uint16_t)((RTLSDR_BLOCK_DEMOD << 8) | 0x10);
uint16_t wIndex = (uint16_t)((page << 8) | address);
return rtlsdr_ctrl_write_raw(h, wValue, wIndex, value, length);
}
/* i2cWrite() equivalent — sdr.ts private i2cWrite(i2cAddr, reg, value).
* Tunnels a single-byte tuner register write through the demod's I2C
* repeater: repeater on (demod 1/0x02 = 0x41) -> vendor write addressed to
* (i2cAddr<<8)|reg -> repeater off (demod 1/0x02 = 0x01). */
static esp_err_t rtlsdr_i2c_write(rtlsdr_exp_handle_t h, uint8_t i2c_addr, uint8_t reg, uint8_t value)
{
esp_err_t err = rtlsdr_demod_write(h, 1, 0x02, 0x41, 1); /* repeater on */
if (err != ESP_OK) return err;
uint16_t wValue = (uint16_t)((0x02 << 8) | 0x10);
uint16_t wIndex = (uint16_t)((i2c_addr << 8) | reg);
err = rtlsdr_ctrl_write_raw(h, wValue, wIndex, value, 1);
esp_err_t err2 = rtlsdr_demod_write(h, 1, 0x02, 0x01, 1); /* repeater off, always attempted */
return (err != ESP_OK) ? err : err2;
}
/* ---------------------------------------------------------------------------
* Tuning — port of sdr.ts's setFrequency()/setSampleRate() (lines
* ~156-180). Same simplifications, same "HARDWARE PASS REQUIRED" caveat:
* real librtlsdr computes exact sdm/vco from the crystal reference; this
* is the same reduced integer-N approximation sdr.ts uses, carried over
* unchanged rather than re-derived (see this project's Honesty-policy note
* — the browser driver's math is the primary reference, not a place to
* invent new, unverified math on top of already-unverified math).
* ------------------------------------------------------------------------ */
esp_err_t rtlsdr_exp_set_frequency(rtlsdr_exp_handle_t h, uint32_t hz)
{
if (!h || !h->dev_hdl) return ESP_ERR_INVALID_STATE;
uint32_t lo_hz = hz + 3570000u; /* R820T IF offset */
uint32_t ref = 28800000u;
uint32_t mix_div = 2u;
uint32_t nint = lo_hz / (ref * mix_div);
uint32_t vco = lo_hz % (ref * mix_div);
uint32_t sdm = (uint32_t)(((uint64_t)vco * 65536u) / (ref * mix_div));
if (sdm > 0xFFFFu) sdm = 0xFFFFu;
uint8_t reg = (uint8_t)(nint & 0x3F);
esp_err_t err;
ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1A, 0x10, reg)), TAG, "pll nint");
ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1A, 0x11, (sdm >> 8) & 0xFF)), TAG, "pll sdm hi");
ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1A, 0x12, sdm & 0xFF)), TAG, "pll sdm lo");
h->cfg.center_freq_hz = hz;
return ESP_OK;
}
esp_err_t rtlsdr_exp_set_sample_rate(rtlsdr_exp_handle_t h, uint32_t hz)
{
if (!h || !h->dev_hdl) return ESP_ERR_INVALID_STATE;
uint32_t crystal = 28800000u;
uint32_t rsamp_ratio = (uint32_t)((((uint64_t)crystal << 22) / hz)) & 0x0FFFFFFCu;
esp_err_t err;
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x9f, (rsamp_ratio >> 16) & 0xFFFF, 2)), TAG, "rsamp hi");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0xa1, rsamp_ratio & 0xFFFF, 2)), TAG, "rsamp lo");
h->cfg.sample_rate_hz = hz;
return ESP_OK;
}
/* ---------------------------------------------------------------------------
* Init sequence — port of sdr.ts's open() body (lines ~110-140): demod
* soft reset -> demod_ctl/suspend-off register block -> standby off -> AGC
* mode -> R820T tuner power-up over the I2C repeater -> sample rate ->
* initial tune -> streaming endpoint reset -> test-mode off. Same order,
* same register addresses/values, unchanged.
* ------------------------------------------------------------------------ */
static esp_err_t rtlsdr_run_init_sequence(rtlsdr_exp_handle_t h)
{
esp_err_t err;
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x01, 0x14, 1)), TAG, "soft reset"); /* soft reset */
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x01, 0x10, 1)), TAG, "reset clear");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 0, 0x01, 0x08, 2)), TAG, "demod_ctl");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 0, 0x06, 0x80, 1)), TAG, "demod_ctl2");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x15, 0x00, 1)), TAG, "suspend off 0x15");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x16, 0x00, 1)), TAG, "suspend off 0x16");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x17, 0x00, 1)), TAG, "suspend off 0x17");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x18, 0x00, 1)), TAG, "suspend off 0x18");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x19, 0x00, 1)), TAG, "suspend off 0x19");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x1a, 0x00, 1)), TAG, "suspend off 0x1a");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x1b, 0x00, 1)), TAG, "suspend off 0x1b");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x1c, 0x00, 1)), TAG, "suspend off 0x1c");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x0d, 0x83, 1)), TAG, "standby off");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x0b, 0x1b, 1)), TAG, "AGC mode");
/* R820T power-up through I2C repeater (addr 0x1a) */
ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1a, 0x05, 0x8f)), TAG, "LNA power");
ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1a, 0x08, 0x80)), TAG, "mixer");
ESP_RETURN_ON_ERROR((err = rtlsdr_i2c_write(h, 0x1a, 0x0a, 0x10)), TAG, "IF filter");
ESP_RETURN_ON_ERROR((err = rtlsdr_exp_set_sample_rate(h, h->cfg.sample_rate_hz)), TAG, "sample rate");
ESP_RETURN_ON_ERROR((err = rtlsdr_exp_set_frequency(h, h->cfg.center_freq_hz)), TAG, "frequency");
/* Reset streaming endpoint before bulk reads begin. */
ESP_RETURN_ON_ERROR((err = rtlsdr_reg_write(h, RTLSDR_BLOCK_USB_EPA, 0x0001, 0xFFFF, 2)), TAG, "ep reset");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x02, 0x00, 1)), TAG, "streaming pre");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 0, 0x02, 0x40, 2)), TAG, "streaming enable");
ESP_RETURN_ON_ERROR((err = rtlsdr_demod_write(h, 1, 0x02, 0x00, 1)), TAG, "test mode off");
return ESP_OK;
}
/* ---------------------------------------------------------------------------
* Bulk-IN read loop — NOT present in sdr.ts in this pipelined form (the
* WebUSB driver does one-shot `await dev.transferIn(...)` calls from
* inside its own async sweep loop; ESP-IDF's USB Host Library is callback-
* driven and async by design, so continuous streaming needs an explicit
* resubmit-on-completion pipeline instead). This is the least-verified
* part of this whole module — see README "What's unverified" — because
* correct chunk size, queue depth, and drop policy all depend on real
* measured USB throughput on real ESP32-P4 silicon, which nobody involved
* in this workstream has access to.
* ------------------------------------------------------------------------ */
static void rtlsdr_bulk_xfer_cb(usb_transfer_t *transfer)
{
rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)transfer->context;
if (transfer->status == USB_TRANSFER_STATUS_COMPLETED && transfer->actual_num_bytes > 0) {
h->stats.bulk_reads_ok++;
rtlsdr_iq_block_t *block = malloc(sizeof(rtlsdr_iq_block_t));
uint8_t *copy = block ? malloc(transfer->actual_num_bytes) : NULL;
if (block && copy) {
memcpy(copy, transfer->data_buffer, transfer->actual_num_bytes);
block->data = copy;
block->len = (size_t)transfer->actual_num_bytes;
/* Non-blocking send: dropping is preferred to stalling the USB
* callback path (see file header). Drop is counted, not silent. */
if (h->iq_block_queue && xQueueSend(h->iq_block_queue, &block, 0) != pdTRUE) {
h->stats.iq_blocks_dropped++;
free(copy);
free(block);
}
} else {
h->stats.iq_blocks_dropped++;
free(copy);
free(block);
}
} else if (transfer->status != USB_TRANSFER_STATUS_COMPLETED) {
h->stats.bulk_reads_failed++;
ESP_LOGW(TAG, "bulk IN transfer failed, status=%d", transfer->status);
}
/* Keep the pipe full: resubmit immediately unless we're stopping. */
if (h->streaming && h->running) {
esp_err_t err = usb_host_transfer_submit(transfer);
if (err != ESP_OK) {
ESP_LOGW(TAG, "failed to resubmit bulk transfer: %d", err);
h->stats.bulk_reads_failed++;
}
}
}
static esp_err_t rtlsdr_start_bulk_streaming(rtlsdr_exp_handle_t h)
{
size_t depth = h->cfg.bulk_read_queue_depth;
if (depth == 0 || depth > (sizeof(h->bulk_transfers) / sizeof(h->bulk_transfers[0]))) {
depth = RTLSDR_EXP_DEFAULT_QUEUE_DEPTH;
}
if (h->cfg.bulk_read_chunk_bytes == 0 || (h->cfg.bulk_read_chunk_bytes % 512) != 0) {
ESP_LOGW(TAG, "bulk_read_chunk_bytes not a multiple of 512, forcing default");
h->cfg.bulk_read_chunk_bytes = RTLSDR_EXP_DEFAULT_CHUNK_BYTES;
}
for (size_t i = 0; i < depth; i++) {
usb_transfer_t *t = NULL;
esp_err_t err = usb_host_transfer_alloc(h->cfg.bulk_read_chunk_bytes, 0, &t);
if (err != ESP_OK) {
ESP_LOGE(TAG, "failed to allocate bulk transfer %zu: %d", i, err);
return err;
}
t->device_handle = h->dev_hdl;
t->bEndpointAddress = h->ep_in_addr;
t->num_bytes = h->cfg.bulk_read_chunk_bytes;
t->callback = rtlsdr_bulk_xfer_cb;
t->context = h;
t->timeout_ms = 0; /* no timeout: this endpoint is expected to be
* continuously producing once streaming is on;
* UNVERIFIED whether 0 (no timeout) is the
* right choice vs. a bounded timeout + retry —
* needs real-hardware behavior to decide. */
h->bulk_transfers[i] = t;
}
h->streaming = true;
for (size_t i = 0; i < depth; i++) {
esp_err_t err = usb_host_transfer_submit(h->bulk_transfers[i]);
if (err != ESP_OK) {
ESP_LOGE(TAG, "failed to submit initial bulk transfer %zu: %d", i, err);
h->streaming = false;
return err;
}
}
ESP_LOGI(TAG, "bulk IQ streaming started: chunk=%zuB depth=%zu",
h->cfg.bulk_read_chunk_bytes, depth);
return ESP_OK;
}
static void rtlsdr_stop_bulk_streaming(rtlsdr_exp_handle_t h)
{
h->streaming = false;
for (size_t i = 0; i < sizeof(h->bulk_transfers) / sizeof(h->bulk_transfers[0]); i++) {
if (h->bulk_transfers[i]) {
usb_host_transfer_free(h->bulk_transfers[i]);
h->bulk_transfers[i] = NULL;
}
}
}
/* ---------------------------------------------------------------------------
* Upstream IQ forwarding — STUB. See README "IQ forwarding architecture
* decision" for the full reasoning. Summary: raw IQ at even a modest
* 2.048 Msps / 8-bit-per-sample-per-channel is ~4.1 MB/s, which is wildly
* incompatible with the regular `POST /api/device/telemetry` shape (per
* the ESP32-P4 Sensor Node spec's contract: readings array capped at 64
* entries, request body capped at 16KB, rate-limited to ~1 req/sec
* sustained per device) — that endpoint is sized for a handful of scalar
* sensor readings, not a continuous binary stream. Rather than distort
* that endpoint's shape (e.g. base64-stuffing IQ bytes into a JSON
* "reading" — which would also add ~33% overhead on top of an already
* too-large payload), this stub targets a SEPARATE, not-yet-implemented
* endpoint carrying raw binary chunks, authenticated the same way (device
* bearer token) but outside the JSON telemetry contract entirely.
*
* This function is intentionally inert by default (returns early unless
* iq_upload_url is set) because:
* 1. No backend endpoint for this exists anywhere in this repo yet —
* building one is explicitly out of scope for this workstream.
* 2. Even the *shape* proposed here (raw octet-stream POST per block,
* small fixed binary header) is a guess pending: (a) real measured
* USB throughput off actual hardware, (b) a product decision on
* whether continuous IQ upload is even desired given typical
* home-WiFi-uplink bandwidth, and (c) whether a WebSocket stream
* would suit the backend's existing async pub/sub model (see the
* spec's /ws/device-feed design) better than repeated POSTs.
* ------------------------------------------------------------------------ */
/* Minimal fixed binary header prepended to each forwarded chunk, so the
* (not-yet-existing) backend endpoint can frame chunks without needing
* JSON/base64 parsing on a hot path. All fields little-endian.
* UNVERIFIED / PROPOSED — not agreed with any backend implementation. */
typedef struct __attribute__((packed)) {
uint32_t magic; /* 'QMIQ' = 0x51 0x4D 0x49 0x51 */
uint32_t seq;
uint32_t center_hz;
uint32_t sample_rate_hz;
uint32_t payload_len;
} rtlsdr_iq_chunk_header_t;
#define RTLSDR_IQ_CHUNK_MAGIC 0x51494D51u /* "QMIQ" */
static esp_err_t rtlsdr_exp_forward_iq_block(rtlsdr_exp_handle_t h, const uint8_t *data, size_t len)
{
if (!h->cfg.iq_upload_url || !h->cfg.device_bearer_token) {
/* No upload target configured — this is the expected state until a
* real backend endpoint exists and a caller opts in. Not an error. */
return ESP_OK;
}
static uint32_t s_seq = 0;
rtlsdr_iq_chunk_header_t hdr = {
.magic = RTLSDR_IQ_CHUNK_MAGIC,
.seq = s_seq++,
.center_hz = h->cfg.center_freq_hz,
.sample_rate_hz = h->cfg.sample_rate_hz,
.payload_len = (uint32_t)len,
};
esp_http_client_config_t http_cfg = {
.url = h->cfg.iq_upload_url,
.method = HTTP_METHOD_POST,
.timeout_ms = 5000,
};
esp_http_client_handle_t client = esp_http_client_init(&http_cfg);
if (!client) return ESP_FAIL;
char auth_header[512];
snprintf(auth_header, sizeof(auth_header), "Bearer %s", h->cfg.device_bearer_token);
esp_http_client_set_header(client, "Authorization", auth_header);
esp_http_client_set_header(client, "Content-Type", "application/octet-stream");
esp_err_t err = esp_http_client_open(client, (int)(sizeof(hdr) + len));
if (err == ESP_OK) {
int written = esp_http_client_write(client, (const char *)&hdr, sizeof(hdr));
if (written == (int)sizeof(hdr)) {
written = esp_http_client_write(client, (const char *)data, (int)len);
}
if (written < 0) err = ESP_FAIL;
esp_http_client_fetch_headers(client);
int status = esp_http_client_get_status_code(client);
if (status < 200 || status >= 300) {
ESP_LOGW(TAG, "IQ upload got HTTP %d (endpoint likely doesn't exist yet)", status);
err = ESP_FAIL;
}
}
esp_http_client_close(client);
esp_http_client_cleanup(client);
if (err == ESP_OK) {
h->stats.bytes_forwarded_upstream += (uint32_t)len;
} else {
h->stats.forward_failures++;
}
return err;
}
static void rtlsdr_forward_task(void *arg)
{
rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)arg;
rtlsdr_iq_block_t *block = NULL;
while (h->running) {
if (xQueueReceive(h->iq_block_queue, &block, pdMS_TO_TICKS(500)) == pdTRUE) {
rtlsdr_exp_forward_iq_block(h, block->data, block->len);
free(block->data);
free(block);
block = NULL;
}
}
/* drain remaining queued blocks on shutdown without forwarding */
while (xQueueReceive(h->iq_block_queue, &block, 0) == pdTRUE) {
free(block->data);
free(block);
}
vTaskDelete(NULL);
}
/* ---------------------------------------------------------------------------
* Device bring-up
* ------------------------------------------------------------------------ */
static bool rtlsdr_vendor_id_matches(uint16_t vid)
{
return vid == RTLSDR_EXP_VENDOR_RTL2832U || vid == RTLSDR_EXP_VENDOR_TERRATEC;
}
static void rtlsdr_log_known_product_hint(uint16_t vid, uint16_t pid)
{
if (vid != RTLSDR_EXP_VENDOR_RTL2832U) return;
for (size_t i = 0; i < RTLSDR_EXP_NUM_KNOWN_PRODUCT_IDS; i++) {
if (RTLSDR_EXP_KNOWN_PRODUCT_IDS[i] == pid) return;
}
ESP_LOGI(TAG, "product id 0x%04x not in the known RTL2832U list — "
"attempting bring-up anyway (vendor-only match, same policy "
"as frontend/src/lib/sdr.ts's WebUSB device filter)", pid);
}
/* Finds the first bulk-IN endpoint on the device's active configuration's
* first interface. Mirrors sdr.ts's:
* iface = dev.configuration.interfaces[0]; alt = iface.alternates[0];
* ep = alt.endpoints.find(e => e.direction==='in' && e.type==='bulk') */
static esp_err_t rtlsdr_find_bulk_in_endpoint(usb_device_handle_t dev_hdl, uint8_t *out_ep)
{
const usb_config_desc_t *config_desc = NULL;
esp_err_t err = usb_host_get_active_config_descriptor(dev_hdl, &config_desc);
if (err != ESP_OK || !config_desc) return ESP_FAIL;
int offset = 0;
const usb_intf_desc_t *intf = usb_parse_interface_descriptor(config_desc, 0, 0, &offset);
if (!intf) return ESP_FAIL;
for (int i = 0; i < intf->bNumEndpoints; i++) {
int ep_offset = offset;
const usb_ep_desc_t *ep = usb_parse_endpoint_descriptor_by_index(
intf, i, config_desc->wTotalLength, &ep_offset);
if (!ep) continue;
bool is_in = (ep->bEndpointAddress & USB_B_ENDPOINT_ADDRESS_EP_DIR_MASK) != 0;
bool is_bulk = (ep->bmAttributes & USB_BM_ATTRIBUTES_XFERTYPE_MASK) == USB_BM_ATTRIBUTES_XFER_BULK;
if (is_in && is_bulk) {
*out_ep = ep->bEndpointAddress;
return ESP_OK;
}
}
return ESP_ERR_NOT_FOUND;
}
static void rtlsdr_teardown_device(rtlsdr_exp_handle_t h)
{
rtlsdr_stop_bulk_streaming(h);
if (h->interface_claimed) {
usb_host_interface_release(h->client_hdl, h->dev_hdl, RTLSDR_EXP_USB_INTERFACE_NUM);
h->interface_claimed = false;
}
if (h->device_open) {
usb_host_device_close(h->client_hdl, h->dev_hdl);
h->device_open = false;
}
h->dev_hdl = NULL;
h->stats.device_present = false;
h->stats.bring_up_ok = false;
h->stats.streaming = false;
}
static void rtlsdr_try_bring_up(rtlsdr_exp_handle_t h, uint8_t dev_addr)
{
esp_err_t err = usb_host_device_open(h->client_hdl, dev_addr, &h->dev_hdl);
if (err != ESP_OK) {
ESP_LOGW(TAG, "usb_host_device_open failed: %d", err);
return;
}
h->device_open = true;
const usb_device_desc_t *dev_desc = NULL;
err = usb_host_get_device_descriptor(h->dev_hdl, &dev_desc);
if (err != ESP_OK || !dev_desc) {
ESP_LOGW(TAG, "could not read device descriptor: %d", err);
rtlsdr_teardown_device(h);
return;
}
if (!rtlsdr_vendor_id_matches(dev_desc->idVendor)) {
ESP_LOGD(TAG, "vendor 0x%04x is not RTL2832U/Terratec, ignoring", dev_desc->idVendor);
rtlsdr_teardown_device(h);
return;
}
rtlsdr_log_known_product_hint(dev_desc->idVendor, dev_desc->idProduct);
ESP_LOGI(TAG, "candidate RTL2832U-class device: vid=0x%04x pid=0x%04x",
dev_desc->idVendor, dev_desc->idProduct);
err = usb_host_interface_claim(h->client_hdl, h->dev_hdl, RTLSDR_EXP_USB_INTERFACE_NUM, 0);
if (err != ESP_OK) {
ESP_LOGW(TAG, "could not claim interface %d: %d — device may be claimed by "
"another driver, or this isn't the RTL2832U bulk interface",
RTLSDR_EXP_USB_INTERFACE_NUM, err);
rtlsdr_teardown_device(h);
return;
}
h->interface_claimed = true;
err = rtlsdr_find_bulk_in_endpoint(h->dev_hdl, &h->ep_in_addr);
if (err != ESP_OK) {
ESP_LOGW(TAG, "no bulk IN endpoint found on interface %d", RTLSDR_EXP_USB_INTERFACE_NUM);
rtlsdr_teardown_device(h);
return;
}
h->stats.device_present = true;
err = rtlsdr_run_init_sequence(h);
if (err != ESP_OK) {
ESP_LOGE(TAG, "RTL2832U init vendor-command sequence failed at some step: %d "
"(UNVERIFIED sequence — see README, this is exactly the kind of "
"failure real hardware bring-up is expected to need to debug)", err);
rtlsdr_teardown_device(h);
return;
}
h->stats.bring_up_ok = true;
ESP_LOGI(TAG, "RTL2832U init sequence completed without a control-transfer error "
"(this does NOT confirm valid IQ is being produced — only a real "
"spectrum/logic-analyzer check against hardware can confirm that)");
err = rtlsdr_start_bulk_streaming(h);
if (err != ESP_OK) {
ESP_LOGE(TAG, "failed to start bulk IQ streaming: %d", err);
rtlsdr_teardown_device(h);
return;
}
h->stats.streaming = true;
}
/* ---------------------------------------------------------------------------
* USB Host Library plumbing: daemon task + client task + client event cb
* ------------------------------------------------------------------------ */
static void rtlsdr_usb_lib_daemon_task(void *arg)
{
rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)arg;
while (h->running) {
uint32_t event_flags = 0;
usb_host_lib_handle_events(pdMS_TO_TICKS(1000), &event_flags);
if (event_flags & USB_HOST_LIB_EVENT_FLAGS_NO_CLIENTS) {
ESP_LOGD(TAG, "USB host lib: no clients");
}
if (event_flags & USB_HOST_LIB_EVENT_FLAGS_ALL_FREE) {
ESP_LOGD(TAG, "USB host lib: all devices free");
}
}
vTaskDelete(NULL);
}
static void rtlsdr_client_event_cb(const usb_host_client_event_msg_t *event_msg, void *arg)
{
rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)arg;
/* Deliberately kept short — per ESP-IDF USB Host guidance, heavy work
* (device open, control transfers) is done back in the client task's
* main loop, not inside this callback. */
switch (event_msg->event) {
case USB_HOST_CLIENT_EVENT_NEW_DEV:
h->pending_new_dev_addr = event_msg->new_dev.address;
break;
case USB_HOST_CLIENT_EVENT_DEV_GONE:
h->pending_dev_gone = true;
break;
default:
break;
}
}
static void rtlsdr_exp_client_task(void *arg)
{
rtlsdr_exp_handle_t h = (rtlsdr_exp_handle_t)arg;
usb_host_client_config_t client_config = {
.is_synchronous = false,
.max_num_event_msg = 5,
.async = {
.client_event_callback = rtlsdr_client_event_cb,
.callback_arg = h,
},
};
esp_err_t err = usb_host_client_register(&client_config, &h->client_hdl);
if (err != ESP_OK) {
ESP_LOGE(TAG, "usb_host_client_register failed: %d", err);
h->running = false;
vTaskDelete(NULL);
return;
}
while (h->running) {
/* This call is also what dispatches completion callbacks for any
* control/bulk transfers this client has outstanding — see file
* header. Timeout keeps the loop responsive to h->running. */
usb_host_client_handle_events(h->client_hdl, pdMS_TO_TICKS(200));
if (h->pending_new_dev_addr != 0) {
uint8_t addr = h->pending_new_dev_addr;
h->pending_new_dev_addr = 0;
if (!h->dev_hdl) { /* only attempt bring-up if we don't already hold a device */
rtlsdr_try_bring_up(h, addr);
}
}
if (h->pending_dev_gone) {
h->pending_dev_gone = false;
ESP_LOGW(TAG, "RTL2832U device disconnected");
rtlsdr_teardown_device(h);
}
}
rtlsdr_teardown_device(h);
usb_host_client_deregister(h->client_hdl);
h->client_hdl = NULL;
vTaskDelete(NULL);
}
/* ---------------------------------------------------------------------------
* Public lifecycle API
* ------------------------------------------------------------------------ */
esp_err_t rtlsdr_exp_init(const rtlsdr_exp_config_t *config, rtlsdr_exp_handle_t *out_handle)
{
if (!config || !out_handle) return ESP_ERR_INVALID_ARG;
rtlsdr_exp_handle_t h = calloc(1, sizeof(struct rtlsdr_exp_handle_s));
if (!h) return ESP_ERR_NO_MEM;
h->cfg = *config;
if (h->cfg.center_freq_hz == 0) h->cfg.center_freq_hz = RTLSDR_EXP_DEFAULT_CENTER_HZ;
if (h->cfg.sample_rate_hz == 0) h->cfg.sample_rate_hz = RTLSDR_EXP_DEFAULT_SAMPLE_RATE;
if (h->cfg.bulk_read_chunk_bytes == 0) h->cfg.bulk_read_chunk_bytes = RTLSDR_EXP_DEFAULT_CHUNK_BYTES;
if (h->cfg.bulk_read_queue_depth == 0) h->cfg.bulk_read_queue_depth = RTLSDR_EXP_DEFAULT_QUEUE_DEPTH;
h->iq_block_queue = xQueueCreate(RTLSDR_EXP_FORWARD_QUEUE_LEN, sizeof(rtlsdr_iq_block_t *));
if (!h->iq_block_queue) {
free(h);
return ESP_ERR_NO_MEM;
}
*out_handle = h;
return ESP_OK;
}
esp_err_t rtlsdr_exp_deinit(rtlsdr_exp_handle_t h)
{
if (!h) return ESP_ERR_INVALID_ARG;
if (h->running) {
rtlsdr_exp_stop(h);
}
if (h->iq_block_queue) vQueueDelete(h->iq_block_queue);
free(h);
return ESP_OK;
}
esp_err_t rtlsdr_exp_start(rtlsdr_exp_handle_t h)
{
if (!h) return ESP_ERR_INVALID_ARG;
if (h->running) return ESP_ERR_INVALID_STATE;
/* NOTE: usb_host_install() installs a library instance shared by the
* whole firmware process. If Workstream I's core skeleton (or any
* future module) also needs USB host for something else, installing
* it twice is an error — this experimental module assumes it is the
* ONLY USB Host client in the project. Document/resolve this at merge
* time; see README integration notes. */
usb_host_config_t host_config = {
.skip_phy_setup = false,
.intr_flags = ESP_INTR_FLAG_LEVEL1,
};
esp_err_t err = usb_host_install(&host_config);
if (err != ESP_OK) {
ESP_LOGE(TAG, "usb_host_install failed: %d", err);
return err;
}
h->running = true;
BaseType_t ok = xTaskCreate(rtlsdr_usb_lib_daemon_task, "rtlsdr_usb_lib", 4096, h, 5, &h->lib_task);
if (ok != pdPASS) {
h->running = false;
usb_host_uninstall();
return ESP_ERR_NO_MEM;
}
ok = xTaskCreate(rtlsdr_exp_client_task, "rtlsdr_client", 6144, h, 5, &h->client_task);
if (ok != pdPASS) {
h->running = false;
vTaskDelete(h->lib_task);
usb_host_uninstall();
return ESP_ERR_NO_MEM;
}
ok = xTaskCreate(rtlsdr_forward_task, "rtlsdr_forward", 4096, h, 4, &h->forward_task);
if (ok != pdPASS) {
h->running = false;
vTaskDelete(h->client_task);
vTaskDelete(h->lib_task);
usb_host_uninstall();
return ESP_ERR_NO_MEM;
}
ESP_LOGI(TAG, "rtlsdr_experimental started (UNVERIFIED against real hardware — "
"see firmware/esp32p4-sensor-node/README.md)");
return ESP_OK;
}
esp_err_t rtlsdr_exp_stop(rtlsdr_exp_handle_t h)
{
if (!h) return ESP_ERR_INVALID_ARG;
if (!h->running) return ESP_OK;
h->running = false;
/* Tasks self-delete once they observe h->running == false; give them a
* moment. A production version should use task-completion notification
* instead of a fixed delay — left as a TODO, not hardware-dependent but
* still unverified/untuned. */
vTaskDelay(pdMS_TO_TICKS(500));
usb_host_uninstall(); /* see rtlsdr_exp_start() note re: sole USB client assumption */
return ESP_OK;
}
void rtlsdr_exp_get_stats(rtlsdr_exp_handle_t h, rtlsdr_exp_stats_t *out_stats)
{
if (!h || !out_stats) return;
*out_stats = h->stats;
}