drjones/qtalker---
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

firmware: add I2S MEMS microphone driver; fix missing essence migration

Browse Source 
Mic: confirmed via its pinout (L/R, WS, SCK, SD, VCC, GND) that the third
target module is a standard I2S digital MEMS mic (INMP441-family). Added
mems_mic.c/h using ESP-IDF's current driver/i2s_std.h API — reports RMS
audio level in dBFS as sensor_type "evp" rather than attempting on-device
voice-band FFT (the browser EVP mode's approach); the backend's existing
statistical anomaly detector handles spike detection from the raw level,
same as it already does for temperature/pressure/presence.

Also fixes a real gap Workstream B's report flagged: User.essence (a live
model column used throughout merged code — /auth/me, inventory purchases,
summon trickle) had no migration line in main.py's lifespan, which would
have broken on the actual production Postgres database.
This commit is contained in:
Indiana
2026-07-24 21:52:03 +00:00
parent 7ebedf363b
commit f09e077199
6 changed files with 288 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
firmware/esp32p4-sensor-node/README.md
View File

@@ -16,6 +16,12 @@ with 28 programmable GPIOs). All pin assignments and the Wi-Fi bring-up
approach below are specific to this board — see the Wi-Fi section for the
reserved SDIO pins and [Wiring / pinout](#wiring--pinout) for sensor pins.
**Target sensors:** a **GY-BMP280** breakout (temperature/pressure, I2C),
an **Ai-Thinker RD-03E** 24GHz mmWave radar (presence/distance/gesture,
UART), and an **I2S digital MEMS microphone** (INMP441-family pinout —
`L/R`/`WS`/`SCK`/`SD` — EVP-style audio level). See
[Wiring / pinout](#wiring--pinout) for exact connections.
## Honesty policy — READ THIS FIRST
> This app's whole ethos is "real signal processing on real data, and it
@@ -54,7 +60,8 @@ firmware/esp32p4-sensor-node/
├── sensor_driver.h the sensor_driver_t registry interface
├── sensor_registry.{h,c} the concrete list of compiled-in drivers
├── bmp280.{h,c} temperature/pressure over I2C
└── rd03e.{h,c} presence/distance/gesture over UART
├── rd03e.{h,c} presence/distance/gesture over UART
└── mems_mic.{h,c} EVP-style audio RMS level over I2S
```
## Build instructions
@@ -136,6 +143,12 @@ consistent; no known syntax errors or obviously-wrong API usage):
driver reports it as a raw code in `metadata` rather than guessing at a
translated label. Cross-confirmed from multiple sources: 256000 baud,
8N1 UART framing.
- I2S MEMS microphone driver (`mems_mic.c`): uses ESP-IDF's current
`driver/i2s_std.h` API (standard/Philips mode, mono, 32-bit slot). The
24-bit-in-32-bit-slot right-shift and dBFS reference level are the
commonly-documented values for the INMP441 family this module's pinout
(`L/R`/`WS`/`SCK`/`SD`) matches. Deliberately does RMS-level reporting
only, not on-device voice-band FFT — see its header comment for why.
- Sensor driver registry (`sensor_driver.h`, `sensor_registry.c`): a
`sensor_driver_t { name, init, read }` struct, a compile-time array of
them, and generic init/collect functions that `app_main.c` and
@@ -159,6 +172,11 @@ consistent; no known syntax errors or obviously-wrong API usage):
third-party source rather than an official datasheet. Verify against a
logic analyzer capture before trusting field values, and treat the
gesture code's meaning as genuinely unknown until cross-checked.
- I2S mic timing/levels: the BCLK/WS timing relationship, whether the
24-bit-in-32-bit-slot shift is exactly right for this specific module
revision, and whether the dBFS numbers land in a sane, usable range —
none of this has been bench-tested. Confirm by talking near the mic and
checking the reported level actually rises before trusting it unattended.
- Wi-Fi reconnect behavior under real-world conditions (router reboot,
weak signal, captive portals) — the backoff logic is reasoned about, not
soak-tested.
@@ -277,6 +295,36 @@ only the module's free-running "simple report" frames — no configuration
handshake is sent or required. GPIO numbers and baud rate are `#define`s
at the top of `rd03e.h`.
### I2S MEMS microphone — EVP-style audio level
Confirmed against the actual part in use via its pinout (`L/R`, `WS`,
`SCK`, `SD`, `VCC`, `GND` — the standard INMP441-family I2S digital MEMS
mic breakout naming). Unlike the browser-based EVP mode's client-side
voice-band FFT, this driver does not attempt on-device spectral analysis —
it samples a short audio block per cycle, computes RMS level in dBFS, and
reports that as a plain numeric reading. The backend's existing
statistical anomaly detector (the same one already used for
temperature/pressure) does the spike detection from there — simpler and
more honest than pretending to replicate real voice-band filtering without
ever having tested it.
| Mic pin | Connects to |
|---------|-------------------------------------------|
| VCC | 3V3 |
| GND | GND |
| L/R | GND (selects left-channel output — tie to 3V3 instead for right, either works, just match the driver's `I2S_STD_SLOT_LEFT` default or change it) |
| WS | GPIO11 (`MEMS_MIC_I2S_WS_GPIO`) — word select / LRCLK |
| SCK | GPIO10 (`MEMS_MIC_I2S_BCLK_GPIO`) — bit clock |
| SD | GPIO12 (`MEMS_MIC_I2S_DIN_GPIO`) — serial data, mic OUT to ESP32 IN |
(GPIO10/11/12 avoid this board's reserved SDIO range, the BMP280's I2C
pins, and the RD-03E's UART pins — a reasonable, currently-unused pick,
same verification caveat as the other sensors' pins above.)
16kHz sample rate, 256ms sample block per reporting cycle (4096 samples) —
`#define`s at the top of `mems_mic.h`. Reports `sensor_type: "evp"`,
`unit: "dbfs"`.
## Sensor driver registry — the extensibility pattern
`sensor_driver.h` defines:

12
firmware/esp32p4-sensor-node/main/CMakeLists.txt
View File

@@ -14,6 +14,7 @@ idf_component_register(
"sensor_registry.c"
"bmp280.c"
"rd03e.c"
"mems_mic.c"
INCLUDE_DIRS
"."
REQUIRES
@@ -30,8 +31,9 @@ idf_component_register(
# Note (unverified): recent ESP-IDF versions (v5.3+) split the old
# monolithic "driver" component into per-peripheral components
# (esp_driver_i2c, esp_driver_uart, ...); "driver" is kept as a
# backward-compatible umbrella that still pulls those in, which is why a
# plain REQUIRES driver is used above. If a real build against your exact
# IDF version complains it can't find driver/i2c_master.h or driver/uart.h,
# add esp_driver_i2c / esp_driver_uart explicitly to REQUIRES.
# (esp_driver_i2c, esp_driver_uart, esp_driver_i2s, ...); "driver" is kept
# as a backward-compatible umbrella that still pulls those in, which is why
# a plain REQUIRES driver is used above. If a real build against your exact
# IDF version complains it can't find driver/i2c_master.h, driver/uart.h,
# or driver/i2s_std.h, add esp_driver_i2c / esp_driver_uart / esp_driver_i2s
# explicitly to REQUIRES.

147
firmware/esp32p4-sensor-node/main/mems_mic.c Normal file
View File

@@ -0,0 +1,147 @@
// I2S MEMS microphone driver — see mems_mic.h for wiring and honesty notes.
//
// UNVERIFIED AGAINST REAL HARDWARE. Written against ESP-IDF's documented
// `driver/i2s_std.h` API (the current idiomatic I2S driver, superseding the
// older monolithic `driver/i2s.h`) and the INMP441 family's well-documented
// output format: 24-bit signed PCM, MSB-first, left-justified in a 32-bit
// I2S slot (Philips/standard I2S timing). The right-shift-by-8 used below
// to recover the 24-bit sample from the 32-bit slot, and the dBFS
// reference level (2^23, a 24-bit signed sample's full-scale magnitude),
// are the commonly-documented values for this exact mic family — but
// "commonly documented" is not "verified against this specific board," so
// treat the very first real readings as a sanity check, not a given: talk
// near the mic and confirm the reported level actually rises before
// trusting it unattended.
#include <string.h>
#include <stdbool.h>
#include <math.h>
#include <stdlib.h>
#include "mems_mic.h"
#include "driver/i2s_std.h"
#include "esp_log.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
static const char *TAG = "mems_mic";
// dBFS reference: full-scale magnitude of a 24-bit signed sample.
#define FULL_SCALE_24BIT (8388608.0) // 2^23
static i2s_chan_handle_t s_rx_chan = NULL;
static bool s_ready = false;
static int32_t *s_sample_buf = NULL; // heap-allocated, MEMS_MIC_SAMPLES_PER_READ entries
esp_err_t mems_mic_init(void) {
s_sample_buf = (int32_t *)malloc(MEMS_MIC_SAMPLES_PER_READ * sizeof(int32_t));
if (s_sample_buf == NULL) {
ESP_LOGE(TAG, "sample buffer allocation failed");
return ESP_ERR_NO_MEM;
}
i2s_chan_config_t chan_cfg = I2S_CHANNEL_DEFAULT_CONFIG(MEMS_MIC_I2S_PORT, I2S_ROLE_MASTER);
esp_err_t err = i2s_new_channel(&chan_cfg, NULL, &s_rx_chan);
if (err != ESP_OK) {
ESP_LOGE(TAG, "i2s_new_channel failed: %s", esp_err_to_name(err));
free(s_sample_buf);
s_sample_buf = NULL;
return err;
}
i2s_std_config_t std_cfg = {
.clk_cfg = I2S_STD_CLK_DEFAULT_CONFIG(MEMS_MIC_SAMPLE_RATE_HZ),
.slot_cfg = I2S_STD_PHILIPS_SLOT_DEFAULT_CONFIG(
I2S_DATA_BIT_WIDTH_32BIT, I2S_SLOT_MODE_MONO),
.gpio_cfg = {
.mclk = I2S_GPIO_UNUSED,
.bclk = MEMS_MIC_I2S_BCLK_GPIO,
.ws = MEMS_MIC_I2S_WS_GPIO,
.dout = I2S_GPIO_UNUSED, // RX-only channel, no data output pin
.din = MEMS_MIC_I2S_DIN_GPIO,
.invert_flags = {
.mclk_inv = false,
.bclk_inv = false,
.ws_inv = false,
},
},
};
// Left channel per this driver's documented default wiring (mic's L/R
// pin tied to GND) -- change to I2S_STD_SLOT_RIGHT to match a mic
// wired the other way.
std_cfg.slot_cfg.slot_mask = I2S_STD_SLOT_LEFT;
err = i2s_channel_init_std_mode(s_rx_chan, &std_cfg);
if (err != ESP_OK) {
ESP_LOGE(TAG, "i2s_channel_init_std_mode failed: %s", esp_err_to_name(err));
free(s_sample_buf);
s_sample_buf = NULL;
return err;
}
err = i2s_channel_enable(s_rx_chan);
if (err != ESP_OK) {
ESP_LOGE(TAG, "i2s_channel_enable failed: %s", esp_err_to_name(err));
free(s_sample_buf);
s_sample_buf = NULL;
return err;
}
s_ready = true;
ESP_LOGI(TAG, "I2S mic init ok (%d Hz, port %d)", MEMS_MIC_SAMPLE_RATE_HZ, MEMS_MIC_I2S_PORT);
return ESP_OK;
}
esp_err_t mems_mic_read(sensor_reading_t *out, size_t max_out, size_t *out_count) {
*out_count = 0;
if (!s_ready) {
return ESP_ERR_INVALID_STATE;
}
if (max_out < 1) {
return ESP_ERR_NO_MEM;
}
size_t bytes_to_read = MEMS_MIC_SAMPLES_PER_READ * sizeof(int32_t);
size_t bytes_read = 0;
esp_err_t err = i2s_channel_read(s_rx_chan, s_sample_buf, bytes_to_read,
&bytes_read, pdMS_TO_TICKS(500));
if (err != ESP_OK) {
ESP_LOGW(TAG, "i2s_channel_read failed: %s", esp_err_to_name(err));
return err;
}
size_t n_samples = bytes_read / sizeof(int32_t);
if (n_samples == 0) {
ESP_LOGD(TAG, "no I2S samples this cycle");
return ESP_OK;
}
// RMS over the block. The mic's 24-bit sample is left-justified in the
// 32-bit I2S slot -- shift right 8 to recover it before squaring, so
// the magnitude lines up with FULL_SCALE_24BIT below.
double sum_sq = 0.0;
for (size_t i = 0; i < n_samples; i++) {
double sample = (double)(s_sample_buf[i] >> 8);
sum_sq += sample * sample;
}
double rms = sqrt(sum_sq / (double)n_samples);
// dBFS: 20*log10(rms / full_scale). A true-silent input gives rms=0,
// which is -inf in dB -- clamp to a floor rather than emit a value the
// JSON encoder/backend can't handle.
double dbfs;
if (rms < 1.0) {
dbfs = -120.0; // effective noise floor
} else {
dbfs = 20.0 * log10(rms / FULL_SCALE_24BIT);
if (dbfs < -120.0) dbfs = -120.0;
}
memset(&out[0], 0, sizeof(out[0]));
strncpy(out[0].sensor_type, "evp", SENSOR_READING_TYPE_MAXLEN - 1);
out[0].value = dbfs;
strncpy(out[0].unit, "dbfs", SENSOR_READING_UNIT_MAXLEN - 1);
out[0].metadata = NULL;
*out_count = 1;
return ESP_OK;
}

75
firmware/esp32p4-sensor-node/main/mems_mic.h Normal file
View File

@@ -0,0 +1,75 @@
// I2S digital MEMS microphone driver (INMP441-family — L/R, WS, SCK, SD
// pinout, confirmed against the actual module in use).
//
// Unlike the browser-based EVP mode's client-side voice-band FFT
// (frontend/src/lib/evp.ts), this driver does NOT attempt on-device
// spectral analysis — it samples a short audio block per reporting cycle
// and reports its RMS energy level in dBFS as a plain numeric reading.
// The backend's existing statistical anomaly detector (the same one that
// already handles temperature/pressure) does the spike detection — no
// need to duplicate baseline-tracking logic on the device. This is
// simpler and more honest than pretending to replicate real voice-band
// filtering without ever having tested it.
//
// Implements the sensor_driver_t interface (see sensor_driver.h).
//
// Wiring (module pin names: L/R, WS, SCK, SD, VCC, GND):
// Mic VCC -> 3V3
// Mic GND -> GND
// Mic L/R -> GND (selects left-channel output; tie to 3V3 instead for
// right-channel — either is fine, this driver reads whichever
// channel the mic is configured to output)
// Mic WS -> ESP32 GPIO (MEMS_MIC_I2S_WS_GPIO) -- word select / LRCLK
// Mic SCK -> ESP32 GPIO (MEMS_MIC_I2S_BCLK_GPIO) -- bit clock
// Mic SD -> ESP32 GPIO (MEMS_MIC_I2S_DIN_GPIO) -- serial data OUT of
// the mic, IN to the ESP32
// GPIO picks avoid this board's reserved SDIO range (14-19, 54), the
// BMP280's I2C pins (8/9), and the RD-03E's UART pins (4/5) -- see
// README.md's pinout table for the full picture.
#pragma once
#include "esp_err.h"
#include "sensor_driver.h"
#ifdef __cplusplus
extern "C" {
#endif
#ifndef MEMS_MIC_I2S_PORT
#define MEMS_MIC_I2S_PORT 0
#endif
#ifndef MEMS_MIC_I2S_BCLK_GPIO
#define MEMS_MIC_I2S_BCLK_GPIO 10
#endif
#ifndef MEMS_MIC_I2S_WS_GPIO
#define MEMS_MIC_I2S_WS_GPIO 11
#endif
#ifndef MEMS_MIC_I2S_DIN_GPIO
#define MEMS_MIC_I2S_DIN_GPIO 12
#endif
// 16kHz is standard for voice-band work (Nyquist covers the ~300Hz-3.4kHz
// band the browser-based EVP mode watches) without the data-rate/CPU cost
// of a higher rate this simple RMS-only driver doesn't need.
#ifndef MEMS_MIC_SAMPLE_RATE_HZ
#define MEMS_MIC_SAMPLE_RATE_HZ 16000
#endif
// One reporting cycle's sample block: 256ms at 16kHz. Long enough to
// average out single-sample noise, short enough to stay responsive to a
// genuine brief EVP-style spike.
#ifndef MEMS_MIC_SAMPLES_PER_READ
#define MEMS_MIC_SAMPLES_PER_READ 4096
#endif
// sensor_driver_t-compatible entry points.
esp_err_t mems_mic_init(void);
esp_err_t mems_mic_read(sensor_reading_t *out, size_t max_out, size_t *out_count);
#ifdef __cplusplus
}
#endif

2
firmware/esp32p4-sensor-node/main/sensor_registry.c
View File

@@ -11,6 +11,7 @@
#include "sensor_registry.h"
#include "bmp280.h"
#include "rd03e.h"
#include "mems_mic.h"
#include "esp_log.h"
static const char *TAG = "sensor_registry";
@@ -21,6 +22,7 @@ static const char *TAG = "sensor_registry";
static const sensor_driver_t s_drivers[] = {
{ .name = "bmp280", .init = bmp280_init, .read = bmp280_read },
{ .name = "rd03e", .init = rd03e_init, .read = rd03e_read },
{ .name = "mems_mic", .init = mems_mic_init, .read = mems_mic_read },
// Add new drivers here, e.g.:
// { .name = "my_sensor", .init = my_sensor_init, .read = my_sensor_read },
};