firmware: add I2S MEMS microphone driver; fix missing essence migration

Mic: confirmed via its pinout (L/R, WS, SCK, SD, VCC, GND) that the third
target module is a standard I2S digital MEMS mic (INMP441-family). Added
mems_mic.c/h using ESP-IDF's current driver/i2s_std.h API — reports RMS
audio level in dBFS as sensor_type "evp" rather than attempting on-device
voice-band FFT (the browser EVP mode's approach); the backend's existing
statistical anomaly detector handles spike detection from the raw level,
same as it already does for temperature/pressure/presence.

Also fixes a real gap Workstream B's report flagged: User.essence (a live
model column used throughout merged code — /auth/me, inventory purchases,
summon trickle) had no migration line in main.py's lifespan, which would
have broken on the actual production Postgres database.
This commit is contained in:
Indiana
2026-07-24 21:52:03 +00:00
parent 7ebedf363b
commit f09e077199
6 changed files with 288 additions and 6 deletions

View File

@@ -54,6 +54,14 @@ async def lifespan(app: FastAPI):
await conn.execute(text( await conn.execute(text(
"ALTER TABLE entities ADD COLUMN IF NOT EXISTS traits JSONB NOT NULL DEFAULT '{}'::jsonb" "ALTER TABLE entities ADD COLUMN IF NOT EXISTS traits JSONB NOT NULL DEFAULT '{}'::jsonb"
)) ))
# Workstream C (character-depth-ghost-log spec) — missing from C's
# own commit, added by the integrator after Workstream B's report
# flagged that User.essence had a live model column and application
# code (auth/me, inventory purchases, summon trickle) but no
# migration, which would have broken on the real production DB.
await conn.execute(text(
"ALTER TABLE users ADD COLUMN IF NOT EXISTS essence INTEGER NOT NULL DEFAULT 0"
))
# Workstream B (character-depth-ghost-log spec). # Workstream B (character-depth-ghost-log spec).
await conn.execute(text( await conn.execute(text(
"ALTER TABLE users ADD COLUMN IF NOT EXISTS favor DOUBLE PRECISION NOT NULL DEFAULT 0.0" "ALTER TABLE users ADD COLUMN IF NOT EXISTS favor DOUBLE PRECISION NOT NULL DEFAULT 0.0"

View File

@@ -16,6 +16,12 @@ with 28 programmable GPIOs). All pin assignments and the Wi-Fi bring-up
approach below are specific to this board — see the Wi-Fi section for the approach below are specific to this board — see the Wi-Fi section for the
reserved SDIO pins and [Wiring / pinout](#wiring--pinout) for sensor pins. reserved SDIO pins and [Wiring / pinout](#wiring--pinout) for sensor pins.
**Target sensors:** a **GY-BMP280** breakout (temperature/pressure, I2C),
an **Ai-Thinker RD-03E** 24GHz mmWave radar (presence/distance/gesture,
UART), and an **I2S digital MEMS microphone** (INMP441-family pinout —
`L/R`/`WS`/`SCK`/`SD` — EVP-style audio level). See
[Wiring / pinout](#wiring--pinout) for exact connections.
## Honesty policy — READ THIS FIRST ## Honesty policy — READ THIS FIRST
> This app's whole ethos is "real signal processing on real data, and it > This app's whole ethos is "real signal processing on real data, and it
@@ -54,7 +60,8 @@ firmware/esp32p4-sensor-node/
├── sensor_driver.h the sensor_driver_t registry interface ├── sensor_driver.h the sensor_driver_t registry interface
├── sensor_registry.{h,c} the concrete list of compiled-in drivers ├── sensor_registry.{h,c} the concrete list of compiled-in drivers
├── bmp280.{h,c} temperature/pressure over I2C ├── bmp280.{h,c} temperature/pressure over I2C
└── rd03e.{h,c} presence/distance/gesture over UART ├── rd03e.{h,c} presence/distance/gesture over UART
└── mems_mic.{h,c} EVP-style audio RMS level over I2S
``` ```
## Build instructions ## Build instructions
@@ -136,6 +143,12 @@ consistent; no known syntax errors or obviously-wrong API usage):
driver reports it as a raw code in `metadata` rather than guessing at a driver reports it as a raw code in `metadata` rather than guessing at a
translated label. Cross-confirmed from multiple sources: 256000 baud, translated label. Cross-confirmed from multiple sources: 256000 baud,
8N1 UART framing. 8N1 UART framing.
- I2S MEMS microphone driver (`mems_mic.c`): uses ESP-IDF's current
`driver/i2s_std.h` API (standard/Philips mode, mono, 32-bit slot). The
24-bit-in-32-bit-slot right-shift and dBFS reference level are the
commonly-documented values for the INMP441 family this module's pinout
(`L/R`/`WS`/`SCK`/`SD`) matches. Deliberately does RMS-level reporting
only, not on-device voice-band FFT — see its header comment for why.
- Sensor driver registry (`sensor_driver.h`, `sensor_registry.c`): a - Sensor driver registry (`sensor_driver.h`, `sensor_registry.c`): a
`sensor_driver_t { name, init, read }` struct, a compile-time array of `sensor_driver_t { name, init, read }` struct, a compile-time array of
them, and generic init/collect functions that `app_main.c` and them, and generic init/collect functions that `app_main.c` and
@@ -159,6 +172,11 @@ consistent; no known syntax errors or obviously-wrong API usage):
third-party source rather than an official datasheet. Verify against a third-party source rather than an official datasheet. Verify against a
logic analyzer capture before trusting field values, and treat the logic analyzer capture before trusting field values, and treat the
gesture code's meaning as genuinely unknown until cross-checked. gesture code's meaning as genuinely unknown until cross-checked.
- I2S mic timing/levels: the BCLK/WS timing relationship, whether the
24-bit-in-32-bit-slot shift is exactly right for this specific module
revision, and whether the dBFS numbers land in a sane, usable range —
none of this has been bench-tested. Confirm by talking near the mic and
checking the reported level actually rises before trusting it unattended.
- Wi-Fi reconnect behavior under real-world conditions (router reboot, - Wi-Fi reconnect behavior under real-world conditions (router reboot,
weak signal, captive portals) — the backoff logic is reasoned about, not weak signal, captive portals) — the backoff logic is reasoned about, not
soak-tested. soak-tested.
@@ -277,6 +295,36 @@ only the module's free-running "simple report" frames — no configuration
handshake is sent or required. GPIO numbers and baud rate are `#define`s handshake is sent or required. GPIO numbers and baud rate are `#define`s
at the top of `rd03e.h`. at the top of `rd03e.h`.
### I2S MEMS microphone — EVP-style audio level
Confirmed against the actual part in use via its pinout (`L/R`, `WS`,
`SCK`, `SD`, `VCC`, `GND` — the standard INMP441-family I2S digital MEMS
mic breakout naming). Unlike the browser-based EVP mode's client-side
voice-band FFT, this driver does not attempt on-device spectral analysis —
it samples a short audio block per cycle, computes RMS level in dBFS, and
reports that as a plain numeric reading. The backend's existing
statistical anomaly detector (the same one already used for
temperature/pressure) does the spike detection from there — simpler and
more honest than pretending to replicate real voice-band filtering without
ever having tested it.
| Mic pin | Connects to |
|---------|-------------------------------------------|
| VCC | 3V3 |
| GND | GND |
| L/R | GND (selects left-channel output — tie to 3V3 instead for right, either works, just match the driver's `I2S_STD_SLOT_LEFT` default or change it) |
| WS | GPIO11 (`MEMS_MIC_I2S_WS_GPIO`) — word select / LRCLK |
| SCK | GPIO10 (`MEMS_MIC_I2S_BCLK_GPIO`) — bit clock |
| SD | GPIO12 (`MEMS_MIC_I2S_DIN_GPIO`) — serial data, mic OUT to ESP32 IN |
(GPIO10/11/12 avoid this board's reserved SDIO range, the BMP280's I2C
pins, and the RD-03E's UART pins — a reasonable, currently-unused pick,
same verification caveat as the other sensors' pins above.)
16kHz sample rate, 256ms sample block per reporting cycle (4096 samples) —
`#define`s at the top of `mems_mic.h`. Reports `sensor_type: "evp"`,
`unit: "dbfs"`.
## Sensor driver registry — the extensibility pattern ## Sensor driver registry — the extensibility pattern
`sensor_driver.h` defines: `sensor_driver.h` defines:

View File

@@ -14,6 +14,7 @@ idf_component_register(
"sensor_registry.c" "sensor_registry.c"
"bmp280.c" "bmp280.c"
"rd03e.c" "rd03e.c"
"mems_mic.c"
INCLUDE_DIRS INCLUDE_DIRS
"." "."
REQUIRES REQUIRES
@@ -30,8 +31,9 @@ idf_component_register(
# Note (unverified): recent ESP-IDF versions (v5.3+) split the old # Note (unverified): recent ESP-IDF versions (v5.3+) split the old
# monolithic "driver" component into per-peripheral components # monolithic "driver" component into per-peripheral components
# (esp_driver_i2c, esp_driver_uart, ...); "driver" is kept as a # (esp_driver_i2c, esp_driver_uart, esp_driver_i2s, ...); "driver" is kept
# backward-compatible umbrella that still pulls those in, which is why a # as a backward-compatible umbrella that still pulls those in, which is why
# plain REQUIRES driver is used above. If a real build against your exact # a plain REQUIRES driver is used above. If a real build against your exact
# IDF version complains it can't find driver/i2c_master.h or driver/uart.h, # IDF version complains it can't find driver/i2c_master.h, driver/uart.h,
# add esp_driver_i2c / esp_driver_uart explicitly to REQUIRES. # or driver/i2s_std.h, add esp_driver_i2c / esp_driver_uart / esp_driver_i2s
# explicitly to REQUIRES.

View File

@@ -0,0 +1,147 @@
// I2S MEMS microphone driver — see mems_mic.h for wiring and honesty notes.
//
// UNVERIFIED AGAINST REAL HARDWARE. Written against ESP-IDF's documented
// `driver/i2s_std.h` API (the current idiomatic I2S driver, superseding the
// older monolithic `driver/i2s.h`) and the INMP441 family's well-documented
// output format: 24-bit signed PCM, MSB-first, left-justified in a 32-bit
// I2S slot (Philips/standard I2S timing). The right-shift-by-8 used below
// to recover the 24-bit sample from the 32-bit slot, and the dBFS
// reference level (2^23, a 24-bit signed sample's full-scale magnitude),
// are the commonly-documented values for this exact mic family — but
// "commonly documented" is not "verified against this specific board," so
// treat the very first real readings as a sanity check, not a given: talk
// near the mic and confirm the reported level actually rises before
// trusting it unattended.
#include <string.h>
#include <stdbool.h>
#include <math.h>
#include <stdlib.h>
#include "mems_mic.h"
#include "driver/i2s_std.h"
#include "esp_log.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
static const char *TAG = "mems_mic";
// dBFS reference: full-scale magnitude of a 24-bit signed sample.
#define FULL_SCALE_24BIT (8388608.0) // 2^23
static i2s_chan_handle_t s_rx_chan = NULL;
static bool s_ready = false;
static int32_t *s_sample_buf = NULL; // heap-allocated, MEMS_MIC_SAMPLES_PER_READ entries
esp_err_t mems_mic_init(void) {
s_sample_buf = (int32_t *)malloc(MEMS_MIC_SAMPLES_PER_READ * sizeof(int32_t));
if (s_sample_buf == NULL) {
ESP_LOGE(TAG, "sample buffer allocation failed");
return ESP_ERR_NO_MEM;
}
i2s_chan_config_t chan_cfg = I2S_CHANNEL_DEFAULT_CONFIG(MEMS_MIC_I2S_PORT, I2S_ROLE_MASTER);
esp_err_t err = i2s_new_channel(&chan_cfg, NULL, &s_rx_chan);
if (err != ESP_OK) {
ESP_LOGE(TAG, "i2s_new_channel failed: %s", esp_err_to_name(err));
free(s_sample_buf);
s_sample_buf = NULL;
return err;
}
i2s_std_config_t std_cfg = {
.clk_cfg = I2S_STD_CLK_DEFAULT_CONFIG(MEMS_MIC_SAMPLE_RATE_HZ),
.slot_cfg = I2S_STD_PHILIPS_SLOT_DEFAULT_CONFIG(
I2S_DATA_BIT_WIDTH_32BIT, I2S_SLOT_MODE_MONO),
.gpio_cfg = {
.mclk = I2S_GPIO_UNUSED,
.bclk = MEMS_MIC_I2S_BCLK_GPIO,
.ws = MEMS_MIC_I2S_WS_GPIO,
.dout = I2S_GPIO_UNUSED, // RX-only channel, no data output pin
.din = MEMS_MIC_I2S_DIN_GPIO,
.invert_flags = {
.mclk_inv = false,
.bclk_inv = false,
.ws_inv = false,
},
},
};
// Left channel per this driver's documented default wiring (mic's L/R
// pin tied to GND) -- change to I2S_STD_SLOT_RIGHT to match a mic
// wired the other way.
std_cfg.slot_cfg.slot_mask = I2S_STD_SLOT_LEFT;
err = i2s_channel_init_std_mode(s_rx_chan, &std_cfg);
if (err != ESP_OK) {
ESP_LOGE(TAG, "i2s_channel_init_std_mode failed: %s", esp_err_to_name(err));
free(s_sample_buf);
s_sample_buf = NULL;
return err;
}
err = i2s_channel_enable(s_rx_chan);
if (err != ESP_OK) {
ESP_LOGE(TAG, "i2s_channel_enable failed: %s", esp_err_to_name(err));
free(s_sample_buf);
s_sample_buf = NULL;
return err;
}
s_ready = true;
ESP_LOGI(TAG, "I2S mic init ok (%d Hz, port %d)", MEMS_MIC_SAMPLE_RATE_HZ, MEMS_MIC_I2S_PORT);
return ESP_OK;
}
esp_err_t mems_mic_read(sensor_reading_t *out, size_t max_out, size_t *out_count) {
*out_count = 0;
if (!s_ready) {
return ESP_ERR_INVALID_STATE;
}
if (max_out < 1) {
return ESP_ERR_NO_MEM;
}
size_t bytes_to_read = MEMS_MIC_SAMPLES_PER_READ * sizeof(int32_t);
size_t bytes_read = 0;
esp_err_t err = i2s_channel_read(s_rx_chan, s_sample_buf, bytes_to_read,
&bytes_read, pdMS_TO_TICKS(500));
if (err != ESP_OK) {
ESP_LOGW(TAG, "i2s_channel_read failed: %s", esp_err_to_name(err));
return err;
}
size_t n_samples = bytes_read / sizeof(int32_t);
if (n_samples == 0) {
ESP_LOGD(TAG, "no I2S samples this cycle");
return ESP_OK;
}
// RMS over the block. The mic's 24-bit sample is left-justified in the
// 32-bit I2S slot -- shift right 8 to recover it before squaring, so
// the magnitude lines up with FULL_SCALE_24BIT below.
double sum_sq = 0.0;
for (size_t i = 0; i < n_samples; i++) {
double sample = (double)(s_sample_buf[i] >> 8);
sum_sq += sample * sample;
}
double rms = sqrt(sum_sq / (double)n_samples);
// dBFS: 20*log10(rms / full_scale). A true-silent input gives rms=0,
// which is -inf in dB -- clamp to a floor rather than emit a value the
// JSON encoder/backend can't handle.
double dbfs;
if (rms < 1.0) {
dbfs = -120.0; // effective noise floor
} else {
dbfs = 20.0 * log10(rms / FULL_SCALE_24BIT);
if (dbfs < -120.0) dbfs = -120.0;
}
memset(&out[0], 0, sizeof(out[0]));
strncpy(out[0].sensor_type, "evp", SENSOR_READING_TYPE_MAXLEN - 1);
out[0].value = dbfs;
strncpy(out[0].unit, "dbfs", SENSOR_READING_UNIT_MAXLEN - 1);
out[0].metadata = NULL;
*out_count = 1;
return ESP_OK;
}

View File

@@ -0,0 +1,75 @@
// I2S digital MEMS microphone driver (INMP441-family — L/R, WS, SCK, SD
// pinout, confirmed against the actual module in use).
//
// Unlike the browser-based EVP mode's client-side voice-band FFT
// (frontend/src/lib/evp.ts), this driver does NOT attempt on-device
// spectral analysis — it samples a short audio block per reporting cycle
// and reports its RMS energy level in dBFS as a plain numeric reading.
// The backend's existing statistical anomaly detector (the same one that
// already handles temperature/pressure) does the spike detection — no
// need to duplicate baseline-tracking logic on the device. This is
// simpler and more honest than pretending to replicate real voice-band
// filtering without ever having tested it.
//
// Implements the sensor_driver_t interface (see sensor_driver.h).
//
// Wiring (module pin names: L/R, WS, SCK, SD, VCC, GND):
// Mic VCC -> 3V3
// Mic GND -> GND
// Mic L/R -> GND (selects left-channel output; tie to 3V3 instead for
// right-channel — either is fine, this driver reads whichever
// channel the mic is configured to output)
// Mic WS -> ESP32 GPIO (MEMS_MIC_I2S_WS_GPIO) -- word select / LRCLK
// Mic SCK -> ESP32 GPIO (MEMS_MIC_I2S_BCLK_GPIO) -- bit clock
// Mic SD -> ESP32 GPIO (MEMS_MIC_I2S_DIN_GPIO) -- serial data OUT of
// the mic, IN to the ESP32
// GPIO picks avoid this board's reserved SDIO range (14-19, 54), the
// BMP280's I2C pins (8/9), and the RD-03E's UART pins (4/5) -- see
// README.md's pinout table for the full picture.
#pragma once
#include "esp_err.h"
#include "sensor_driver.h"
#ifdef __cplusplus
extern "C" {
#endif
#ifndef MEMS_MIC_I2S_PORT
#define MEMS_MIC_I2S_PORT 0
#endif
#ifndef MEMS_MIC_I2S_BCLK_GPIO
#define MEMS_MIC_I2S_BCLK_GPIO 10
#endif
#ifndef MEMS_MIC_I2S_WS_GPIO
#define MEMS_MIC_I2S_WS_GPIO 11
#endif
#ifndef MEMS_MIC_I2S_DIN_GPIO
#define MEMS_MIC_I2S_DIN_GPIO 12
#endif
// 16kHz is standard for voice-band work (Nyquist covers the ~300Hz-3.4kHz
// band the browser-based EVP mode watches) without the data-rate/CPU cost
// of a higher rate this simple RMS-only driver doesn't need.
#ifndef MEMS_MIC_SAMPLE_RATE_HZ
#define MEMS_MIC_SAMPLE_RATE_HZ 16000
#endif
// One reporting cycle's sample block: 256ms at 16kHz. Long enough to
// average out single-sample noise, short enough to stay responsive to a
// genuine brief EVP-style spike.
#ifndef MEMS_MIC_SAMPLES_PER_READ
#define MEMS_MIC_SAMPLES_PER_READ 4096
#endif
// sensor_driver_t-compatible entry points.
esp_err_t mems_mic_init(void);
esp_err_t mems_mic_read(sensor_reading_t *out, size_t max_out, size_t *out_count);
#ifdef __cplusplus
}
#endif

View File

@@ -11,6 +11,7 @@
#include "sensor_registry.h" #include "sensor_registry.h"
#include "bmp280.h" #include "bmp280.h"
#include "rd03e.h" #include "rd03e.h"
#include "mems_mic.h"
#include "esp_log.h" #include "esp_log.h"
static const char *TAG = "sensor_registry"; static const char *TAG = "sensor_registry";
@@ -21,6 +22,7 @@ static const char *TAG = "sensor_registry";
static const sensor_driver_t s_drivers[] = { static const sensor_driver_t s_drivers[] = {
{ .name = "bmp280", .init = bmp280_init, .read = bmp280_read }, { .name = "bmp280", .init = bmp280_init, .read = bmp280_read },
{ .name = "rd03e", .init = rd03e_init, .read = rd03e_read }, { .name = "rd03e", .init = rd03e_init, .read = rd03e_read },
{ .name = "mems_mic", .init = mems_mic_init, .read = mems_mic_read },
// Add new drivers here, e.g.: // Add new drivers here, e.g.:
// { .name = "my_sensor", .init = my_sensor_init, .read = my_sensor_read }, // { .name = "my_sensor", .init = my_sensor_init, .read = my_sensor_read },
}; };