Merge pull request 'docs: expand README with full feature catalog and API tables' (#1) from docs/readme-full-catalog-and-api into main

Reviewed-on: drjones/nfc-pn534-warlord#1

README.md

@@ -7,61 +7,112 @@ ESP32-S3 + PN532 + a serious web UI — no desktop app, no dongle software, no m
 
 ## Why this is the best tool for PN532 workflows
 
-1. **Full remote control in the browser** — Dashboard, live scan, deep capture, read/write helpers, raw PN532 frames, and status — all over HTTP + WebSocket. You can stand across the room with your phone while the hardware sits on the bench.
+1. **Full remote control in the browser** — Dashboard, live scan, deep capture, read/write helpers, dictionary attack, raw PN532 commands, and live status — all over **HTTP + WebSocket**. You can stand across the room with your phone while the hardware sits on the bench.
 
-2. **“Deep capture” is actually deep (for a PN532)** — On each new tag we don’t stop at UID: we pull **PN532 general status bytes**, **full inventory (ATQA/SAK/UID)**, then either a **MIFARE Classic sector sweep** (default key set, Key A and Key B per sector trailer) with **every readable block as hex**, or an **Ultralight/NTAG-style page sweep** until the tag stops responding. Unknown SAKs still get inventory + controller status so nothing is silently dropped.
+2. **“Deep capture” is actually deep (for a PN532)** — On each **new UID** the firmware doesn’t stop at inventory: it builds a **JSON profile** with **MIFARE Classic** sector sweeps (default keys, Key **A** then **B** per sector trailer, **1K and 4K layouts**) with **every readable block as hex**, or an **Ultralight/NTAG-style page sweep** (up to **240 pages** in firmware). Unknown or non-Classic paths still get solid **ATQA/SAK/UID** plus whatever the stack can return — nothing is silently dropped.
 
-3. **RAM session buffer with hard stop + phone download** — Captures are stored in **on-chip RAM** (default **48 KB** of NDJSON lines). When the buffer is full, **RF polling pauses** so you never lose data to silent overflow. You tap **Download NDJSON** on your phone, get a single file with every profile, then **Clear** to resume. That workflow is purpose-built for field audits and clone/research sessions.
+3. **RAM session buffer with a hard stop + phone download** — Captures live in **on-chip RAM** (**48 KB** buffer, NDJSON lines). When the buffer is full, **RF polling pauses** so you never lose data to silent overflow. You tap **Download NDJSON**, get one file with every profile, then **Clear** to resume. That workflow is built for field audits, bench sessions, and anything where “I need the dump on my phone, now” matters.
 
-4. **Self-hosted on the device** — Default **open SoftAP** SSID `PN532-Toolkit` (no password, lab default), **mDNS** `pn532tool.local`. No cloud, no account, no telemetry.
+4. **Self-hosted on the device** — Default **SoftAP**: SSID **`PN532-Toolkit`**, password **`nfc-toolkit`** (WPA2). **mDNS** hostname **`pn532tool.local`** (HTTP port **80**). No cloud, no account, no telemetry.
 
-5. **Honest architecture** — ESP-IDF, explicit components (`pn532_host`, `nfc_engine`, `net_service`), embedded Vite/React UI flashed to SPIFFS. You can extend it like real firmware, not a black-box sketch.
+5. **Honest architecture** — **ESP-IDF**, explicit components (`pn532_host`, `nfc_engine`, `net_service`), **Vite/React** UI embedded in **SPIFFS**. You extend it like real firmware, not a black-box sketch. PN532 talks **SPI, I2C, or UART (HSU)** — pins and bus are **menuconfig**, not guesses.
 
-6. **Raw frame escape hatch** — When the high-level UI isn’t enough, hit **Raw** and send PN532 command bytes (frame wrapper handled in firmware). That’s how you stay aligned with the real chip, not a toy abstraction.
+6. **Raw PN532 path** — When the high-level UI isn’t enough, **Raw** sends **command bytes**; the stack handles transport framing. **`GET /api/pn532/general-status`** exposes the controller’s **general status bytes** for debugging. That’s how you stay aligned with the real chip, not a toy abstraction.
 
-This stack is **not** a Proxmark replacement (no LF, no raw carrier manipulation). For **hosted NFC with PN532**, it’s built to be the **most complete pocket operator**: remote UI, deep reads, session export, and a path to grow.
+7. **UI that matches the ambition** — Live **WebSocket** stream (scan + capture), **browser-side log** (thousands of events, exportable JSON), optional **`apiBase`** for talking to the box from another host, **HashRouter** so refreshes don’t fight the server, and **lab fixtures** (synthetic card blobs, keys, SHA checks) so you can validate the UI without burning tags.
+
+This stack is **not** a Proxmark replacement (no LF, no raw carrier manipulation, no FPGA tricks). For **hosted NFC with a PN532**, it’s the **full pocket operator**: remote UI, deep reads, session export, Classic dictionary work, Ultralight read/write, card emulation hooks, and a straight line to the command set when you need it.
+
+---
+
+## Every feature, in one pass
+
+### Firmware (what the chip actually does)
+
+| Capability | What you get |
+|------------|----------------|
+| **Transport** | PN532 over **SPI** (default pins in Kconfig), **I2C**, or **UART (HSU)** — bit rate and GPIOs set in `menuconfig`. |
+| **Continuous scan** | Background task polls passive targets; **~65 ms** when not in deep mode, **~220 ms** when deep capture is on (more time per tag for heavy work). Toggle via **`POST /api/nfc/scan`** or the Dashboard. |
+| **RF sensitivity** | On init, **high passive-activation retries** (`RFConfiguration` **0x05**) so marginal tags get more chances to answer. |
+| **Deep profile** | Per new UID: Classic sector layout detection (1K / 4K), per-sector auth with built-in default keys, block hex arrays; Ultralight/NTAG page reads up to firmware limit; structured **JSON** for export. |
+| **Session RAM** | **~48 KB** NDJSON capture buffer; mutex-protected; **full** flag stops polling until you clear. |
+| **HTTP API** | JSON in/out on documented routes; **CORS** headers on API responses; **`OPTIONS /*`** preflight for cross-origin clients. POST bodies are read **completely** (chunked TCP-safe). |
+| **WebSocket** | **`/ws`**: JSON envelopes `{"channel":"…","payload":…}` — **`scan`** (inventory / `present:false` when tag leaves), **`capture`** (`recorded`, `bufferFull`, etc.), **`pong`** for keepalive. |
+| **OTA endpoint** | **`POST /api/ota`** returns **501** with a stub message — there is **no** in-field OTA in this repo yet; use **`idf.py flash`** (or the included scripts). |
+| **Onboard RGB (DevKit-style)** | Optional **boot-time** shutdown of a **WS2812/SK6812** on **GPIO 48** (Kconfig) so the addressable LED isn’t left on a random color — **not** on all boards; power LEDs are hardware. |
+| **Status** | **`/api/status`**: uptime, free heap, Wi‑Fi mode, **PN532 firmware version** (when reachable), scan flag, session stats. |
+
+### Web UI (each page)
+
+| Page | Purpose |
+|------|---------|
+| **Dash** | Live **status** poll, **continuous scan** on/off, **manual poll** (uses live tag from WebSocket when present), hero readout. |
+| **Read-all (Capture)** | Toggle **deep capture**, **export** NDJSON from device RAM, **clear** buffer, progress vs **48 KB** budget. |
+| **Read** | **MIFARE** block read with key + Key A/B; **Ultralight** page read; paste a **sector trailer** to decode **access bits** in the browser. |
+| **Write** | **MIFARE** block write (16-byte hex, confirm dialog); **Ul** page write; **lab fixture** loader for offline testing flows; ties into **live UID** from WebSocket when available. |
+| **Brute** | **Classic 1K / 4K** dictionary attack: firmware’s built-in key set + your lines, optional **bounded variations**; shows JSON result. |
+| **Emu** | **`TgInitAsTarget`** and friends via **`POST /api/nfc/emulate-raw`** — you supply hex; **experimental** (bad sequences can require power cycle). |
+| **KeyLab** | **XOR span** across up to three 6-byte base keys (all combinations → candidates for Brute/Keys); **Shannon entropy** on arbitrary hex blobs. **Stage for Keys** copies candidates into **sessionStorage** for the Keys page. |
+| **Lib** | **Saved hex snippets** in localStorage; **seed** from built-in **lab catalog** samples; fixture loader for repeatable demos. |
+| **Keys** | **Browser-local** key dictionary (defaults + your lines); **merge lab keys**; **import Key Lab stash** from staged session. Used when you copy keys into Brute manually — **not** auto-uploaded to flash. |
+| **Raw** | Hex **command** payload → **`/api/raw/pn532`**; **general status** dump from **`/api/pn532/general-status`**; local TX/RX log. |
+| **Set** | **`apiBase`** (empty = same host), **theme** (light/dark), reminder of SoftAP + mDNS. |
+
+### Browser extras (no extra server)
+
+- **Top bar** — **Download log JSON**: last **5 000** WebSocket messages from **sessionStorage** (scan + capture + anything else the firmware sends).
+- **Scan feedback** — Visual/audio “hit” flourishes on new tag / capture events (debounced so it doesn’t spam).
+- **Validation fixtures** — Shared **lab** blobs and notes (`validationFixtures.ts`) for UI testing and teaching.
 
 ---
 
 ## Remote deep capture (how to use it)
 
-1. **Power on** → firmware starts **continuous scan** automatically (you can still toggle it on the Dashboard).
-2. **Read-all** (Capture page) → enable **passive read-all** (`POST /api/session/deep`) when you want automatic full dumps per new tag into device RAM.
+1. **Power on** → firmware starts **continuous scan** by default (toggle on the Dashboard).
+2. Open **Read-all (Capture)** → enable **passive read-all** (`POST /api/session/deep`).
 3. Present tags; each **new UID** appends one **NDJSON** line: full deep profile JSON.
 4. When the **progress bar hits the end**, scanning **pauses** automatically.
-5. Tap **Download NDJSON** on your phone (or laptop) — browser saves `pn532-deep-capture-*.ndjson`.
-6. Tap **Clear buffer** to free RAM and **resume** scanning.
+5. Tap **Download NDJSON** — browser saves `pn532-deep-capture-*.ndjson`.
+6. Tap **Clear buffer** to free RAM and **resume**.
 
-API (for automation):
+### Session API (automation)
 
 | Method | Path | Purpose |
 |--------|------|---------|
-| `GET` | `/api/status` | Includes `session.usedBytes`, `maxBytes`, `lines`, `full`, `deepCapture` |
-| `POST` | `/api/session/deep` | Body `{"enable":true}` — toggle deep capture |
-| `GET` | `/api/session/export` | Attachment: all captured lines |
-| `POST` | `/api/session/clear` | Wipe buffer, clear `full` |
+| `GET` | `/api/status` | App name, uptime, heap, Wi‑Fi mode, PN532 version, `scanning`, `session` (`usedBytes`, `maxBytes`, `lines`, `full`, `deepCapture`) |
+| `POST` | `/api/session/deep` | Body `{"enable": true}` or `false` |
+| `GET` | `/api/session/export` | `text/plain` / NDJSON attachment |
+| `POST` | `/api/session/clear` | Clears RAM buffer |
 
-WebSocket `/ws`: channels `scan` (inventory) and `capture` (recorded / bufferFull events).
+### Live NFC API
 
-### Browser memory log (nothing missed in the UI)
+| Method | Path | Purpose |
+|--------|------|---------|
+| `POST` | `/api/nfc/poll` | One-shot poll; body may be `{}` — returns `present`, optional `tag` |
+| `POST` | `/api/nfc/scan` | Body `{"enable": true}` or `false` — continuous background scan |
+| `GET` | `/api/pn532/general-status` | Raw PN532 **general status** byte array |
+| `POST` | `/api/mifare/read-block` | `block`, `key` (12 hex), `keyB` |
+| `POST` | `/api/mifare/write-block` | `block`, `key`, `keyB`, `data` (32 hex) |
+| `POST` | `/api/ul/read-page` | `page` |
+| `POST` | `/api/ul/write-page` | `page`, `data` (8 hex) |
+| `POST` | `/api/raw/pn532` | `frame` — hex command bytes |
+| `POST` | `/api/mifare/dictionary-attack` | `readerType` (`classic1k` / `classic4k`), `variations`, optional `keysHex[]`, optional `sectorFirst` / `sectorLast` |
+| `POST` | `/api/nfc/emulate-raw` | `hex` — raw command bytes (e.g. **0x8C** sequences) |
+| `POST` | `/api/ota` | **Not implemented** — returns **501** |
 
-The SPA keeps the **last 5 000** WebSocket events in **sessionStorage** (scan + capture + anything else the firmware pushes). Use the top bar **Download log JSON** to save a pretty file on your phone or PC without touching device RAM. Clear the log separately from the device capture buffer.
+**WebSocket:** `GET /ws` — JSON text frames as above; send **`ping`**, receive **`pong`**.
 
-### “Maximum sensitivity” (firmware)
+### Browser memory log
 
-On init the PN532 is configured for **high passive-activation retries** (`RFConfiguration` 0x05), and the poll loop runs at **~65 ms** when not doing deep capture — weak coupling / marginal tags get more chances to answer.
+The SPA keeps the **last 5 000** WebSocket events in **sessionStorage**. **Download log JSON** saves a pretty file on your phone or PC **without** using device RAM. Clear that log separately from the device capture buffer.
 
 ### Dictionary “brute” (Classic)
 
-**Brute** page → pick **Classic 1K / 4K map**, optional **variations**, paste extra keys. Firmware runs **`POST /api/mifare/dictionary-attack`**: built-in **public default keys** (subset of community lists such as the [Proxmark3 mfc_default_keys.dic](https://github.com/RfidResearchGroup/proxmark3/blob/master/client/dictionaries/mfc_default_keys.dic)) plus your lines, Key **A** then **B** per sector trailer. **Variations** add bounded XOR / nibble tweaks — **not** a full 2⁴⁸ keyspace search. You will **not** magically open every door; you will systematically try keys people actually leak online. ### Emulation
+**Brute** page → **Classic 1K / 4K**, optional **variations**, paste extra keys. Firmware runs **`POST /api/mifare/dictionary-attack`**: built-in **public default keys** (same family as community lists like [Proxmark3 `mfc_default_keys.dic`](https://github.com/RfidResearchGroup/proxmark3/blob/master/client/dictionaries/mfc_default_keys.dic)) plus your lines, Key **A** then **B** per sector trailer. **Variations** add bounded XOR / nibble-style tweaks — **not** a full 2⁴⁸ search. You won’t magically open every sector; you **systematically** try keys people actually leak in the wild.
 
-**Emulate** sends raw **`TgInitAsTarget` (0x8C)** payloads via **`POST /api/nfc/emulate-raw`**. You are responsible for correct bytes (NXP UM0701). This is **experimental**; bad sequences can require a power cycle.
+### Card emulation (PN532 target mode)
 
-| Method | Path | Purpose |
-|--------|------|---------|
-| `POST` | `/api/mifare/dictionary-attack` | Body: `readerType`, `variations`, optional `keysHex[]` |
-| `POST` | `/api/nfc/emulate-raw` | Body: `{ "hex": "8C..." }` |
-| `POST` | `/api/ul/write-page` | Body: `{ "page": 4, "data": "AABBCCDD" }` (8 hex) |
+**Emulate** sends raw **`TgInitAsTarget` (0x8C)** payloads via **`POST /api/nfc/emulate-raw`**. You own the bytes (see NXP **UM0701**). This is **experimental**; a bad sequence can leave the front end wedged until **power cycle**.
 
 ---
 
@@ -73,21 +124,11 @@ On init the PN532 is configured for **high passive-activation retries** (`RFConf
 ./start-firmware.sh
 ```
 
-What it does:
+What it does (see script for flags): bootstraps ESP-IDF under a **local prefix** if needed, builds the web UI into **`firmware/data`**, targets **esp32s3**, auto-detects serial when possible, **builds and flashes**.
 
-- bootstraps ESP-IDF locally if needed
-- builds the web UI into `firmware/data`
-- configures target `esp32s3`
-- auto-detects the serial port
-- builds and flashes the device
+Options include **`--build-only`**, **`--monitor`**, **`--port`**, etc.
 
-Useful options:
-
-- `./start-firmware.sh --build-only`
-- `./start-firmware.sh --monitor`
-- `./start-firmware.sh --port /dev/cu.usbmodemXXXX`
-
-### Web UI → flash image
+### Web UI → SPIFFS image
 
 ```bash
 cd web
@@ -100,35 +141,13 @@ npm run build:fw
 ```bash
 cd firmware
 idf.py set-target esp32s3
-idf.py menuconfig   # PN532 Host: SPI / I2C / UART + pins
+idf.py menuconfig   # PN532 Host: SPI / I2C / UART + pins; Board indicators (RGB) if you use DevKitC-1
 idf.py build flash monitor
 ```
 
-See [docs/FLASHING.md](docs/FLASHING.md) and [docs/PINOUT.md](docs/PINOUT.md).
+From `firmware/`, with IDF already in your environment, you can also **`./flash.sh`** (uses **`ESPPORT`** or the first argument as the serial device).
 
----
-
-## 9 meaningful upgrades we don’t have yet (roadmap)
-
-1. **FeliCa / Type B surfaces** — More first-class UI for non–Type A paths the PN532 can speak.
-
-2. **Configurable RAM budget + optional PSRAM** — Compile-time or NVS `maxBytes`, and external SPIRAM for **multi‑hundred‑KB** sessions on N8R8 modules.
-
-3. **Chunked / resumable export** — HTTP range or multipart export so **multi‑MB** captures don’t require one giant `httpd_resp_send`.
-
-4. **User-supplied key dictionary on device** — Upload common keys file to flash and run **automatic sector retries** without typing keys in the UI.
-
-5. **NDEF record editor** — Parse TLV/NDEF in the browser, edit records, write back through page/block APIs with lock-byte warnings.
-
-6. **ISO14443-4 / Type B automation** — Higher-layer APDU helpers where PN532 allows; clearer UI for “Type A only” vs “RATS/PPS” paths.
-
-7. **Wi-Fi STA onboarding** — Captive portal or dedicated SSID scan + save credentials to NVS (today defaults to AP).
-
-8. **Real OTA from the UI** — Replace `501` stub with signed `esp_https_ota` + rollback partition sanity checks.
-
-9. **Installable PWA + richer diagnostics** — `manifest.json`, icons, `theme-color`, and UI for **error histograms / timing** from the PN532 status path.
-
-10. **Magic UID / Gen2** — Not exposed; PN532 is a legitimate reader/writer, not a UID-spoofing modem.
+See [docs/FLASHING.md](docs/FLASHING.md) and [docs/PINOUT.md](docs/PINOUT.md). Limitations and scope notes: [docs/LIMITATIONS.md](docs/LIMITATIONS.md).
 
 ---
 
@@ -136,6 +155,7 @@ See [docs/FLASHING.md](docs/FLASHING.md) and [docs/PINOUT.md](docs/PINOUT.md).
 
 | Path | Role |
 |------|------|
-| `firmware/` | ESP-IDF: PN532 transport, NFC engine, **session RAM buffer**, **deep profile**, HTTP/WS |
-| `web/` | React UI: Dashboard, **Capture**, Read/Write, Library, Keys, Raw, Settings |
+| `firmware/` | ESP-IDF: PN532 transport, NFC engine, **session buffer**, **deep profile**, HTTP/WebSocket, SPIFFS |
+| `web/` | React UI: Dashboard, Capture, Read, Write, Brute, Emulate, KeyLab, Library, Keys, Raw, Settings |
 | `docs/` | Flashing, pinout, workflows, limitations |
+| `start-firmware.sh` | One-shot IDF bootstrap + build + flash |