Add operator validation ladder and Windows mining phase docs.
Document L0-L8 post-landing gates, feature slices, and honest gaps; link from README and tests README.
This commit is contained in:
AetherForge
105
docs/VALIDATION_LADDER.md
Normal file
105
docs/VALIDATION_LADDER.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# Validation ladder (L0-L8)
|
||||
|
||||
Operator guide for **what to run after code lands** - from zero-dependency mining smoke through live fleet work. Automated gates live in [`tests/README.md`](../tests/README.md); secure payload steps in [`docs/E2E_VALIDATION.md`](E2E_VALIDATION.md); Windows lifecycle phases in [`docs/WINDOWS_MINING_PHASES.md`](WINDOWS_MINING_PHASES.md).
|
||||
|
||||
**Mindset:** CI proves contracts and mocks; it does not prove Defender, domain GPO, or your AWS IAM. Climb the ladder until the change you shipped is covered at the lowest honest rung, then stop. [`PROBLEMS.md`](../PROBLEMS.md) lists what automation will never close.
|
||||
|
||||
---
|
||||
|
||||
## Ladder overview
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
L0["L0 - Mining only<br/>mine-validate, no C2"]
|
||||
L1["L1 - Fast unit slice<br/>test-suite -SkipE2E -SkipBuild"]
|
||||
L2["L2 - Full unit gate<br/>test-suite -SkipE2E"]
|
||||
L3["L3 - Dashboard E2E<br/>test-suite phases 1-8"]
|
||||
L4["L4 - Feature slices<br/>-Recon -S3Swarm -P2 ..."]
|
||||
L5["L5 - Docker mining proof<br/>ci-docker-mining"]
|
||||
L6["L6 - Isolated forge + VM<br/>e2e-validate Tier 3"]
|
||||
L7["L7 - Live spread / recon<br/>owned lab hosts"]
|
||||
L8["L8 - Cloud + IAM<br/>operator AWS account"]
|
||||
|
||||
L0 --> L1 --> L2 --> L3
|
||||
L2 --> L4
|
||||
L2 --> L5
|
||||
L5 --> L6
|
||||
L6 --> L7
|
||||
L7 --> L8
|
||||
```
|
||||
|
||||
| Rung | When | Command / action | Proves | Does not prove |
|
||||
|------|------|------------------|--------|----------------|
|
||||
| **L0** | RandomX / pool worker change | `cd agent; go run ./cmd/mine-validate -seconds 20 -threads 2` | CPU mining engine, pool login | C2, forge, spread, persistence |
|
||||
| **L1** | Quick post-edit smoke | `.\scripts\test-suite.ps1 -SkipE2E -SkipBuild` | Go server+agent + Vitest (phases 1-4) | Production builds, Playwright |
|
||||
| **L2** | Default post-landing gate | `.\scripts\test-suite.ps1 -SkipE2E` | Phases 1-7b, compile checks | Live browser E2E on `:18989` |
|
||||
| **L3** | UI / WS / routing change | `.\scripts\test-suite.ps1` | Full 8 phases incl. Playwright | Real Windows payload, live pool fleet |
|
||||
| **L4** | Single feature after parallel agents | `.\scripts\test-suite.ps1 -<Feature>` (see table below) | Targeted Go + Vitest (+ E2E when not `-SkipE2E`) | Cross-feature integration on real hosts |
|
||||
| **L5** | Mining + C2 regression in isolation | `.\scripts\ci-docker-mining.ps1` | Server healthy, Linux agent online, H/s > 0 | Windows-only tiers, GPU, spread |
|
||||
| **L6** | Forge / persistence / Crucible on Windows | `.\scripts\e2e-validate.ps1` + disposable Hyper-V VM | Full Windows agent on owned lab VM | Production wallet, internet-wide spread |
|
||||
| **L7** | Spread lanes, Deploy Recon, P2 | Manual lab: WinRM/GPO/BITS, SSRF probe paste, port scan to owned targets | Behavior on machines you own | CI coverage of third-party networks |
|
||||
| **L8** | S3 swarm, SSM, Fargate, fan-out | Operator AWS creds + IAM in your account | Real PutObject, SendCommand, ECS | Anything in shared repo CI |
|
||||
|
||||
Counts (refresh when drift): Go server **1007**, agent **680**, Vitest **867**, Playwright **32** - see [`tests/README.md`](../tests/README.md).
|
||||
|
||||
---
|
||||
|
||||
## Feature gates (L4 slices)
|
||||
|
||||
Use one slice when a parallel agent landing touched only that surface. Each exits after its phase unless noted.
|
||||
|
||||
| Feature | Command |
|
||||
|---------|---------|
|
||||
| **Full gate** | `.\scripts\test-suite.ps1` |
|
||||
| **Quick verify** | `.\scripts\test-suite.ps1 -SkipE2E` |
|
||||
| **Fast slice** | `.\scripts\test-suite.ps1 -SkipE2E -SkipBuild` |
|
||||
| **P2 mining/spread** | `.\scripts\test-suite.ps1 -P2` |
|
||||
| **Fleet recon** | `.\scripts\test-suite.ps1 -ReconOnly` |
|
||||
| **Deploy Recon** | `.\scripts\test-suite.ps1 -Recon` |
|
||||
| **Subnet recon** | `.\scripts\test-suite.ps1 -SubnetRecon` |
|
||||
| **S3Swarm** | `.\scripts\test-suite.ps1 -S3Swarm` |
|
||||
| **CloudMap** | `.\scripts\test-suite.ps1 -CloudMap` |
|
||||
| **Fargate** | `.\scripts\test-suite.ps1 -Fargate` |
|
||||
| **PolicyFanout** | `.\scripts\test-suite.ps1 -PolicyFanout` |
|
||||
| **SSM** | `.\scripts\test-suite.ps1 -SSM` |
|
||||
| **Seer** | `.\scripts\test-suite.ps1 -Seer` |
|
||||
| **Oath** | `.\scripts\test-suite.ps1 -Oath` |
|
||||
| **Contingency** | `.\scripts\test-suite.ps1 -Contingency` |
|
||||
| **CloudVenue** | `.\scripts\test-suite.ps1 -CloudVenue` |
|
||||
|
||||
With server already up: `.\scripts\smoke-test.ps1 -BaseUrl http://127.0.0.1:8989` (REST matrix B-01-B-10).
|
||||
|
||||
---
|
||||
|
||||
## Honest gaps (from PROBLEMS.md)
|
||||
|
||||
Automation closes regressions; these stay **operator or architecture** scope:
|
||||
|
||||
| Area | Honest limit |
|
||||
|------|----------------|
|
||||
| **By design** | `bof_execute` disabled; process hollowing ~50% ETW/Defender failure; non-Windows cloudflared stub; mesh without `-tags p2p`; Linux/macOS GPU RVN uses Windows miner binaries only. |
|
||||
| **Scale** | Subnet derived at query time; scan caps (128 hosts / spread sem 16); SQLite single-writer; WS `init` full fleet blob; 3D map capped at 200 nodes. |
|
||||
| **Container mining** | AV still sees Docker; GPU-in-container on Windows is operator-dependent; worker image build not auto-forged. |
|
||||
| **Deferred architecture** | `tunnel_stream`; Path Tracer RAM sessions; NAT without STUN; monolithic WS context; no CI HTTP forge without `LIVE_FORGE=1`. |
|
||||
| **AWS cloud** | S3/CloudFront/SSM/Fargate/EventBridge need **your** IAM; CI uses mocks. |
|
||||
| **Manual / live** | Live S3 signed magnets; SSM SendCommand; Fargate RunTask; fan-out Lambda; Cloud Map `route_via` server wiring; live P2 spread; Deploy Recon port scan/crawl from **dashboard host**; SSRF probe is manual paste. |
|
||||
|
||||
Full tables: [`PROBLEMS.md`](../PROBLEMS.md). Test gap pointers: [`tests/README.md`](../tests/README.md) (P2 spread lanes, gaps section).
|
||||
|
||||
---
|
||||
|
||||
## Operator checklist after landing
|
||||
|
||||
1. **L2** `.\scripts\test-suite.ps1 -SkipE2E` from repo root.
|
||||
2. If the change touched one feature row above, add **L4** slice.
|
||||
3. If forge/agent Windows behavior changed, plan **L6** VM snapshot test (wallet + `data-e2e\` only).
|
||||
4. If spread/recon/AWS changed, read PROBLEMS **Manual / live** before claiming production-ready.
|
||||
5. Update test counts in `tests/README.md` / `PROBLEMS.md` only when `go test -list` / Vitest / Playwright totals drift.
|
||||
|
||||
---
|
||||
|
||||
## Related docs
|
||||
|
||||
- [`docs/E2E_VALIDATION.md`](E2E_VALIDATION.md) - Tier 0-3 secure payload matrix, VM topology, Crucible checklist
|
||||
- [`docs/WINDOWS_MINING_PHASES.md`](WINDOWS_MINING_PHASES.md) - Windows boot -> recon -> deploy -> mining -> spread phase maps
|
||||
- [`tests/README.md`](../tests/README.md) - phase map, LOTL glossary, regression tables
|
||||
Reference in New Issue
Block a user