AetherForge/tests/README.md

# AetherForge Test Suite

One command runs everything:

```bat
test.bat
```

Or with PowerShell directly:

```powershell
.\scripts\test-suite.ps1
.\scripts\test-suite.ps1 -SkipE2E -SkipBuild
.\scripts\test-suite.ps1 -ReconOnly
```

## Phases

| Phase | What it runs | Location |
|-------|----------------|----------|
| 1 | Go server unit + integration tests | `server/` |
| 2 | Go agent tests | `agent/` |
| 3 | Fusion module unit tests + compile check | `fusion/` |
| 4 | Frontend unit tests (Vitest) | `server/web/` |
| 5 | Frontend production build | `server/web/` |
| 6 | Server binary compile | `server/` |
| 7 | Agent binary compile (Windows) | `agent/` → `bin/install-worker.exe` |
| 7b | Agent cross-compile (linux/darwin) | `agent/` → `bin/install-worker-*` |
| 8 | E2E smoke (Playwright) | `server/web/e2e/` — starts temp server on :18989 |

Phases 5–7 and 7b are skipped with `-SkipBuild`. Phase 8 is skipped with `-SkipE2E`.

`-ReconOnly` runs the fleet recon subset (vuln/CVE, cred graph, service graph, triple-onion gates, network hints, Path Tracer discover, recon UI Vitest) and exits — useful after parallel agent landings.

## Operator quick start (LOTL + fleet recon)

1. **Forge with LOTL Onion** — Forge → Operation mode → **LOTL Onion** (in-process RandomX, native-tool spread chain). Set your **XMR wallet** and forge once. With `lotl_policy_from_server` on (preset default), tier order comes from Calibrate `server.lotl_onion_tiers` on agent auth — **re-forge only when changing wallet, build, or preset flags**, not to reorder tiers.
2. **Probe & Join** — Crucible → select online node(s) → **Probe & Join** (`discover_and_join`). Agent runs service discovery, server signs a deploy plan, and the best LOTL lane executes. Risk/join-lane badges update on the next stats tick.
3. **Deployment credentials vault** — For cred-assisted spread (`spread_cred`, SMB/WinRM lanes), add profiles to `data/config.json`:

   ```json
   "deployment_credentials": [
     { "label": "Lab ops", "username": "corp\\\\ops", "vault_ref": "deployment-creds/lab.vault" }
   ]
   ```

   Store the password in `data/deployment-creds/<id>.vault` as plain text or `{"password":"..."}` (0600). Never commit vault files. Affinity ordering is covered by `TestOrderDeploymentCredProfiles_Affinity` and `TestLoadDeploymentCredPasswordFromVault`.

See `/docs/SPREAD_TECHNIQUES.html#lotl-onion` for the ten-tier chain and `scripts/test-suite.ps1 -ReconOnly` after landing agents.

## Run individual suites

```bat
cd server && go test ./...
cd agent && go test ./...
cd server\web && npm test
cd server\web && npm run test:e2e
```

### Fleet recon quick-run (P0 subset)

Matches `scripts/test-suite.ps1 -ReconOnly`:

```bat
cd server && go test ./internal/api/... -run "PathTracer|SpreadCred|MergeService|DeployPlan" -count=1
cd server && go test ./internal/db/... -run CredEdge -count=1
cd server && go test . -run "OrderDeploymentCred|LoadDeploymentCred" -count=1
cd agent && go test ./vulnprobe/... ./miner/... -run "TripleOnion|Correlate|VulnProbe|Risk" -count=1
cd agent && go test ./deploy/... -run "NetworkHints|ServiceDiscovery|CredSpread|Discover" -count=1
cd agent && go test ./client/... -run "Vuln|SpreadCred|ChainOrder" -count=1
cd server\web && npm run test -- --run src/help/reconRisk.test.ts src/components/Fleet/ReconBadges.test.tsx src/components/Fleet/CrucibleExpandedOps.test.tsx
```

## E2E only (server already running)

E2E credentials match `server/internal/api/integration_test.go` (`testuser` / `testpass`). Seed
`users.json` in the server data directory **before** first start, or the server generates a random
`admin` password instead.

```bat
set AETHERFORGE_E2E_USER=testuser
set AETHERFORGE_E2E_PASS=testpass
powershell -NoProfile -Command "[IO.File]::WriteAllText('data\\users.json','{\"testuser\":\"testpass\"}')"
set AETHERFORGE_URL=http://127.0.0.1:8989
cd server\web && npm run test:e2e
```

`test.bat` phase 8 seeds `users.json` automatically in a temp data dir. Override creds with
`AETHERFORGE_E2E_USER` / `AETHERFORGE_E2E_PASS` (used by Playwright, `test-suite.ps1`, and
`scripts/smoke-test.ps1`).

## Adding tests

- **Go:** `*_test.go` next to the code under test
- **Frontend:** `src/**/*.test.ts` (Vitest)
- **E2E:** `server/web/e2e/*.spec.ts` (Playwright)

All Go packages under `server/` and `agent/` are picked up automatically by `go test ./...` in
`scripts/test-suite.ps1`. Vitest discovers any `*.test.ts(x)` under `server/web/src/`.

## Priority tests (P0 / P1)

### P0 — security and command validation

| Test | What it validates | File | Phase |
|------|-------------------|------|-------|
| `TestIntegrationRouterCommandFullRoundTrip` | API `POST /command` → agent WS → `command_result` → dashboard WS | `server/internal/api/integration_test.go` | 1 |
| `TestAllowAgentWSUpgradeRateLimit` | 31st `/ws/agent` upgrade from same IP within 1 min rejected; empty IP allowed | `server/internal/api/agent_ws_limiter_test.go` | 1 |
| Crucible exec E2E | Online stub agent; **whoami**, terminal **echo**, and **LOTL tier badge** on `/crucible` | `server/web/e2e/crucible-command.spec.ts` | 8 |
| `TestPathForgeRootPathOutsideAllowedRoots` | PathForge `root_path` outside allowlist → HTTP 400, `Placed=0` | `server/internal/builder/pathforge_test.go` | 1 |
| `TestUploadCommandRejectsPathTraversal` | Agent `upload` blocks `../../` via `ResolveRemotePath` | `agent/client/client_upload_test.go` | 2 |

### P1 — additional hardening

| Test | What it validates | File | Phase |
|------|-------------------|------|-------|
| `TestDownloadCommandRejectsPathTraversal` | Agent `download` (read) blocks traversal paths like upload | `agent/client/client_upload_test.go` | 2 |

Run P0 Go tests quickly:

```bat
cd server && go test ./internal/api/... -run "TestIntegrationRouterCommandFullRoundTrip|TestAllowAgentWSUpgradeRateLimit" -count=1
cd server && go test ./internal/builder/... -run TestPathForgeRootPathOutsideAllowedRoots -count=1
cd agent && go test ./client/... -run "Upload|Download" -count=1
```


Run Crucible P0 E2E only (needs a live server on `AETHERFORGE_URL`, default `:8989`; `test-suite.ps1` phase 8 uses `:18989`):

```bat
set AETHERFORGE_E2E_USER=testuser
set AETHERFORGE_E2E_PASS=testpass
set AETHERFORGE_URL=http://127.0.0.1:8989
cd server\web && npx playwright test e2e/crucible-command.spec.ts
```

`e2e/remote-actions.spec.ts` mocks the dashboard WebSocket `init` payload (Crucible prefers live WS fleet data over REST). Playwright HTTP `page.route` alone cannot intercept WebSockets in this toolchain version. Asserts mining Pause/Resume in `.cop-mining` and bulk Pause in `.fleet-bulk-bar` when only an offline agent is selected.

Run remote-actions only (no stub agent; mocks offline fleet via WS):

```bat
set AETHERFORGE_E2E_USER=testuser
set AETHERFORGE_E2E_PASS=testpass
set AETHERFORGE_URL=http://127.0.0.1:8989
cd server\web && npx playwright test e2e/remote-actions.spec.ts
```

## Coverage map (recent features)

| Feature | Test file(s) | Suite phase |
|---------|----------------|-------------|
| P0 command round-trip (integration) | `server/internal/api/integration_test.go` | 1 |
| P0 agent WS rate limit | `server/internal/api/agent_ws_limiter_test.go` | 1 |
| P0 PathForge root rejection | `server/internal/builder/pathforge_test.go` | 1 |
| P0 Crucible exec + LOTL badge E2E | `server/web/e2e/crucible-command.spec.ts` | 8 |
| P0 upload path traversal (P1 download) | `agent/client/client_upload_test.go` | 2 |
| Cascading fallback chain | `agent/miner/fallback_chain_test.go` | 2 |
| Container mining / execution mode | `agent/miner/execution_test.go` | 2 |
| Mining diagnostics JSON + blockers | `agent/client/mining_diagnostics_test.go` | 2 |
| Mining chain order hooks | `agent/client/mining_chain_test.go` | 2 |
| Chain cooldown / inprocess skips container | `agent/miner/fallback_chain_test.go` | 2 |
| Stats batch WS coalescing | `server/internal/api/websocket_test.go` | 1 |
| Agents API pagination + subnet | `server/internal/db/agents_list_test.go`, `server/internal/api/handlers_test.go` | 1 |
| Mining status relay (`mining_status` / `mining_fallback`) | `server/internal/api/websocket_test.go` | 1 |
| Agent name preserved on reconnect | `server/internal/api/websocket_test.go` | 1 |
| `applyStatsUpdate` / `stats_batch` mining fields | `server/web/src/help/applyStatsUpdate.test.ts`, `wsStatsCoalesce.test.ts` | 4 |
| Fleet → Crucible redirect | `server/web/src/pages/AgentsPage.test.tsx`, `e2e/pages.spec.ts` | 4 / 8 |
| Crucible terminal `_seq` cursor | `server/web/src/pages/CruciblePage.test.tsx` | 4 |
| CrucibleAgentMeta / bulk toolbar | `CrucibleAgentMeta.test.tsx`, `CruciblePage.test.tsx` | 4 |
| Defender exclusion helper | `server/web/src/help/defenderExclusion.test.ts` | 4 |
| AV-Safe forge preset | `forgeOperationModes.test.ts`, `forgeMissionWizard.test.ts` | 4 |
| Forge progress polling | `server/web/src/pages/BuilderPage.test.tsx` | 4 |
| Emberwake pinA/pinB ref fix | `server/web/src/pages/EmberwakePage.test.tsx` | 4 |
| Mining status in AgentRemoteActions | `server/web/src/components/components.test.tsx` | 4 |
| PathTracerPage (12 tests) | `server/web/src/pages/PathTracerPage.test.tsx` | 4 |
| SystemStatusBar WS fleet count | `server/web/src/components/components.test.tsx` | 4 |
| WebSocket `stats_batch` handler | `server/web/src/context/WebSocketProvider.test.tsx` | 4 |
| Remote action wiring (`mining_diagnostics`) | `server/web/src/help/remoteActions.test.ts` | 4 |

### Fleet recon (2026-06-06 parallel agents)

| Feature | Test file(s) | Suite phase |
|---------|----------------|-------------|
| `vuln_findings` + CVE correlate | `agent/vulnprobe/scan_test.go`, `agent/client/vuln_scan_test.go`, `agent/client/cve_scan_test.go` | 2 |
| `cred_edges` + affinity spread | `server/internal/db/cred_edges_test.go`, `server/deployment_creds_test.go`, `server/internal/api/spread_cred_test.go`, `agent/client/spread_cred_test.go` | 1 / 2 |
| `service_graph` + join_lane mapping | `agent/deploy/service_discovery_test.go`, `server/internal/api/pathtracer_discover_test.go` | 1 / 2 |
| `discover_and_join` deploy plan | `server/internal/api/pathtracer_discover_test.go`, `agent/deploy/discover_join_test.go`, `service_deploy_test.go` | 1 / 2 |
| Triple onion gates (`patch_first`, `skip_mining_on_high_risk`) | `agent/miner/triple_onion_test.go` (`TestEvaluateTripleOnionGates*`, `TestTripleOnionOrchestrator*`), `server/internal/api/server_policy_test.go` | 1 / 2 |
| Deployment cred vault + affinity | `server/deployment_creds_test.go` (`TestOrderDeploymentCredProfiles_Affinity`, `TestLoadDeploymentCredPasswordFromVault`) | 1 |
| CVE / KEV client correlate | `agent/client/cve_scan_test.go`, `agent/client/vuln_scan_test.go` | 2 |
| Vuln catalog API | `server/internal/api/vuln_handler_test.go`, `server/internal/vuln/correlator_test.go` | 1 |
| `network_hints` (ARP, DNS SRV, cert) | `agent/deploy/network_hints_test.go` | 2 |
| Path Tracer API extensions (discover, spread, service merge) | `server/internal/api/pathtracer_handler_test.go`, `pathtracer_discover_test.go` | 1 |
| Risk badge + `reconRisk` helpers | `server/web/src/help/reconRisk.test.ts`, `ReconBadges.test.tsx` | 4 |
| Credential graph table (Spread tab) | `ReconBadges.test.tsx` (`CredentialGraphTable`) | 4 |
| Probe & Join (`discover_and_join`) | `CrucibleExpandedOps.test.tsx` (`Probe & Join` button → `discover_and_join`) | 4 |
| Crucible bulk pause/resume E2E | `server/web/e2e/crucible-bulk.spec.ts` | 8 |
| Fleet bulk actions hook | `server/web/src/hooks/useFleetBulkActions.test.ts` | 4 |
| War Room LOTL/join-lane telemetry | `server/web/src/help/warRoomTelemetry.test.ts` | 4 |
| Spread template export panel | `server/web/src/help/spreadTemplateExport.test.ts`, `SpreadTemplateExportPanel.tsx` | 4 |
| Service graph summary UI | `CrucibleExpandedOps.test.tsx` (mocked `ServiceGraphSummary`) | 4 |
| `vuln_findings` / `join_lane` WS stats merge | `applyStatsUpdate.test.ts`, `wsStatsCoalesce.test.ts` | 4 |
| Emberwake `join_lane` funnel tag | `ReconBadges.test.tsx` (`JoinLaneBadge`), `WarRoomFunnelBoard.tsx` | 4 |
| `vuln_probe` recon tier in mining chain | `agent/miner/tier_vuln_probe_test.go`, `mining_chain_test.go` | 2 |

### P1 — LOTL tiered mining (onion feature set)

| Test | What it validates | File | Phase |
|------|-------------------|------|-------|
| `TestSelectMiningTierChain*` | Diagnostics-driven tier chain order, AV/GPU/WSL pruning, force/skip tiers | `agent/miner/lotl_tier_test.go` | 2 |
| `TestTierOrchestrator*` | Sequential tier attempts, wallet parity, GPU addon gating, tier_report events | `agent/miner/lotl_orchestrator_test.go`, `lotl_tier_test.go` | 2 |
| `TestTryChainRunTierHooksPopulatesLOTLFields` | Fallback chain ↔ tier orchestrator integration | `agent/miner/fallback_chain_test.go` | 2 |
| `TestDefaultFallbackChain*` / launcher tests | powershell, dotnet, wsl, docker_load, container execution tiers | `agent/miner/*_launcher_test.go`, `fallback_chain_test.go` | 2 |
| `TestRunWMITier*` / `TestRunScheduledTaskTier*` / `TestRunGPUComputeTier*` / `TestRunWebView2Probe*` | Windows/Linux execution tiers with mocked binaries | `agent/miner/tier_*_test.go` | 2 |
| `TestAppendLinuxPyOpenCL*` | linux_pyopencl tier insertion | `agent/miner/pyopencl_test.go` | 2 |
| `TestApplyAuthLotlPolicy*` / `TestMiningTierPolicy*` | Server-pulled mining tier policy from auth | `agent/client/mining_policy_test.go` | 2 |
| `TestMiningDiagnostics*` / `TestInferMiningBlockers*` | Diagnostics JSON + tier chain fields + blockers | `agent/client/mining_diagnostics_test.go` | 2 |
| `TestChainOrderForConfig*` | Client mining chain order hooks | `agent/client/mining_chain_test.go` | 2 |
| `TestNormalizeLotlTiers*` / `TestTryLotlTier*` | Spread onion tier normalization + unix stub tiers | `agent/deploy/lotl_tiers_test.go`, `lotl_onion_stub_test.go` | 2 |
| `TestStagingRejectsPathTraversal*` / `TestVerifyFileSHA256*` | BITS/curl/certutil staging path hygiene + hash verify | `agent/deploy/staging_test.go` | 2 |
| `TestValidateUNCSpreadPath*` / `TestSMBUNCSvcName*` | SMB sc.exe spread helpers | `agent/deploy/smb_unc_spread_test.go` | 2 |
| `TestMiningStatusRelayCoalescedToStatsBatch` | `mining_hashrate`, `lotl_tier`, `lotl_attempts` in stats_batch | `server/internal/api/websocket_test.go` | 1 |
| `TestStatsBatchCoalescesSameAgent` | Same-agent coalesce preserves LOTL fields | `server/internal/api/websocket_test.go` | 1 |
| `TestAgentLotlFieldsJSONRoundTrip` | Agent model JSON exposes tier telemetry | `server/internal/models/agent_test.go` | 1 |
| `TestApplyLotlOnionPreset` / `TestNormalizeLotlOnionTiers` | Fusion/builder LOTL Onion preset | `server/internal/builder/lotl_onion_test.go` | 1 |
| `applyStatsUpdate` / `wsStatsCoalesce` LOTL fields | `lotl_tier`, `lotl_attempts`, `mining_hashrate` merge | `server/web/src/help/applyStatsUpdate.test.ts`, `wsStatsCoalesce.test.ts` | 4 |
| `LotlTierBadge` / `LotlAttemptsList` | Crucible tier badge + attempt list UI | `server/web/src/components/Fleet/LotlTierBadge.test.tsx` | 4 |
| `WebSocketProvider` stats_batch LOTL | Dashboard WS applies tier fields | `server/web/src/context/WebSocketProvider.test.tsx` | 4 |
| Forge LOTL Onion preset UI | `applyOperationMode('lotl_onion')` flags | `server/web/src/help/forgeOperationModes.test.ts` | 4 |
| LOTL onion tier docs | Ten-tier spread chain constants | `server/web/src/help/lotlOnionTiers.test.ts` | 4 |
| Fleet health bulk pause/resume | Bulk command framing + toolbar wiring | `server/internal/api/fleet_handler_test.go`, `components.test.tsx` | 1 / 4 |

Run LOTL Go tests quickly:

```bat
cd agent && go test ./miner/... ./client/... ./deploy/... -run "Lotl|LOTL|Tier|Staging|Fallback|Mining|PyOpenCL|WMI|WebView|GPUCompute|Scheduled|UNC" -count=1
cd server && go test ./internal/api/... ./internal/builder/... ./internal/models/... -run "Lotl|LOTL|Tier|StatsBatch|Mining" -count=1
cd server\web && npm test -- --run src/help/lotlOnionTiers.test.ts src/components/Fleet/LotlTierBadge.test.tsx src/help/applyStatsUpdate.test.ts src/context/WebSocketProvider.test.tsx
```

### Gaps (hard to unit-test)

- **Real Docker/Podman container start** — requires OCI runtime on host; covered by chain logic mocks only.
- **`DetectContainerRuntime` CLI probe** — depends on `exec.LookPath`; execution mode tests use `SetRuntimeDetector` inject instead.
- **Full agent `MiningChainRunner.Start` lifecycle** — needs live pool + optional GPU binary; hook order covered via `ChainController` tests.
- **Real WinRM/GPO/systemd/crontab spread execution** — requires elevated Windows domain or Linux init; tier stubs and normalization covered in deploy tests.
- **Real BITS/curl/certutil download** — network + OS tooling; staging path/hash logic covered in `staging_test.go`.
- **E2E Crucible lotl_tier badge** — covered in `crucible-command.spec.ts` (stub sends `lotl_tier` + `lotl_attempts` via WS `stats`; requires live server — phase 8 or `AETHERFORGE_URL`).
- **Playwright fleet recon flow** — Probe & Join and risk badges covered in Vitest; no full discover→spread E2E yet.

### Agent logs (not a missing API)

- **`get_log` command** — Fleet Roster → Remote Control → Fetch Log (or `GET /api/v1/agents/{id}/log?refresh=1` triggers the command and returns cached tail)
- **`upload_log` AI tool** — when AI autonomy is enabled, the agent reports log content via `/api/v1/agent/report` after an Ollama tool call

Unit tests cover `AgentRemoteActions` offline gating and mining live-stats in `components.test.tsx`; Playwright `e2e/remote-actions.spec.ts` mocks an offline agent and asserts disabled buttons.

### Frontend types (`types/index.ts`)

TypeScript interfaces in `server/web/src/types/` are compile-time contracts only — no runtime JSON schema guards. Validation lives in forms, forge preflight, and server-side handlers.