drjones/AetherForge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1413a9394fa28e94f6b8d309baf69afaa3f80a2f
AetherForge/PROBLEMS.md
AetherForge feba06e008 Improve portable launch, forge persistence, and operator auth UX. 
Persist build extra_files for Build Manager history, print dashboard login on every start, add libp2p for Mesh P2P forge, defer WebSocket until login, and split devrun.bat from LAUNCH.bat with USB deck auto-detection.
2026-05-31 18:56:43 -07:00

24 KiB
Raw Blame History

Problems

Findings from systematic bug-hunt and test expansion (May 2026).

Verification: test.bat from project root (→ scripts/test-suite.ps1), or go test ./... in server/agent and npm test in server/web. Live API matrix: scripts/smoke-test.ps1 with server on :8989. AI handler only: cd server && go test ./internal/api/... -run AI -v.

Open

Critical / security

  • [HIGH] server/internal/api/fleet_handler.go — Remote code execution via authenticated API (powershell/exec/upload). By design — treat dashboard login as root.

Frontend production (server/web)

  • [LOW] server/web — Vite build still warns on three vendor chunk (~760 kB gzip ~201 kB). Split out of DashboardPage; inherent library size — revisit on next Three/R3F bump.
  • [LOW] server/web — npm audit fix (no --force, May 2026) applied 0 semver-safe patches; 6 remain (5 moderate, 1 critical). Critical: happy-dom@15.x (Vitest DOM env only — not shipped; fix needs happy-dom@20.9+, major bump). Moderate: esbuild@0.21.x via vite@5 → vitest/vite-node (dev-server SSRF — [GHSA-67mh-4wv8-2f99]; prod build unaffected; fix needs vite@8+, major bump). Do not run npm audit fix --force — also pulls Recharts v3 / React 19 transitives. Planned upgrade batch: Vite 8 + Vitest 3 + happy-dom 20 + Recharts 3 + React 18→19.
  • [LOW] server/web — Transitive three-mesh-bvh@0.7.8 deprecated vs pinned three@0.170 (drei pulls 0.7.x; needs 0.8.0+ on next Three/R3F bump). @types/three now pinned to 0.170.0 (aligned with runtime).

Untested packages (next coverage targets)

  • [LOW] server/internal/builder/ — Unit tests cover compile/fusion/disguise helpers (fake-go success/fail/cancel, compileWorker, buildFileFusion paired/embedded, spread-kit/fusion finish paths, manifest patching). Integration-only (needs real toolchain on host): successful agent/fusion go build/garble binaries, Windows PE disguise (go-winres, system icon extract), code signing (signtool/osslsigncode), Darwin .app bundle, full buildAgent/buildUniversalAgent happy-path end-to-end.
  • [LOW] agent/stats/ — Live kernel/sysctl/vm_stat sampling still integration-only; unit tests cover pure parsers (parseKB, parseMeminfo, parseVmStat*, filetimeToUint64, cpuBusyPercentFromDeltas).
  • [LOW] agent/deploy/ — Live SSDP multicast/SMB/SSH/cloudflared still integration-only; unit tests cover UPnP XML/URL/SOAP helpers, SSDP LOCATION parsing, passive-spread name/launcher helpers, and platform stubs (go test ./deploy/...).
  • [LOW] agent/client/ — Live WS/commands/posture probes still integration-only; unit tests cover pure helpers in client.go/ai.go (truncateStr, JSON types, job payload parsing, mock HTTP for /decide//heartbeat//report).
  • [LOW] agent/miner/ — RandomX hashing (engine.go), pool worker loop, and stratum TCP login/submit still integration-only; unit tests cover target/schedule helpers, stratum wire types, resource-limit guards, and hex/target math (no VM, no live TCP).
  • [LOW] server/web/src/types/ws.ts + server/internal/api/ws_types.go — WS payloads typed in two places; drift risk. Fixed May 2026 — testdata/ws_types_fixture.json golden file; Go ws_types_test.go reflects struct tags; TS ws.test.ts asserts same keys/shapes.

Documented / by design

Findings reclassified after code verification (May 2026). Not bugs — documented in README, tests/README, help text, or tests.

Was Resolution
[HIGH] Plaintext passwords in users.json users.json stores bcrypt hashes (cost 12). Legacy plaintext auto-migrates on startup and login via checkPassword in router.go. Documented in README Security + First-run login.
[HIGH] upload_log no dedicated ingest API By design: logs via get_log command / Fetch Log UI and AI upload_log tool reports. Documented in README Fleet Roster, tests/README, Field Guide tips.
[MEDIUM] Compact list / expand on click Implemented in AgentListItem.tsx (compact-row, click toggles expand). Documented in README Fleet Roster UX.
[MEDIUM] Remote actions disabled when offline Intentional — requires live WebSocket. Documented in README, settingHelp.ts, Field Guide tips. Vitest + Playwright e2e/remote-actions.spec.ts.
[MEDIUM] Fusion uses vendored go-winres Optional tool; devrun.bat installs, builder uses go run github.com/tc-hib/go-winres. Documented in README Forge section.
[LOW] SettingsPage = Calibrate / no CalibratePage Nav label Calibrate → route /settings → SettingsPage. Navigation table in README. (A11y htmlFor remains in Open.)
[LOW] types/index.ts no runtime guards Compile-time contracts only; comment block at top of file + tests/README Architecture note.
[LOW] No e2e for remote actions Added server/web/e2e/remote-actions.spec.ts (offline agent mock → disabled buttons). Vitest coverage in components.test.tsx.
[LOW] Mesh P2P needs p2p tag Forge adds -tags p2p when mesh enabled (compile.go); manual builds documented in README agent section.

Fixed (this session — dead code / wiring pass)

  • [LOW] server/web + server/internal/db — BuildRecord.extra_files persisted in SQLite (extra_files JSON column). Build Manager lists extra artifact downloads via api.buildArtifactUrl for historical builds, not only the post-forge strip on Forge.

  • [LOW] server/internal/api/router.go + server/web/src/api/client.ts — Documented agent-only POST /agent/{decide,report,heartbeat} (fleet-secret REST; intentionally omitted from dashboard client).

  • [LOW] server/web/src/types/index.ts — BuildRecord.download_url required in TS; added BuildExtraFile type. Build Manager uses stored download_url for primary download (ZIP/artifact paths).

  • server/web/src/api/client.ts — Added rotateFleetSecret() for POST /server/rotate-secret (was raw fetch in Calibrate only).

  • server/web/src/pages/SettingsPage.tsx — Fleet secret rotation uses api.rotateFleetSecret().

  • server/web/src/pages/BuilderPage.tsx — Wired blueprintDiff + compareBlueprint / blueprintName into a dismissible diff panel after blueprint load/import; extra forge artifacts use api.buildArtifactUrl on the last-build strip.

  • server/web/src/pages/BuildManagerPage.tsx — Primary download uses stored build.download_url when present (artifact/ZIP paths from universal/Fusion builds).

Fixed (this session — frontend production pass)

  • [LOW] server/web — Vite large-chunk warning for DashboardPage (~898 kB → ~33 kB): manualChunks splits three / recharts / vendor in vite.config.ts; lazy HashrateChart, FleetTopologyMap, and MatrixStreamOverlay in DashboardPage.tsx. npm run build PASS (remaining warning is isolated three chunk only).
  • [LOW] server/web — React Router v7 future flags (v7_startTransition, v7_relativeSplatPath) opted in via shared routerFuture in main.tsx and Vitest MemoryRouter wrappers; Vitest stderr warnings cleared.
  • [LOW] server/web/package.json — Moved @types/qrcode and @types/three to devDependencies (types-only; runtime deps unchanged). npm ci, npm run build, npm test -- --run, and depcheck all PASS (33 files / 372 tests).

Fixed (this session — May 2026 full pass)

  • [MEDIUM] server/web/src/pages/BuilderPage.tsx — Added useEffect cleanup on unmount that calls api.cancelBuild(cancelTokenRef.current) and sets batchCancelRef.current = true. Navigating away from the Forge page now cancels any in-progress server-side compile (M14 UI desync closed).
  • [MEDIUM] agent/miner/pool.go — Added atomic jobGen counter incremented in SetJob. Workers snapshot jobGen before each 256-nonce inner loop and break early when it changes, eliminating the "stale batch" window. Engine updates moved outside the pool write-lock (each Engine has its own RWMutex).
  • [LOW] server/internal/api/fleet_handler.go — xmrPriceCache moved from package-global vars (xmrPriceMu, xmrPriceCache) into FleetHandler struct fields (xmrPriceMu, xmrPriceCache). Multiple routers in one process no longer share a stale cache. Tests updated.
  • [LOW] server/config.go — LoadConfig now calls mergeConfigExplicit (with a key-presence map) instead of legacy mergeConfig. Boolean fields absent from a hand-edited config.json now keep DefaultConfig values rather than being zeroed on restart.
  • [LOW] server/internal/api/config_handler.go — Removed unused db *db.Database field from ConfigHandler and updated NewConfigHandler signature. All call sites updated (router_test.go, integration_test.go, config_handler_test.go, main.go).
  • [LOW] server/main.go — UpdateConfigFromJSON now validates semantic constraints before merging: port ranges 1–65535, pool port range, non-negative max_agents/stats_retention_hours/build_retention_days/max_build_size_mb. Invalid values return "invalid config: …" 400 without touching disk.
  • [LOW] server/internal/db/agent_meta.go — decodeTags now logs corrupt tag JSON via log.Printf instead of silently discarding it.
  • [LOW] server/web/src/pages/SettingsPage.tsx — All <label> elements paired with text inputs now carry htmlFor attributes matching corresponding id attributes on their inputs (33 label/input pairs). A11y issue closed.

Fixed (this session — May 2026 security pass)

  • [CRITICAL] server/internal/api/router.go — Build download/artifact/uninstall routes (/api/v1/builds/{id}/download, /artifact/, /uninstall) now require either X-Fleet-Secret (for agent self-upgrade) or Basic Auth. Removed unconditional public bypass. Tests updated: TestBasicAuthMiddlewareBuildDownloadRequiresAuth, TestRouterBuildDownloadAuth.
  • [CRITICAL] server/internal/api/router.go — Agent API paths (/api/v1/agent/*) now explicitly return 503 when fleet secret is not configured, rather than silently allowing unauthenticated access. Fleet secret is always auto-generated at first startup via main.go so this state should not occur in production.
  • [CRITICAL] server/internal/api/ai_handler.go — SSRF fixed: handleDecide no longer accepts or uses the caller-supplied ollama_endpoint to create a new engine. It now requires the engine to be pre-registered when the agent authenticates via WebSocket, returning 403 otherwise. Tests updated: TestAIHandleDecideSuccess, TestAIHandleDecideOllamaFailureFallback, renamed TestAIHandleDecideCreatesEngineOnFirstRequest → TestAIHandleDecideRejectsUnregisteredAgent.
  • [HIGH] agent/deploy/hollow_windows.go — Added bounds checks in rvaToFileOffset (section header array), relocation entry loop (2-byte entry boundary), and RunHollowed section-write loop (header and raw-data bounds). Prevents out-of-bounds panics on malformed/truncated PE payloads.
  • [HIGH] agent/deploy/autospread.go — StartAutoSpreader moved from unconditional startup in main.go to AgentClient.authenticate() behind a sync.Once. Lateral movement only begins after the server accepts the fleet secret, ensuring the agent is on an owned fleet. First-run spread marker also gated behind auth.
  • [LOW] server/internal/builder/platform.go — platformsForRequest universal + TargetArch: arm64 now returns ALL matching platforms (linux-arm64 and darwin-arm64), not just the first. Test TestPlatformsForRequestUniversalFilteredArch updated.
  • [LOW] server/internal/maintenance/retention.go — DB record is now deleted before artifact files (so failed DB deletes don't leave orphaned rows pointing to deleted files). os.RemoveAll errors are now logged instead of silently discarded.
  • [LOW] server/internal/maintenance/retention.go — StopRetentionJobs cancels the background retention loop; main.go defers it on exit. Test: TestStopRetentionJobs_StopsBackgroundLoop.

Fixed (prior session)

  • agent/deploy/ — Added natpunch_test.go (10), hollow_test.go, tunnel_test.go, autospread_test.go (2): UPnP XML/SOAP mocks, xmlEscape/getSubnet/intSliceStr, tunnel URL validation, autospread stub paths, hollow unavailable without -tags hollow.
  • agent/client/ — Added client_test.go (8), listen_ports_test.go (2), dns_config_test.go (3), posture_windows_test.go (5), listen_ports_parse_test.go (5, !windows): server URL list/WS URL builders, log tail, listen-port/patch JSON, DNS JSON parser, ss/netstat parsers, Windows posture JSON helpers.
  • agent/client/client.go — readLogTail ignored trailing newline when counting lines; tail_lines=2 on a 4-line log with final \n returned only "line4\n" instead of last two content lines.
  • agent/stats/ — Added reporter_test.go (4 smoke tests), reporter_linux_test.go (parseKB, linux tag).
  • server/internal/models/ — Added agent_test.go (10 tests): JSON round-trips for all exported structs; omitempty/minimal decode.
  • server/internal/ollama/ — Added engine_test.go (14 tests): NewEngine defaults, type JSON round-trips, mock decide/health paths, markdown JSON extraction, error branches.
  • server/internal/sys/ — Added firewall_test.go (2 tests): invalid port; non-Windows stub error.
  • server/internal/alerts/ — Added notify_test.go (6 tests): Telegram/email no-op paths, SMTP defaults, NotifyAll no-panic.
  • server/internal/pool/ — Added manager_test.go (8 tests): validation, poolKey, status levels, setters, ListStatus.
  • server/internal/db/sqlite.go — SetPinnedBuild returns error when id not found; TestSetPinnedBuildUnknownID.
  • agent/client/ — Added protocol_test.go, posture_types_test.go, resource_pressure_test.go (25 tests).
  • agent/client/client.go — Empty "error" in job payload no longer treated as server error.
  • agent/config/ — Added schedule_test.go (6 tests): mining mode, clock parse, schedule windows.
  • agent/deploy/ — Added common_test.go, identity_test.go (18 tests): naming, install paths, agent ID lifecycle.
  • agent/job/ — Added job_test.go (2 tests): JSON round-trip.
  • server/internal/api/websocket.go — checkDashboardWSToken compared plain password to bcrypt hash; dashboard WS auth failed after user migration. Now uses checkPassword.
  • server/internal/api/ — Added unit/integration tests for remaining handlers: handlers.go (agent/build REST), router.go (auth middleware, users, rotate-secret, SPA/dropper routes), websocket.go (agent/dashboard WS, fleet secret, max agents, log tail), dropper_handler.go, blueprint_handler.go, ws_types.go. New files: router_test.go, dropper_handler_test.go, blueprint_handler_test.go, websocket_test.go, ws_types_test.go; expanded handlers_test.go. go test ./internal/api/... — 159 tests PASS.
  • server/web/src/components/ — Added components.test.tsx (57 tests) covering all 22 component TSX modules (NeonCard, HelpTip, downloads, ErrorBoundary, SessionGate, charts, fleet panels/toolbar/list/remote actions, forge hints, visual widgets, layout, ambient/matrix/cursor). Vitest environmentMatchGlobs includes src/components/**.
  • server/web/src/pages/AgentsPage.test.tsx — AgentRemoteActions mocked to avoid live listBuilds / ECONNREFUSED :3000 in detail-panel tests.
  • server/web/src/api/client.ts — fetchJSON spread ...options after merged headers could drop Content-Type and Authorization when callers pass options.headers; headers now merged after rest spread.
  • server/web/src/api/ — Added client.test.ts (20) and download.test.ts (6): paths, query params, auth headers, FormData fusion builds, error bodies. Expanded auth.test.ts (+1 sessionStorage throw path).
  • server/web/src/context/ — Added WebSocketContext.test.tsx (2), WebSocketProvider.test.tsx (8), ForgeContext.test.tsx (4): mock WebSocket connect URL/token, message handlers, _seq ring buffer, reconnect timer, forge state machine.
  • server/web/src/pages/BuilderPage.tsx — Load failure no longer stuck on “Loading forge defaults…” when form is null; error message shown instead. Wallet placeholder/short-wallet hint aligned to 90–106 chars. Exported formatBytes helper.
  • server/web/src/pages/SettingsPage.tsx — Wallet placeholder aligned to 90–106 chars. Exported deepMerge helper (config import).
  • server/web/src/pages/ — Added BuilderPage.test.tsx (13) and SettingsPage.test.tsx (11, Calibrate UI at /settings). Page suite now 4 files / 46 tests.
  • server/web/src/test/fixtures.ts — Added mockServerConfig() for page/API tests.
  • server/internal/api/config_handler.go — PUT errors return valid JSON; invalid config: maps to HTTP 400; GET sets explicit 200.
  • server/config.go — mergeConfigExplicit tracks nested key presence; partial PUT {"server":{"dashboard_subtitle":"x"}} no longer resets sibling booleans (H14 nested shallow-merge).
  • server/internal/api/config_handler_test.go — 10 handler unit tests (GET/PUT, 405, invalid JSON, 400/500 paths, JSON escaping).
  • server/config_test.go — 9 mergeConfigExplicit regression tests (partial PUT, nested merge, defaults, bool false, fallback).
  • server/internal/maintenance/ — Added retention_test.go (12 tests): StartRetentionJobs no-op/disabled, immediate run, 6h tick interval, stats/build purge via temp sqlite + filesystem, zero-retention skips, closed-DB error logs, combined stats+builds pass. Coverage ~97%. Exported retentionTickInterval + runRetentionFn hooks for testability only.
  • server/web/src/pages/AgentsPage.tsx — Bulk command errors now alert user (parity with Dashboard B13).
  • server/web/src/pages/DashboardPage.tsx — Share log table uses composite React key when share.id absent; exported formatShareTime helper.
  • server/web/src/pages/AgentsPage.tsx — listAgents no longer overwrites live WS agent list when socket already connected (isConnectedRef guard).
  • server/web/src/pages/ — Added DashboardPage.test.tsx (11) and AgentsPage.test.tsx (12); vitest config extended for .tsx + @testing-library/react.
  • server/internal/db/retention.go — ListBuildsOlderThan used a partial column list; now uses buildSelectCols + scanBuild for consistent full BuildRecord fields.
  • server/web/e2e/smoke.spec.ts — E2E login used hardcoded drjones/czapiewski; now reads AETHERFORGE_E2E_USER/AETHERFORGE_E2E_PASS via e2e/fixtures.ts (defaults testuser/testpass, matching integration_test.go). test-suite.ps1 seeds BOM-free users.json before E2E server start; smoke-test.ps1 defaults updated.
  • server/web/src/help/fleetAnalytics.ts — contributionBars included offline agents in total hashrate denominator, skewing contribution percentages on the dashboard.
  • server/web/src/pages/SettingsPage.tsx — Access Control help text still referenced removed default credentials; updated to describe first-run console password.
  • server/internal/api/ai_handler_test.go — Expanded unit tests for HandleDecide, HandleReport, HandleHeartbeat, engine lifecycle, numeric constants (1000 report cap, 60s heartbeat, 120-char reasoning truncate), Ollama-failure sleep fallback, event broadcaster, recordActivity merge.
  • server/internal/api/fleet_handler_test.go — Unit tests for all exported FleetHandler methods (GetAlerts, GetPoolStatus, GetAIActivity, GetXMRPrice, GetEarnings/GetEarningsEstimate, GetAgentLog, PostAgentCommand, PutAgentMeta, PostBulkCommand), EstimateXMRPerDay/parseFloatQuery, earnings/XMR price cache TTLs, SupportXMR field normalization, HTTP error branches (503/502/400), and WS command paths via mock transport + test agent WS.
  • server/web/src/help/forgeCompatibility.ts — Wallet preflight message said length 95–106 but validator accepts 90–106; message aligned with looksLikeXMRWallet().
  • server/web/src/help/ — Added/expanded vitest coverage: forgeCompatibility.test.ts (37), forgeRules.test.ts (46), settingHelp.test.ts (8).
  • server/web/src/help/buildManager.test.ts — 9 tests: blueprintDiff (added/removed/changed, sort, nested, arrays, empty), buildRequestFromRecord merge/override.
  • server/web/src/help/cheatSheetContent.test.ts — 19 tests: pipeline/network/fusion/AI guides, FORGE_VS_CALIBRATE, TROUBLESHOOTING, ROADMAP_FEATURES, CHEAT_SECTIONS registry.
  • server/web/src/help/forgeDefaults.test.ts — 7 tests: FORGE_BUILD_DEFAULTS shape, forgeDefaultsFromServer public URL / pool / sign / obfuscate.
  • server/web/src/help/remoteActions.test.ts — Expanded to 11 tests: aggressiveActionHint, spread/mesh gating, legacy undefined caps.
  • server/web/src/help/cheatSheetContent.ts — Troubleshooting "Shares all rejected" wallet text aligned to 90–106 chars (was stale "95 chars").
  • server/internal/builder/ — Added unit tests across compile, disguise, fusion media, polymorph, limits, media lock, handler HTTP/cancel, spread-kit helpers (~110 tests). Fixed wallet validation error text (90–106 chars). go test ./internal/builder/... — PASS.
  • server/web/src/help/settingHelp.ts — calibrate_wallet / wallet help aligned to 90–106 chars (was stale "~95 characters"); settingHelp.test.ts assertions.
  • server/web/src/components/Charts/GaugeRing.tsx — Center label now uses clamped value (matches SVG arc 0–100%); components.test.tsx updated.
  • server/web/src/pages/AgentsPage.tsx — Fleet Roster FleetToolbar wired with onSelectAllFiltered / filteredCount (Dashboard parity).
  • server/web/src/api/client.ts — estimateFusion client-side prep-file guard (parity with buildAgent); client.test.ts reject test.
  • agent/miner/ — Added stratum_test.go (8), pool_test.go (8); expanded target_test.go (+6), schedule_test.go (+3). 29 tests PASS — endpoints, Stratum JSON wire types, nonce hex, difficulty/target math, schedule guard.
  • agent/miner/schedule.go — MiningModeNormalized() "schedule" was treated as always-on; now accepts "scheduled" and "schedule"; allowedAt for deterministic tests.
  • server/web — Pinned @types/three@0.170.0 to match three@0.170.0 (was ^0.184.1, skewed from runtime; skipLibCheck masked). @react-three/drei@9.122 / @react-three/fiber@8.18 unchanged; tsc, npm run build, vitest (373) — PASS.

Fixed (earlier passes)

See git history and prior audit IDs (B1–B42, C1–C6, H1–H8, etc.) in README / tests/README.md.

Recommended next section

  1. agent/stats/ + agent/deploy/ integration paths — platform reporters, autospread/hollow
  2. Agent WS token auth (S2) — security hardening

Test run snapshot (this session)

Suite Result
server/internal/api/... (full) PASS
server Go tests PASS (all packages)
agent Go tests PASS (full ./...)
server/web vitest (full suite) PASS — 33 files, 373 tests
server/web npm run build (tsc + vite) PASS
server/web npm ci + depcheck PASS — no missing/unused deps
server/web Playwright e2e PASS — 5 tests

Backend dependency audit (May 2026)

Module go mod tidy go build ./... go test ./... Notes
server/ Updated PASS PASS Promoted golang.org/x/crypto to direct (bcrypt in router.go). Removed stale indirects (go-winres, nfnt/resize) — winres stays go run at build time per README.
agent/ Updated PASS PASS Promoted github.com/libp2p/go-libp2p, github.com/google/uuid to direct (mesh_p2p.go / p2p tag). go build -tags p2p and -tags hollow PASS on Windows.
fusion/ No dep changes PASS N/A (no _test.go) Stdlib-only module; matches test-suite.ps1 phase 3/8.

Dead/orphan code: None removed. All agent/server/fusion .go files belong to wired packages; build-tag stubs (mesh_p2p_stub, hollow_stub_*, platform splits) are intentional feature gates, not duplicates.

Fix applied: fleet_handler_test.go nil-WS cases used struct value-copy (bad := *fh) which tripped go vet (mutex copy). Replaced with NewFleetHandler(..., nil, ...).

Reference in New Issue View Git Blame Copy Permalink