android-fleet-godhead v1: fleet control node + ProxyFly engine + dashboard

2026-08-13 16:41:45 -07:00
commit 01bd139af7
10 changed files with 2000 additions and 0 deletions
--- a/docs/README.md
+++ b/docs/README.md
@@ -0,0 +1,73 @@
+# Android Fleet Godhead — Operator Manual
+
+Dashboard: http://10.30.20.31:8080 · CT704 · systemd: android-fleet
+
+## What it is
+Master control node for the Android VM fleet. Clones Android-x86 VMs from template 1301 on demand. Each clone gets a random MAC → random DHCP IP from the LAN router → auto-registers via adb (:5555) → gets a unique egress proxy from the NordVPN bank. Full live telemetry in the dashboard.
+
+## Quick commands
+
+```bash
+# spawn 3 fleet members
+curl -X POST http://10.30.20.31:8080/api/spawn -H "Content-Type: application/json" -d '{"count":3,"name_prefix":"fleet"}'
+
+# watch the job
+curl -s http://10.30.20.31:8080/api/spawn/jobs
+
+# list devices
+curl -s http://10.30.20.31:8080/api/devices
+
+# shell (whitelisted cmds)
+curl -X POST http://10.30.20.31:8080/api/device/<MAC>/shell -H "Content-Type: application/json" -d '{"cmd":"getprop ro.build.fingerprint"}'
+
+# screenshot
+curl -X POST http://10.30.20.31:8080/api/device/<MAC>/screenshot -o shot.png
+
+# reboot / kill (kill = destroy VM + release proxy)
+curl -X POST http://10.30.20.31:8080/api/device/<MAC>/reboot
+curl -X POST http://10.30.20.31:8080/api/device/<MAC>/kill
+
+# add proxy
+curl -X POST http://10.30.20.31:8080/api/proxies -H "Content-Type: application/json" -d '{"name":"nord-tokyo","host":"10.30.20.154","port":1080,"proto":"socks5"}'
+
+# assign proxy to device
+curl -X POST http://10.30.20.31:8080/api/proxies/4/assign -H "Content-Type: application/json" -d '{"mac":"bc:24:11:cb:6a:f7"}'
+
+# direct adb (from CT704)
+pct exec 704 -- adb connect 10.30.20.195:5555
+pct exec 704 -- adb -s 10.30.20.195:5555 shell "pm list packages"
+```
+
+## How pieces connect
+```
+Proxmox (clone/start/destroy via PVE API token)
+  ↕
+CT704 godhead (Flask :8080 + SQLite + adb + ws)
+  ├─ scan 10.30.20.0/24:5555 every 10s ─▶ new devices auto-register
+  ├─ adb metrics every 15s (cpu/mem/battery/net)
+  ├─ proxy health every 60s → auto-assign → settings put global http_proxy on device
+  └─ WS push → dashboard live cards + charts
+```
+
+## How AI agents drive it
+- Agents = Hermes cron jobs on local GPU Ollama (light_reaper granite4.1:3b / shadow-death gemma4:26b). NEVER MacBook models, never LLMs on CTs.
+- Pattern: agent reads /api/devices → picks device(s) → calls shell/screenshot/spawn endpoints → acts (e.g. install APK, run app, tap via `input` — extend the whitelist regex in app.py for new cmds).
+- MCP: add a remote HTTP MCP in Hermes config (url http://10.30.20.31:8080/mcp) wrapping the same endpoints when needed.
+
+## Troubleshooting
+| Symptom | Fix |
+|---|---|
+| Device offline | `pct exec 704 -- adb connect <ip>:5555` manually; check VM status `qm status <vmid>`; reboot via VM |
+| Never appears | DHCP lease? check router; ensure adb port open: `nc -z -w2 <ip> 5555` |
+| Spawn job error | `curl /api/spawn/jobs` — free-vmid scan 1310+; check Proxmox storage space (local dir, full clone = 12GB each) |
+| Proxy dead | `pct exec 680 -- gost` is the process; check nordvpn: `pct exec 680 -- nordvpn status`; then health loop auto-recovers |
+| No egress IP shown | proxy assigned but check failed — verify from CT704: `curl -x socks5h://10.30.20.71:1080 https://api.ipify.org` |
+| Boot hang at "Detecting Android-x86" | system.sfs rebuilt with wrong compression — MUST be gzip/128K blocks |
+
+## Template surgery (when updating the image)
+1. Pick a known-good running clone. 2. `qm stop <vmid>`. 3. qemu-nbd map its disk (NOT the base image — templates are immutable). 4. Extract /android-9.0-r2/system.sfs → unsquashfs → loop-mount system.img → edit build.prop / etc/init.sh → umount → `mksquashfs -comp gzip -b 131072`. 5. Write back, unmount, disconnect. 6. `qm destroy 1301 --purge` → `qm clone <vmid> 1301 --full` → set args (ip=dhcp, -vnc 5961) → `qm template 1301`.
+
+## Backups / state
+- /opt/android-fleet/{fleet.db, config.json} on CT704 — copy both before any surgery.
+- Template base: /var/lib/vz/images/1301/base-1301-disk-1.qcow2 (immutable, don't hand-edit).
+- Docs: Obsidian Hermes/android-fleet-godhead.md · fleet db mirrors to Teable if wired.