Session 2.5: Qdrant recovery + fleet discoveries

This commit is contained in:
2026-06-16 00:05:14 -07:00
parent 196b124b5d
commit 4f5fac68cb

View File

@@ -1,25 +1,19 @@
## Session 2 — SSH Fleet Wiring (June 15, 2026)
## Session 2.5 — Service Recovery (June 15, 2026)
### Mission: Wire every host with key-based SSH auth
### Fixed ✅
| Service | Issue | Fix |
|---------|-------|-----|
| **Qdrant** (68:6333) | Binary needed GLIBC_2.38, CT has 2.35. + crash-looped 1,393 times | Downgraded to v1.12.5 + cleared corrupt data. Now serving ✅ |
| **Vikunja** (222) | Port listed as 80 but actually running on **3456** | ✅ Working, API responds |
| **Jellyfin** (88:8096) | Was listed at .86 (stale), actually at .88 via DHCP | ✅ Found and verified |
| **SSH fleet** | Key auth broken fleet-wide | Deployed id_ed25519, fixed n8n /root perms (777→755) |
### Results
🔑 Deployed `drjones@MBook-Air.local` ed25519 key to **10 hosts**
**10/10 BatchMode verified** — key-based auth working fleet-wide
### Discovered 📍
- Proxmox hosts **28 VMs/CTs** — 18 running, 10 stopped
- Key stopped services: C2builders(111), Langfuse(115), ArchiveBox(506), Pagodo(105)
- Key running: Jellyfin(538), qdrant(509/NOW), qbittorrent(106), SUPERkali(109), browserless(154)
### Issues Fixed
| Fix | Details |
|-----|---------|
| 🐛 n8n `/root` was 777 | Blocked StrictModes → publickey auth failed. Chmod'd to 755 |
| 🔑 Key deployment | Used sshpass + ssh-copy-id across all hosts |
| 🐛 n8n sshd was socket-activated | systemd managed port 22, sshd wasn't running as daemon |
### Fleet Credentials
- **User**: `root` on everything except Router (`drjones`)
- **Password**: `czapiewski` (primary), `Czapiewski1!` (alt)
- **SSH key**: `id_ed25519` deployed everywhere
### Next Session
- Fix Proxmox API token (now have SSH, can create one)
- Docker inspect on alive hosts
- Restart Qdrant + Vikunja services
- Check n8n workflows
### Current Fleet State
- **22 hosts** on the network
- **10 SSH-accessible** (all BatchMode)
- **3 dark** (150, 166, 27 - likely powered off)