Autonomous Publishing System — full stack: orchestrator, 8 vertical sites, admin dashboard, analytics, cron pipeline

2026-08-03 21:44:26 -07:00
commit 493776b9f8
16 changed files with 2962 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,107 @@
+# Auto Publisher — README
+
+## Autonomous AI Publishing System
+
+A fully autonomous, self-hosted publishing platform that continuously discovers valuable topics, generates original useful content, and publishes it to a collection of evergreen authority websites.
+
+### Sites
+
+| Site | URL | Status |
+|------|-----|--------|
+| AI | https://ai.thetempleofdoom.com | 🚀 |
+| Tech | https://tech.thetempleofdoom.com | 🚀 |
+| Science | https://science.thetempleofdoom.com | 🚀 |
+| Crypto | https://crypto.thetempleofdoom.com | 🚀 |
+| Linux | https://linux.thetempleofdoom.com | 🚀 |
+| Gaming | https://gaming.thetempleofdoom.com | 🚀 |
+| DIY | https://diy.thetempleofdoom.com | 🚀 |
+| Guides | https://guides.thetempleofdoom.com | 🚀 |
+
+### Architecture
+
+```
+cron (6AM daily) → Trend Discovery → Topic Scoring → Research (ornith:latest)
+→ Multi-Agent Writing → SEO → Site Builder → Deploy to Proxmox CTs
+```
+
+### Quick Start
+
+```bash
+# Initialize database
+python3 core/orchestrator.py init
+
+# Discover trending topics
+python3 core/orchestrator.py discover
+
+# Run full pipeline (3 articles)
+python3 core/orchestrator.py run --max 3
+
+# Check status
+python3 core/orchestrator.py status
+
+# Admin dashboard
+python3 dashboard/app.py
+# → http://localhost:5106
+```
+
+### Infrastructure
+
+- **Orchestrator**: MacBook (cron via launchd)
+- **LLM Inference**: 
+  - MacBook: qwen3.5:4b (fast, cheap tasks)
+  - GamingPC RTX 3070: ornith:latest (quality writing/research)
+- **Sites**: 8 Proxmox LXC containers (nginx on each)
+- **Tunnels**: Cloudflare home tunnel (d2871458)
+- **Code**: Gitea (10.30.20.149:3000)
+- **Analytics**: Self-hosted, privacy-first
+
+### Directory Structure
+
+```
+auto-publisher/
+├── core/               # Orchestrator, deploy scripts
+├── services/           # Microservice APIs
+├── sites/              # Per-vertical static sites
+├── shared/             # CSS, JS, assets
+├── dashboard/          # Admin control panel
+├── analytics/          # Tracking & learning loop
+├── cron/               # Scheduled jobs
+└── docs/               # Architecture docs
+```
+
+### Pipeline Steps
+
+1. **Discover**: Scans HN, Reddit, GitHub, arXiv for trending topics
+2. **Score**: LLM evaluates popularity, competition, freshness, evergreen value
+3. **Assign**: Routes topics to appropriate vertical site
+4. **Research**: Deep research via ornith:latest (GamingPC), builds knowledge package
+5. **Write**: Multi-agent pipeline (outline → draft → edit → SEO → fact-check)
+6. **Build**: Generates static HTML, RSS, sitemap, JSON-LD
+7. **Deploy**: Pushes to Proxmox CT, restarts nginx
+8. **Learn**: Nightly analytics feedback loop improves future topic selection
+
+### LLM Strategy
+
+| Stage | Model | Host | Rationale |
+|-------|-------|------|-----------|
+| Topic scoring | qwen3.5:4b | MacBook | Fast pattern matching |
+| Research | ornith:latest | GamingPC | Deep reasoning, accuracy |
+| Writing | ornith:latest | GamingPC | Quality output |
+| Editing | qwen3.5:4b | MacBook | Fast iteration |
+| SEO | qwen3.5:4b | MacBook | Template-driven |
+| Fact-check | ornith:latest | GamingPC | Critical accuracy |
+
+### Environment Variables
+
+```bash
+# Optional overrides
+OLLAMA_MACBOOK=http://localhost:11434
+OLLAMA_GAMINGPC=http://10.30.20.186:11434
+DASHBOARD_SECRET=auto-publisher-2026
+```
+
+### Monitoring
+
+- Admin dashboard: http://localhost:5106
+- Pipeline logs: `core/orchestrator.log`
+- Analytics: per-CT at `:5199/a/stats`