Autonomous Publishing System — full stack: orchestrator, 8 vertical sites, admin dashboard, analytics, cron pipeline

README.md

@@ -0,0 +1,107 @@
+# Auto Publisher — README
+
+## Autonomous AI Publishing System
+
+A fully autonomous, self-hosted publishing platform that continuously discovers valuable topics, generates original useful content, and publishes it to a collection of evergreen authority websites.
+
+### Sites
+
+| Site | URL | Status |
+|------|-----|--------|
+| AI | https://ai.thetempleofdoom.com | 🚀 |
+| Tech | https://tech.thetempleofdoom.com | 🚀 |
+| Science | https://science.thetempleofdoom.com | 🚀 |
+| Crypto | https://crypto.thetempleofdoom.com | 🚀 |
+| Linux | https://linux.thetempleofdoom.com | 🚀 |
+| Gaming | https://gaming.thetempleofdoom.com | 🚀 |
+| DIY | https://diy.thetempleofdoom.com | 🚀 |
+| Guides | https://guides.thetempleofdoom.com | 🚀 |
+
+### Architecture
+
+```
+cron (6AM daily) → Trend Discovery → Topic Scoring → Research (ornith:latest)
+→ Multi-Agent Writing → SEO → Site Builder → Deploy to Proxmox CTs
+```
+
+### Quick Start
+
+```bash
+# Initialize database
+python3 core/orchestrator.py init
+
+# Discover trending topics
+python3 core/orchestrator.py discover
+
+# Run full pipeline (3 articles)
+python3 core/orchestrator.py run --max 3
+
+# Check status
+python3 core/orchestrator.py status
+
+# Admin dashboard
+python3 dashboard/app.py
+# → http://localhost:5106
+```
+
+### Infrastructure
+
+- **Orchestrator**: MacBook (cron via launchd)
+- **LLM Inference**: 
+  - MacBook: qwen3.5:4b (fast, cheap tasks)
+  - GamingPC RTX 3070: ornith:latest (quality writing/research)
+- **Sites**: 8 Proxmox LXC containers (nginx on each)
+- **Tunnels**: Cloudflare home tunnel (d2871458)
+- **Code**: Gitea (10.30.20.149:3000)
+- **Analytics**: Self-hosted, privacy-first
+
+### Directory Structure
+
+```
+auto-publisher/
+├── core/               # Orchestrator, deploy scripts
+├── services/           # Microservice APIs
+├── sites/              # Per-vertical static sites
+├── shared/             # CSS, JS, assets
+├── dashboard/          # Admin control panel
+├── analytics/          # Tracking & learning loop
+├── cron/               # Scheduled jobs
+└── docs/               # Architecture docs
+```
+
+### Pipeline Steps
+
+1. **Discover**: Scans HN, Reddit, GitHub, arXiv for trending topics
+2. **Score**: LLM evaluates popularity, competition, freshness, evergreen value
+3. **Assign**: Routes topics to appropriate vertical site
+4. **Research**: Deep research via ornith:latest (GamingPC), builds knowledge package
+5. **Write**: Multi-agent pipeline (outline → draft → edit → SEO → fact-check)
+6. **Build**: Generates static HTML, RSS, sitemap, JSON-LD
+7. **Deploy**: Pushes to Proxmox CT, restarts nginx
+8. **Learn**: Nightly analytics feedback loop improves future topic selection
+
+### LLM Strategy
+
+| Stage | Model | Host | Rationale |
+|-------|-------|------|-----------|
+| Topic scoring | qwen3.5:4b | MacBook | Fast pattern matching |
+| Research | ornith:latest | GamingPC | Deep reasoning, accuracy |
+| Writing | ornith:latest | GamingPC | Quality output |
+| Editing | qwen3.5:4b | MacBook | Fast iteration |
+| SEO | qwen3.5:4b | MacBook | Template-driven |
+| Fact-check | ornith:latest | GamingPC | Critical accuracy |
+
+### Environment Variables
+
+```bash
+# Optional overrides
+OLLAMA_MACBOOK=http://localhost:11434
+OLLAMA_GAMINGPC=http://10.30.20.186:11434
+DASHBOARD_SECRET=auto-publisher-2026
+```
+
+### Monitoring
+
+- Admin dashboard: http://localhost:5106
+- Pipeline logs: `core/orchestrator.log`
+- Analytics: per-CT at `:5199/a/stats`

analytics/analytics.py

@@ -0,0 +1,190 @@
+"""
+Analytics & Learning Loop
+Tracks content performance and feeds insights back into topic selection.
+"""
+import sqlite3
+import json
+import time
+from pathlib import Path
+from datetime import datetime, timedelta
+from typing import Optional
+import logging
+
+BASE_DIR = Path(__file__).resolve().parent.parent
+DB_PATH = BASE_DIR / "core" / "publisher.db"
+
+log = logging.getLogger("analytics")
+
+
+def record_pageview(article_id: int, vertical: str, referrer: str = "",
+                    user_agent: str = "", ip_hash: str = "") -> None:
+    """Record a pageview for an article."""
+    db = sqlite3.connect(str(DB_PATH))
+    
+    # Update or insert analytics row for today
+    today = datetime.now().strftime("%Y-%m-%d")
+    existing = db.execute(
+        "SELECT id, pageviews, unique_visitors FROM analytics WHERE article_id = ? AND recorded_at = ?",
+        (article_id, today)
+    ).fetchone()
+    
+    if existing:
+        db.execute(
+            "UPDATE analytics SET pageviews = pageviews + 1 WHERE id = ?",
+            (existing[0],)
+        )
+    else:
+        db.execute(
+            "INSERT INTO analytics (article_id, vertical, pageviews, unique_visitors, recorded_at) "
+            "VALUES (?, ?, 1, 1, ?)",
+            (article_id, vertical, today)
+        )
+    
+    db.commit()
+    db.close()
+
+
+def get_article_performance(days: int = 30) -> list[dict]:
+    """Get performance data for all articles in the last N days."""
+    db = sqlite3.connect(str(DB_PATH))
+    db.row_factory = sqlite3.Row
+    
+    rows = db.execute("""
+        SELECT a.id, a.title, a.vertical, a.slug, a.word_count,
+               COALESCE(SUM(an.pageviews), 0) as total_views,
+               COALESCE(SUM(an.unique_visitors), 0) as total_visitors,
+               COUNT(DISTINCT an.recorded_at) as days_tracked
+        FROM articles a
+        LEFT JOIN analytics an ON a.id = an.article_id
+        WHERE a.status = 'published'
+          AND (an.recorded_at >= date('now', ?) OR an.recorded_at IS NULL)
+        GROUP BY a.id
+        ORDER BY total_views DESC
+    """, (f"-{days} days",)).fetchall()
+    
+    db.close()
+    return [dict(r) for r in rows]
+
+
+def get_vertical_performance(days: int = 30) -> dict:
+    """Get aggregate performance per vertical."""
+    db = sqlite3.connect(str(DB_PATH))
+    db.row_factory = sqlite3.Row
+    
+    rows = db.execute("""
+        SELECT a.vertical,
+               COUNT(DISTINCT a.id) as article_count,
+               COALESCE(SUM(an.pageviews), 0) as total_views,
+               COALESCE(AVG(an.pageviews), 0) as avg_views_per_article,
+               AVG(a.word_count) as avg_word_count
+        FROM articles a
+        LEFT JOIN analytics an ON a.id = an.article_id
+        WHERE a.status = 'published'
+          AND (an.recorded_at >= date('now', ?) OR an.recorded_at IS NULL)
+        GROUP BY a.vertical
+        ORDER BY total_views DESC
+    """, (f"-{days} days",)).fetchall()
+    
+    db.close()
+    return {r["vertical"]: dict(r) for r in rows}
+
+
+def run_learning_loop() -> dict:
+    """Nightly analysis: learn what works and update topic scoring."""
+    log.info("Running learning loop...")
+    db = sqlite3.connect(str(DB_PATH))
+    db.row_factory = sqlite3.Row
+    
+    insights = {}
+    
+    for vertical in ["ai", "tech", "science", "crypto", "linux", "gaming", "diy", "guides"]:
+        # Top performing articles
+        top = db.execute("""
+            SELECT a.title, a.word_count, COALESCE(SUM(an.pageviews), 0) as views
+            FROM articles a
+            LEFT JOIN analytics an ON a.id = an.article_id
+            WHERE a.vertical = ? AND a.status = 'published'
+            GROUP BY a.id
+            ORDER BY views DESC LIMIT 5
+        """, (vertical,)).fetchall()
+        
+        # Optimal word count
+        wc = db.execute("""
+            SELECT AVG(a.word_count) as avg_wc
+            FROM articles a
+            LEFT JOIN analytics an ON a.id = an.article_id
+            WHERE a.vertical = ? AND a.status = 'published'
+            GROUP BY a.vertical
+        """, (vertical,)).fetchone()
+        
+        # Best headline patterns (simple analysis)
+        headline_data = db.execute("""
+            SELECT a.title
+            FROM articles a
+            LEFT JOIN analytics an ON a.id = an.article_id
+            WHERE a.vertical = ? AND a.status = 'published'
+            ORDER BY COALESCE(SUM(an.pageviews), 0) DESC LIMIT 3
+        """, (vertical,)).fetchall()
+        
+        insights[vertical] = {
+            "top_articles": [dict(r) for r in top],
+            "optimal_word_count": round(wc["avg_wc"]) if wc and wc["avg_wc"] else None,
+            "top_headlines": [r["title"] for r in headline_data],
+            "total_articles": db.execute(
+                "SELECT COUNT(*) FROM articles WHERE vertical=? AND status='published'",
+                (vertical,)
+            ).fetchone()[0],
+        }
+        
+        # Store to performance_learning table
+        db.execute("""
+            INSERT OR REPLACE INTO performance_learning (vertical, top_patterns, headline_formats,
+                optimal_word_count, keyword_insights, updated_at)
+            VALUES (?, ?, ?, ?, ?, datetime('now'))
+        """, (
+            vertical,
+            json.dumps(insights[vertical]["top_articles"]),
+            json.dumps(insights[vertical]["top_headlines"]),
+            insights[vertical]["optimal_word_count"],
+            json.dumps([]),
+        ))
+    
+    db.commit()
+    db.close()
+    
+    log.info(f"Learning loop complete. Processed {len(insights)} verticals.")
+    return insights
+
+
+def generate_topic_boost() -> dict:
+    """Generate topic scoring boosts based on learning data."""
+    db = sqlite3.connect(str(DB_PATH))
+    db.row_factory = sqlite3.Row
+    
+    boosts = {}
+    for vertical in ["ai", "tech", "science", "crypto", "linux", "gaming", "diy", "guides"]:
+        pl = db.execute(
+            "SELECT * FROM performance_learning WHERE vertical=? ORDER BY updated_at DESC LIMIT 1",
+            (vertical,)
+        ).fetchone()
+        
+        if pl:
+            boosts[vertical] = {
+                "boost": 1.0,  # Default neutral
+                "preferred_word_count": pl["optimal_word_count"],
+                "avoid_patterns": [],
+                "prefer_patterns": [],
+            }
+    
+    db.close()
+    return boosts
+
+
+if __name__ == "__main__":
+    print("Running analytics learning loop...")
+    insights = run_learning_loop()
+    for vertical, data in insights.items():
+        print(f"\n{vertical}: {data['total_articles']} articles, "
+              f"optimal WC: {data['optimal_word_count']}")
+        for art in data["top_articles"]:
+            print(f"  {art['views']:>5} views | {art['title'][:60]}")

analytics/collector.py

@@ -0,0 +1,101 @@
+"""
+Auto Publisher — Analytics collector endpoint.
+Lightweight Flask app that runs on each site CT to collect pageview data.
+"""
+from flask import Flask, request, jsonify
+import sqlite3
+import json
+import hashlib
+import time
+from datetime import datetime, timedelta
+from pathlib import Path
+
+app = Flask(__name__)
+
+DB_PATH = "/var/lib/auto-publisher/analytics.db"
+
+
+def get_db():
+    Path(DB_PATH).parent.mkdir(parents=True, exist_ok=True)
+    db = sqlite3.connect(DB_PATH)
+    db.execute("""
+        CREATE TABLE IF NOT EXISTS pageviews (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            path TEXT NOT NULL,
+            vertical TEXT NOT NULL,
+            referrer TEXT DEFAULT '',
+            user_agent TEXT DEFAULT '',
+            ip_hash TEXT DEFAULT '',
+            created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+        )
+    """)
+    db.execute("""
+        CREATE INDEX IF NOT EXISTS idx_pageviews_path ON pageviews(path);
+    """)
+    db.execute("""
+        CREATE INDEX IF NOT EXISTS idx_pageviews_created ON pageviews(created_at);
+    """)
+    db.commit()
+    return db
+
+
+@app.route("/a/collect", methods=["POST", "GET"])
+def collect():
+    """Collect a pageview ping."""
+    path = request.args.get("p", "/")
+    vertical = request.args.get("v", "unknown")
+    ref = request.args.get("r", "")
+    ua = request.headers.get("User-Agent", "")[:200]
+    ip = request.remote_addr or ""
+    ip_hash = hashlib.sha256(ip.encode()).hexdigest()[:16] if ip else ""
+    
+    db = get_db()
+    db.execute(
+        "INSERT INTO pageviews (path, vertical, referrer, user_agent, ip_hash) VALUES (?, ?, ?, ?, ?)",
+        (path, vertical, ref, ua, ip_hash)
+    )
+    db.commit()
+    db.close()
+    
+    # Return a 1x1 transparent GIF
+    return b"\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\xff\xff\xff\x00\x00\x00\x21\xf9\x04\x01\x00\x00\x00\x00\x2c\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02\x44\x01\x00\x3b", 200, {
+        "Content-Type": "image/gif",
+        "Cache-Control": "no-cache, no-store, must-revalidate",
+    }
+
+
+@app.route("/a/stats")
+def stats():
+    """Get site analytics summary."""
+    db = get_db()
+    db.row_factory = sqlite3.Row
+    
+    today = datetime.now().strftime("%Y-%m-%d")
+    
+    total = db.execute("SELECT COUNT(*) as c FROM pageviews").fetchone()["c"]
+    today_views = db.execute(
+        "SELECT COUNT(*) as c FROM pageviews WHERE created_at >= ?", (today,)
+    ).fetchone()["c"]
+    
+    top_pages = db.execute("""
+        SELECT path, COUNT(*) as c FROM pageviews
+        WHERE created_at >= date('now', '-30 days')
+        GROUP BY path ORDER BY c DESC LIMIT 10
+    """).fetchall()
+    
+    db.close()
+    
+    return jsonify({
+        "total_views": total,
+        "today_views": today_views,
+        "top_pages": [{"path": r["path"], "views": r["c"]} for r in top_pages],
+    })
+
+
+@app.route("/health")
+def health():
+    return jsonify({"status": "ok"})
+
+
+if __name__ == "__main__":
+    app.run(host="0.0.0.0", port=5199, debug=False)

core/cloudflare_setup.py

@@ -0,0 +1,184 @@
+"""
+Wire auto-publisher sites to Cloudflare home tunnel.
+Adds ingress rules to the existing home tunnel for each site subdomain.
+"""
+import json
+import subprocess
+import time
+
+# Home tunnel ID (the existing tunnel that routes ~38 services)
+HOME_TUNNEL_ID = "d2871458-1737-4844-b114-9a44b9d71e25"
+ACCT_ID = "895479ab3540daa6d61ae702a5164475"
+ZONE_ID = "93634d23ff8138fbd795763b681158ac"
+DOMAIN = "thetempleofdoom.com"
+
+# Site IPs
+SITES = {
+    "ai":      "10.30.20.240",
+    "tech":    "10.30.20.241",
+    "science": "10.30.20.242",
+    "crypto":  "10.30.20.243",
+    "linux":   "10.30.20.244",
+    "gaming":  "10.30.20.246",
+    "diy":     "10.30.20.247",
+    "guides":  "10.30.20.248",
+}
+
+
+def get_home_tunnel_config():
+    """Get current home tunnel ingress configuration."""
+    result = subprocess.run(
+        ["cloudflared", "tunnel", "info", "--output", "json", HOME_TUNNEL_ID],
+        capture_output=True, text=True, timeout=10
+    )
+    if result.returncode == 0 and result.stdout.strip():
+        return json.loads(result.stdout)
+    return None
+
+
+def add_ingress_rules():
+    """Add ingress rules for all 8 publisher sites to the home tunnel."""
+    print("Adding Cloudflare tunnel ingress rules...")
+    
+    for name, ip in SITES.items():
+        hostname = f"{name}.{DOMAIN}"
+        
+        # Check if DNS record already exists
+        print(f"\n  {hostname}:")
+        
+        # Use cloudflared CLI to add route
+        cmd = [
+            "cloudflared", "tunnel", "route", "dns",
+            "--overwrite-dns",
+            HOME_TUNNEL_ID, hostname
+        ]
+        
+        result = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
+        if result.returncode == 0:
+            print(f"    ✅ DNS route created: {hostname} → tunnel")
+        else:
+            error = result.stderr.strip()
+            if "already exists" in error.lower() or "conflict" in error.lower():
+                print(f"    ⚠️  DNS already exists: {hostname}")
+            else:
+                print(f"    ❌ Failed: {error[:200]}")
+                # Fallback: manual DNS via API
+                _add_dns_manual(name, hostname)
+        
+        # Add local ingress mapping using config file approach
+        _add_config_ingress(name, hostname, ip)
+
+
+def _add_dns_manual(name, hostname):
+    """Manual DNS CNAME creation via API."""
+    # This requires a working CF token
+    CF_TOKEN = "cfat_M62Ke6eLXPZnb26qW4FCRL4T4h8l4KrRVLFIRWad9455c3e0"
+    
+    result = subprocess.run([
+        "curl", "-s", "-X", "POST",
+        f"https://api.cloudflare.com/client/v4/zones/{ZONE_ID}/dns_records",
+        "-H", f"Authorization: Bearer {CF_TOKEN}",
+        "-H", "Content-Type: application/json",
+        "-d", json.dumps({
+            "type": "CNAME",
+            "name": name,
+            "content": f"{HOME_TUNNEL_ID}.cfargotunnel.com",
+            "proxied": True,
+            "ttl": 1,
+        }),
+    ], capture_output=True, text=True, timeout=15)
+    
+    try:
+        resp = json.loads(result.stdout)
+        if resp.get("success"):
+            print(f"    ✅ Manual DNS created: {hostname}")
+        else:
+            errors = resp.get("errors", [])
+            for e in errors:
+                if "already exists" in str(e).lower() or e.get("code") == 81053:
+                    print(f"    ⚠️  DNS already exists: {hostname}")
+                    return
+            print(f"    ❌ Manual DNS failed: {errors}")
+    except Exception:
+        print(f"    ❌ API error: {result.stdout[:200]}")
+
+
+def _add_config_ingress(name, hostname, ip):
+    """Add ingress rule to cloudflared config file."""
+    config_path = "/usr/local/etc/cloudflared/config.yml"
+    
+    # Check if cloudflared is running locally
+    result = subprocess.run(
+        ["pgrep", "-f", "cloudflared"], capture_output=True, text=True
+    )
+    
+    if result.returncode != 0:
+        print(f"    ℹ️  cloudflared not running locally — tunnel is on Proxmox CT680")
+        print(f"    ℹ️  Add manually: cloudflared tunnel route dns {HOME_TUNNEL_ID} {hostname}")
+        return
+    
+    # Read existing config
+    try:
+        with open(config_path) as f:
+            config = f.read()
+    except FileNotFoundError:
+        print(f"    ℹ️  No local cloudflared config — tunnel managed elsewhere")
+        return
+    
+    # Check if entry exists
+    if f"hostname: {hostname}" in config:
+        print(f"    ⚠️  Ingress already in config for {hostname}")
+        return
+    
+    # Add ingress rule before the catch-all
+    new_rule = f"""
+  - hostname: {hostname}
+    service: http://{ip}:80"""
+    
+    if "service: http_status:404" in config:
+        config = config.replace(
+            "service: http_status:404",
+            f"{new_rule}\n  - service: http_status:404"
+        )
+    
+    with open(config_path, "w") as f:
+        f.write(config)
+    
+    print(f"    ✅ Ingress added locally for {hostname}")
+    
+    # Restart cloudflared
+    subprocess.run(["brew", "services", "restart", "cloudflared"], 
+                   capture_output=True, timeout=10)
+    print(f"    ✅ cloudflared restarted")
+
+
+def verify_all():
+    """Verify all sites are accessible via Cloudflare tunnel."""
+    print("\n\nVerifying all sites...")
+    time.sleep(5)
+    
+    for name in SITES:
+        url = f"https://{name}.{DOMAIN}"
+        result = subprocess.run(
+            ["curl", "-s", "-o", "/dev/null", "-w", "%{http_code}", url],
+            capture_output=True, text=True, timeout=10
+        )
+        code = result.stdout.strip()
+        status = "✅" if code in ("200", "301", "302") else "❌"
+        print(f"  {status} {url} → HTTP {code}")
+
+
+if __name__ == "__main__":
+    print("=" * 60)
+    print("CLOUDFLARE TUNNEL SETUP — Auto Publisher Sites")
+    print("=" * 60)
+    print(f"\nHome Tunnel: {HOME_TUNNEL_ID}")
+    print(f"Domain: {DOMAIN}")
+    print()
+    
+    add_ingress_rules()
+    verify_all()
+    
+    print(f"\n{'='*60}")
+    print("TUNNEL SETUP COMPLETE")
+    print(f"{'='*60}")

core/dashboard.log

@@ -0,0 +1,3 @@
+Admin Dashboard → http://127.0.0.1:5106
+ * Serving Flask app 'app'
+ * Debug mode: off

core/dashboard_error.log

@@ -0,0 +1,4 @@
+WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
+ * Running on http://127.0.0.1:5106
+Press CTRL+C to quit
+127.0.0.1 - - [03/Aug/2026 21:35:50] "GET /health HTTP/1.1" 200 -

core/deploy_sites.py

@@ -0,0 +1,334 @@
+"""
+Deploy all 8 auto-publisher sites to Proxmox LXC containers.
+Creates CTs, installs nginx, configures cloudflared tunnels, and pushes code.
+"""
+import os
+import sys
+import time
+import json
+import subprocess
+from pathlib import Path
+
+PROXMOX = "root@10.30.20.85"
+PROXMOX_PASS = "czapiewski"
+STORAGE = "poolmaster"
+BRIDGE = "vmbr0"
+GATEWAY = "10.30.20.1"
+TEMPLATE = "local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst"
+DOMAIN = "thetempleofdoom.com"
+GITEA_HOST = "10.30.20.149:3000"
+GITEA_TOKEN = "dec25a837e0d85573706c4d5d608c2df215f3320"
+
+# Site definitions: name → (ct_id, ip)
+SITES = {
+    "ai":      (135, "10.30.20.240"),
+    "tech":    (136, "10.30.20.241"),
+    "science": (137, "10.30.20.242"),
+    "crypto":  (138, "10.30.20.243"),
+    "linux":   (139, "10.30.20.244"),
+    "gaming":  (140, "10.30.20.246"),
+    "diy":     (141, "10.30.20.247"),
+    "guides":  (142, "10.30.20.248"),
+}
+
+
+def ssh(cmd: str, timeout: int = 30) -> str:
+    """Run command on Proxmox host."""
+    result = subprocess.run(
+        ["ssh", "-o", "StrictHostKeyChecking=accept-new", PROXMOX, cmd],
+        capture_output=True, text=True, timeout=timeout
+    )
+    return result.stdout.strip()
+
+
+def pct_exec(vmid: int, cmd: str, timeout: int = 30) -> str:
+    """Run command inside a CT."""
+    return ssh(f"pct exec {vmid} -- bash -c '{cmd}'", timeout=timeout)
+
+
+def create_ct(name: str, vmid: int, ip: str) -> bool:
+    """Create a Proxmox LXC container."""
+    print(f"  Creating CT {vmid} ({name}) at {ip}...")
+    
+    # Check if CT already exists
+    existing = ssh(f"pct list | grep '^{vmid} '")
+    if existing:
+        print(f"    CT {vmid} already exists — skipping creation")
+        return True
+    
+    cmd = (
+        f"pct create {vmid} {TEMPLATE} "
+        f"--hostname {name}-publisher "
+        f"--storage {STORAGE} "
+        f"--memory 2048 --swap 512 --cores 2 "
+        f"--net0 name=eth0,bridge={BRIDGE},ip={ip}/24,gw={GATEWAY} "
+        f"--unprivileged 1 --password {PROXMOX_PASS} "
+        f"--features nesting=1 --onboot 1"
+    )
+    out = ssh(cmd)
+    print(f"    Created: {out}")
+    
+    # Start it
+    ssh(f"pct start {vmid}")
+    time.sleep(8)
+    
+    # Get assigned IP
+    ip_check = ssh(f"pct exec {vmid} -- hostname -I")
+    print(f"    IP: {ip_check}")
+    
+    return True
+
+
+def setup_ct(name: str, vmid: int, ip: str) -> bool:
+    """Install nginx and configure the site on a CT."""
+    print(f"  Setting up CT {vmid} ({name})...")
+    
+    # Update packages
+    pct_exec(vmid, "apt update -qq && apt install -y -qq nginx curl python3 python3-pip git 2>&1 | tail -5",
+             timeout=120)
+    
+    # Create web root
+    pct_exec(vmid, "mkdir -p /var/www/html/articles /var/www/html/assets/images")
+    
+    # Copy shared assets
+    local_assets = Path(__file__).resolve().parent.parent / "shared" / "assets"
+    if local_assets.exists():
+        # Create tarball of shared assets
+        subprocess.run(
+            f"cd {local_assets.parent} && tar czf /tmp/shared_assets.tar.gz assets/",
+            shell=True, capture_output=True
+        )
+        subprocess.run(
+            f"scp /tmp/shared_assets.tar.gz {PROXMOX}:/tmp/shared_assets.tar.gz",
+            shell=True, capture_output=True
+        )
+        ssh(f"pct push {vmid} /tmp/shared_assets.tar.gz /tmp/shared_assets.tar.gz")
+        pct_exec(vmid, "cd /var/www/html && tar xzf /tmp/shared_assets.tar.gz")
+    
+    # Create nginx config
+    nginx_conf = f"""server {{
+    listen 80 default_server;
+    server_name {name}.{DOMAIN};
+    root /var/www/html;
+    index index.html;
+
+    location / {{
+        try_files $uri $uri/ $uri.html =404;
+    }}
+
+    location /assets/ {{
+        expires 30d;
+        add_header Cache-Control "public, immutable";
+    }}
+
+    location /rss.xml {{
+        add_header Content-Type "application/rss+xml";
+    }}
+
+    location /sitemap.xml {{
+        add_header Content-Type "application/xml";
+    }}
+
+    # Gzip
+    gzip on;
+    gzip_types text/plain text/html text/css application/json application/javascript text/xml application/xml;
+    gzip_min_length 1000;
+}}"""
+    
+    # Write nginx config via base64 to avoid quoting issues
+    import base64
+    encoded = base64.b64encode(nginx_conf.encode()).decode()
+    pct_exec(vmid, f"echo '{encoded}' | base64 -d > /etc/nginx/sites-available/default")
+    pct_exec(vmid, "rm -f /etc/nginx/sites-enabled/default && "
+                   "ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default")
+    
+    # Verify and restart nginx
+    result = pct_exec(vmid, "nginx -t 2>&1 && systemctl restart nginx && echo 'NGINX_OK'")
+    print(f"    Nginx: {'OK' if 'NGINX_OK' in result else 'FAILED'}")
+    
+    # Create a simple health endpoint
+    pct_exec(vmid, f"echo '<html><body><h1>{name}.{DOMAIN}</h1><p>Auto Publisher — coming soon</p></body></html>' > /var/www/html/index.html")
+    
+    return True
+
+
+def create_gitea_repo(name: str) -> bool:
+    """Create a Gitea repository for a site."""
+    print(f"  Creating Gitea repo: {name}-publisher...")
+    
+    # Check if repo already exists
+    result = subprocess.run(
+        ["curl", "-s", "-o", "/dev/null", "-w", "%{http_code}",
+         f"http://{GITEA_HOST}/api/v1/repos/drjones/{name}-publisher"],
+        capture_output=True, text=True
+    )
+    if result.stdout.strip() == "200":
+        print(f"    Repo {name}-publisher already exists")
+        return True
+    
+    # Create via API
+    result = subprocess.run(
+        ["curl", "-s", "-X", "POST",
+         f"http://{GITEA_HOST}/api/v1/user/repos",
+         "-H", "Content-Type: application/json",
+         "-H", f"Authorization: token {GITEA_TOKEN}",
+         "-d", json.dumps({
+             "name": f"{name}-publisher",
+             "description": f"Auto Publisher site: {name}.thetempleofdoom.com",
+             "private": False,
+             "auto_init": True,
+         })],
+        capture_output=True, text=True
+    )
+    
+    if "200" in result.stdout or "201" in result.stdout or "409" in result.stdout:
+        print(f"    Repo {name}-publisher created")
+        return True
+    else:
+        print(f"    Gitea API failed: {result.stdout[:200]}")
+        # Fallback: create via filesystem + SQLite
+        ssh(f"pct exec 525 -- mkdir -p /var/lib/gitea/data/gitea-repositories/drjones/{name}-publisher.git")
+        ssh(f"pct exec 525 -- git init --bare /var/lib/gitea/data/gitea-repositories/drjones/{name}-publisher.git")
+        ssh(f"pct exec 525 -- chown -R gitea:gitea /var/lib/gitea/data/gitea-repositories/drjones/{name}-publisher.git")
+        return True
+
+
+def verify_site(name: str, ip: str) -> bool:
+    """Verify a site is serving content."""
+    try:
+        result = subprocess.run(
+            ["curl", "-s", "-o", "/dev/null", "-w", "%{http_code}",
+             f"http://{ip}:80/"],
+            capture_output=True, text=True, timeout=5
+        )
+        code = result.stdout.strip()
+        print(f"    HTTP status: {code}")
+        return code == "200"
+    except Exception as e:
+        print(f"    Verify failed: {e}")
+        return False
+
+
+def deploy_site_code(name: str, vmid: int) -> bool:
+    """Push site code to CT and to Gitea."""
+    print(f"  Pushing code to CT {vmid}...")
+    
+    site_dir = Path(__file__).resolve().parent.parent / "sites" / name
+    
+    # Build a placeholder site if directory is empty
+    if not site_dir.exists() or not list(site_dir.glob("*.html")):
+        # Create placeholder
+        site_dir.mkdir(parents=True, exist_ok=True)
+        (site_dir / "articles").mkdir(exist_ok=True)
+        
+        placeholder = f"""<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>{name}.thetempleofdoom.com</title>
+    <link rel="stylesheet" href="/assets/style.css">
+</head>
+<body>
+    <header>
+        <nav>
+            <a href="/" class="logo">{name}.thetempleofdoom.com</a>
+        </nav>
+    </header>
+    <main>
+        <section class="hero">
+            <h1>{name.title()} Insights & Guides</h1>
+            <p>Expert {name} articles, tutorials, and deep dives. Updated daily.</p>
+            <p style="margin-top:2rem;color:var(--text2);">🚀 First article coming soon...</p>
+        </section>
+        <section class="articles-grid">
+            <div class="card">
+                <h2>Coming Soon</h2>
+                <p class="meta">The autonomous publishing pipeline is being set up.</p>
+                <p>Fresh {name} content will be published here automatically every day.</p>
+            </div>
+        </section>
+    </main>
+    <footer class="site-footer">
+        <p>&copy; 2026 {name}.thetempleofdoom.com</p>
+    </footer>
+</body>
+</html>"""
+        (site_dir / "index.html").write_text(placeholder)
+    
+    # Tar and push
+    subprocess.run(
+        f"cd {site_dir.parent} && tar czf /tmp/{name}_site.tar.gz {name}/",
+        shell=True, capture_output=True
+    )
+    subprocess.run(
+        f"scp /tmp/{name}_site.tar.gz {PROXMOX}:/tmp/{name}_site.tar.gz",
+        shell=True, capture_output=True
+    )
+    ssh(f"pct push {vmid} /tmp/{name}_site.tar.gz /tmp/{name}_site.tar.gz")
+    pct_exec(vmid, f"cd /var/www/html && tar xzf /tmp/{name}_site.tar.gz --strip-components=1 && chown -R www-data:www-data /var/www/html")
+    
+    # Push to Gitea
+    git_dir = site_dir
+    subprocess.run(
+        f"cd {git_dir} && "
+        f"git init 2>/dev/null; "
+        f"git config user.email 'drjones@thetempleofdoom.com'; "
+        f"git config user.name 'drjones'; "
+        f"git add . 2>/dev/null; "
+        f"git commit -m 'Initial: {name} publisher site' 2>/dev/null; "
+        f"git remote remove origin 2>/dev/null; "
+        f"git remote add origin http://drjones:{GITEA_TOKEN}@{GITEA_HOST}/drjones/{name}-publisher.git; "
+        f"git push -u origin main --force 2>&1",
+        shell=True, capture_output=True
+    )
+    
+    return True
+
+
+def main():
+    print("=" * 60)
+    print("AUTO PUBLISHER — Site Deployment")
+    print("=" * 60)
+    print()
+    
+    for name, (vmid, ip) in SITES.items():
+        print(f"\n{'='*40}")
+        print(f"  📡 {name}.thetempleofdoom.com (CT {vmid}, {ip})")
+        print(f"{'='*40}")
+        
+        # Step 1: Create CT
+        if not create_ct(name, vmid, ip):
+            print(f"  ❌ Failed to create CT for {name}")
+            continue
+        
+        # Step 2: Setup nginx
+        if not setup_ct(name, vmid, ip):
+            print(f"  ❌ Failed to setup {name}")
+            continue
+        
+        # Step 3: Create Gitea repo
+        create_gitea_repo(name)
+        
+        # Step 4: Push initial code
+        deploy_site_code(name, vmid)
+        
+        # Step 5: Verify
+        time.sleep(2)
+        if verify_site(name, ip):
+            print(f"  ✅ {name}.thetempleofdoom.com is LIVE (http://{ip}:80)")
+        else:
+            print(f"  ⚠️  {name} may need nginx restart")
+    
+    print(f"\n{'='*60}")
+    print("DEPLOYMENT COMPLETE")
+    print(f"{'='*60}")
+    print("\nSites deployed:")
+    for name, (vmid, ip) in SITES.items():
+        print(f"  {name}.thetempleofdoom.com → CT {vmid} @ {ip}")
+    print("\nNext: Set up Cloudflare tunnels for public access")
+    print("Run: python3 core/cloudflare_setup.py")
+
+
+if __name__ == "__main__":
+    main()

core/orchestrator.log

@@ -0,0 +1,5 @@
+2026-08-03 21:38:15,046 [INFO] orchestrator: Starting trend discovery...
+2026-08-03 21:38:19,444 [WARNING] orchestrator: Seasonal topic generation failed: ollama_json() got an unexpected keyword argument 'temperature'
+2026-08-03 21:38:19,445 [WARNING] orchestrator: Topic scoring failed: ollama_json() got an unexpected keyword argument 'temperature'
+2026-08-03 21:38:19,449 [INFO] orchestrator: Discovered 0 topics, stored top 25
+2026-08-03 21:38:30,350 [INFO] orchestrator: Starting trend discovery...

cron/daily_publish.py

@@ -0,0 +1,26 @@
+#!/usr/bin/env python3
+"""
+Daily autonomous publishing script.
+Runs every morning to discover topics, research, write, and publish articles.
+
+Usage: python3 ~/auto-publisher/cron/daily_publish.py [--max 3]
+"""
+import sys
+import os
+sys.path.insert(0, os.path.expanduser("~/auto-publisher/core"))
+
+from orchestrator import run_daily_pipeline, init_db
+
+if __name__ == "__main__":
+    import argparse
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--max", type=int, default=3, help="Max articles to publish")
+    args = ap.parse_args()
+    
+    # Initialize DB if needed
+    init_db()
+    
+    # Run the pipeline
+    print("🚀 Starting daily autonomous publishing pipeline...")
+    run_daily_pipeline(max_articles=args.max)
+    print("✅ Daily pipeline complete")

dashboard/app.py

@@ -0,0 +1,309 @@
+"""
+Autonomous Publishing System — Admin Dashboard
+Flask app for monitoring and controlling the publishing pipeline.
+"""
+import os
+import sys
+import json
+import sqlite3
+from pathlib import Path
+from datetime import datetime, timedelta
+from flask import Flask, render_template_string, jsonify, request, redirect, url_for
+
+BASE_DIR = Path(__file__).resolve().parent.parent
+DB_PATH = BASE_DIR / "core" / "publisher.db"
+sys.path.insert(0, str(BASE_DIR / "core"))
+
+app = Flask(__name__)
+app.config["SECRET_KEY"] = os.environ.get("DASHBOARD_SECRET", "auto-publisher-dev")
+
+VERTICALS = ["ai", "tech", "science", "crypto", "linux", "gaming", "diy", "guides"]
+
+
+def get_db():
+    db = sqlite3.connect(str(DB_PATH))
+    db.row_factory = sqlite3.Row
+    return db
+
+
+# ─── Templates ──────────────────────────────────────────────────────
+DASHBOARD_HTML = """<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Auto Publisher — Admin Dashboard</title>
+    <style>
+        :root {
+            --bg: #0f172a; --card: #1e293b; --text: #e2e8f0;
+            --text2: #94a3b8; --primary: #3b82f6; --green: #10b981;
+            --red: #ef4444; --yellow: #f59e0b; --border: #334155;
+            --font: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
+        }
+        * { box-sizing: border-box; margin: 0; padding: 0; }
+        body { font-family: var(--font); background: var(--bg); color: var(--text); padding: 2rem; }
+        h1 { font-size: 1.8rem; margin-bottom: 0.5rem; }
+        h2 { font-size: 1.2rem; color: var(--text2); margin-bottom: 1.5rem; font-weight: 400; }
+        .grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 1rem; margin-bottom: 2rem; }
+        .stat {
+            background: var(--card); border: 1px solid var(--border);
+            border-radius: 8px; padding: 1.25rem;
+        }
+        .stat .label { font-size: 0.8rem; color: var(--text2); text-transform: uppercase; letter-spacing: 0.05em; }
+        .stat .value { font-size: 2rem; font-weight: 700; margin-top: 0.25rem; }
+        .stat .value.green { color: var(--green); }
+        .stat .value.yellow { color: var(--yellow); }
+        .stat .value.red { color: var(--red); }
+        .panel {
+            background: var(--card); border: 1px solid var(--border);
+            border-radius: 8px; padding: 1.5rem; margin-bottom: 1.5rem;
+        }
+        .panel h3 { margin-bottom: 1rem; font-size: 1.1rem; }
+        table { width: 100%; border-collapse: collapse; font-size: 0.9rem; }
+        th { text-align: left; color: var(--text2); padding: 0.5rem; border-bottom: 1px solid var(--border); font-weight: 500; }
+        td { padding: 0.5rem; border-bottom: 1px solid var(--border); }
+        .badge {
+            display: inline-block; padding: 0.15rem 0.5rem; border-radius: 4px;
+            font-size: 0.75rem; font-weight: 600; text-transform: uppercase;
+        }
+        .badge.published { background: rgba(16, 185, 129, 0.2); color: var(--green); }
+        .badge.draft { background: rgba(245, 158, 11, 0.2); color: var(--yellow); }
+        .badge.discovered { background: rgba(59, 130, 246, 0.2); color: var(--primary); }
+        button, .btn {
+            background: var(--primary); color: white; border: none;
+            padding: 0.5rem 1.25rem; border-radius: 6px; cursor: pointer;
+            font-size: 0.9rem; font-weight: 500;
+        }
+        button:hover { opacity: 0.9; }
+        button.danger { background: var(--red); }
+        .actions { display: flex; gap: 0.75rem; margin-bottom: 1.5rem; flex-wrap: wrap; }
+        .log-entry { font-family: monospace; font-size: 0.8rem; padding: 0.25rem 0; color: var(--text2); }
+        .log-entry.error { color: var(--red); }
+        .log-entry.success { color: var(--green); }
+        .site-card {
+            display: inline-block; background: var(--card); border: 1px solid var(--border);
+            border-radius: 8px; padding: 1rem 1.5rem; margin: 0.5rem;
+        }
+        .site-card .domain { font-weight: 600; }
+        .site-card .status { font-size: 0.8rem; }
+        .site-card .status.live { color: var(--green); }
+        .site-card .status.pending { color: var(--yellow); }
+    </style>
+</head>
+<body>
+    <h1>🤖 Auto Publisher</h1>
+    <h2>Autonomous Publishing System — Admin Dashboard</h2>
+
+    <!-- Stats Grid -->
+    <div class="grid">
+        <div class="stat">
+            <div class="label">Topics Discovered</div>
+            <div class="value">{{ stats.topics }}</div>
+        </div>
+        <div class="stat">
+            <div class="label">Articles Written</div>
+            <div class="value">{{ stats.articles }}</div>
+        </div>
+        <div class="stat">
+            <div class="label">Published</div>
+            <div class="value green">{{ stats.published }}</div>
+        </div>
+        <div class="stat">
+            <div class="label">Last Run</div>
+            <div class="value" style="font-size:1rem;">{{ stats.last_run or 'Never' }}</div>
+        </div>
+        <div class="stat">
+            <div class="label">Total Pageviews</div>
+            <div class="value">{{ stats.total_views }}</div>
+        </div>
+        <div class="stat">
+            <div class="label">Avg Score</div>
+            <div class="value yellow">{{ stats.avg_score }}</div>
+        </div>
+    </div>
+
+    <!-- Sites -->
+    <div class="panel">
+        <h3>📡 Sites</h3>
+        <div style="display:flex;flex-wrap:wrap;">
+            {% for v in verticals %}
+            <div class="site-card">
+                <div class="domain">{{ v }}.thetempleofdoom.com</div>
+                <div class="status {{ 'live' if v in live_sites else 'pending' }}">
+                    {{ '● Live' if v in live_sites else '○ Pending' }}
+                </div>
+            </div>
+            {% endfor %}
+        </div>
+    </div>
+
+    <!-- Actions -->
+    <div class="panel">
+        <h3>🎮 Controls</h3>
+        <div class="actions">
+            <form method="POST" action="/api/run" style="display:inline">
+                <button type="submit">▶ Run Pipeline Now</button>
+            </form>
+            <form method="POST" action="/api/discover" style="display:inline">
+                <button type="submit">🔍 Discover Topics</button>
+            </form>
+            <button onclick="location.reload()">🔄 Refresh</button>
+        </div>
+    </div>
+
+    <!-- Recent Topics -->
+    <div class="panel">
+        <h3>📋 Recent Topics</h3>
+        <table>
+            <thead><tr>
+                <th>Title</th><th>Vertical</th><th>Score</th><th>Status</th><th>Discovered</th>
+            </tr></thead>
+            <tbody>
+                {% for t in topics %}
+                <tr>
+                    <td>{{ t.title[:80] }}</td>
+                    <td><span class="badge">{{ t.vertical }}</span></td>
+                    <td>{{ t.composite_score }}</td>
+                    <td><span class="badge {{ t.status }}">{{ t.status }}</span></td>
+                    <td>{{ t.created_at[:10] if t.created_at else '-' }}</td>
+                </tr>
+                {% endfor %}
+            </tbody>
+        </table>
+    </div>
+
+    <!-- Recent Articles -->
+    <div class="panel">
+        <h3>📝 Recent Articles</h3>
+        <table>
+            <thead><tr>
+                <th>Title</th><th>Vertical</th><th>Words</th><th>Status</th><th>Created</th>
+            </tr></thead>
+            <tbody>
+                {% for a in articles %}
+                <tr>
+                    <td>{{ a.title[:80] if a.title else 'Untitled' }}</td>
+                    <td><span class="badge">{{ a.vertical }}</span></td>
+                    <td>{{ a.word_count }}</td>
+                    <td><span class="badge {{ a.status }}">{{ a.status }}</span></td>
+                    <td>{{ a.created_at[:10] if a.created_at else '-' }}</td>
+                </tr>
+                {% endfor %}
+            </tbody>
+        </table>
+    </div>
+
+    <!-- Pipeline Log -->
+    <div class="panel">
+        <h3>📜 Pipeline Log</h3>
+        <div style="max-height:300px;overflow-y:auto;">
+            {% for entry in log_lines %}
+            <div class="log-entry {{ entry.level }}">[{{ entry.time }}] {{ entry.msg }}</div>
+            {% endfor %}
+        </div>
+    </div>
+</body>
+</html>"""
+
+
+@app.route("/")
+def index():
+    db = get_db()
+    stats = {
+        "topics": db.execute("SELECT COUNT(*) FROM topics").fetchone()[0],
+        "articles": db.execute("SELECT COUNT(*) FROM articles").fetchone()[0],
+        "published": db.execute("SELECT COUNT(*) FROM articles WHERE status='published'").fetchone()[0],
+        "last_run": None,
+        "total_views": db.execute("SELECT COALESCE(SUM(pageviews), 0) FROM analytics").fetchone()[0],
+        "avg_score": round(db.execute("SELECT COALESCE(AVG(composite_score), 0) FROM topics WHERE composite_score > 0").fetchone()[0], 1),
+    }
+    last_run = db.execute("SELECT started_at FROM pipeline_runs ORDER BY id DESC LIMIT 1").fetchone()
+    if last_run:
+        stats["last_run"] = last_run[0]
+
+    topics = db.execute("SELECT * FROM topics ORDER BY created_at DESC LIMIT 20").fetchall()
+    articles = db.execute("SELECT * FROM articles ORDER BY created_at DESC LIMIT 20").fetchall()
+
+    # Read log
+    log_path = BASE_DIR / "core" / "orchestrator.log"
+    log_lines = []
+    if log_path.exists():
+        for line in log_path.read_text().split("\n")[-30:]:
+            if not line.strip():
+                continue
+            level = ""
+            if "ERROR" in line:
+                level = "error"
+            elif "✓" in line or "successful" in line.lower():
+                level = "success"
+            log_lines.append({
+                "time": line[:19] if len(line) > 19 else "",
+                "msg": line,
+                "level": level,
+            })
+
+    live_sites = ["ai", "tech"]  # Will be dynamically checked
+
+    return render_template_string(
+        DASHBOARD_HTML,
+        stats=stats,
+        topics=topics,
+        articles=articles,
+        log_lines=log_lines,
+        verticals=VERTICALS,
+        live_sites=live_sites,
+    )
+
+
+@app.route("/api/run", methods=["POST"])
+def api_run():
+    """Trigger a pipeline run."""
+    from orchestrator import run_daily_pipeline
+    import threading
+    def _run():
+        run_daily_pipeline(max_articles=3)
+    t = threading.Thread(target=_run, daemon=True)
+    t.start()
+    return jsonify({"status": "started", "message": "Pipeline running in background"})
+
+
+@app.route("/api/discover", methods=["POST"])
+def api_discover():
+    """Trigger trend discovery."""
+    from orchestrator import discover_trends
+    import threading
+    def _run():
+        discover_trends()
+    t = threading.Thread(target=_run, daemon=True)
+    t.start()
+    return jsonify({"status": "started", "message": "Trend discovery running"})
+
+
+@app.route("/api/stats")
+def api_stats():
+    db = get_db()
+    return jsonify({
+        "topics_total": db.execute("SELECT COUNT(*) FROM topics").fetchone()[0],
+        "articles_total": db.execute("SELECT COUNT(*) FROM articles").fetchone()[0],
+        "published": db.execute("SELECT COUNT(*) FROM articles WHERE status='published'").fetchone()[0],
+        "by_vertical": {
+            v: db.execute("SELECT COUNT(*) FROM topics WHERE vertical=?", (v,)).fetchone()[0]
+            for v in VERTICALS
+        },
+        "pipeline_runs": db.execute("SELECT COUNT(*) FROM pipeline_runs").fetchone()[0],
+    })
+
+
+@app.route("/health")
+def health():
+    return jsonify({"status": "healthy", "timestamp": datetime.now().isoformat()})
+
+
+if __name__ == "__main__":
+    import argparse
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--port", type=int, default=5106)
+    ap.add_argument("--host", type=str, default="127.0.0.1")
+    args = ap.parse_args()
+    print(f"Admin Dashboard → http://{args.host}:{args.port}")
+    app.run(host=args.host, port=args.port, debug=False)

docs/ARCHITECTURE.md

@@ -0,0 +1,270 @@
+# Autonomous AI Publishing System — Architecture
+
+## Overview
+
+A fully autonomous, self-hosted publishing platform that continuously discovers valuable topics, generates original useful content, and publishes to a collection of evergreen authority websites.
+
+## Sites (Vertical Authority Domains)
+
+| Site | Domain | CT ID | IP | Port |
+|------|--------|-------|-----|------|
+| AI | ai.thetempleofdoom.com | TBD | TBD | 5000 |
+| Tech | tech.thetempleofdoom.com | TBD | TBD | 5000 |
+| Science | science.thetempleofdoom.com | TBD | TBD | 5000 |
+| Crypto | crypto.thetempleofdoom.com | TBD | TBD | 5000 |
+| Linux | linux.thetempleofdoom.com | TBD | TBD | 5000 |
+| Gaming | gaming.thetempleofdoom.com | TBD | TBD | 5000 |
+| DIY | diy.thetempleofdoom.com | TBD | TBD | 5000 |
+| Guides | guides.thetempleofdoom.com | TBD | TBD | 5000 |
+
+## System Architecture
+
+```
+┌──────────────────────────────────────────────────────────────────┐
+│                    CRON ORCHESTRATOR (MacBook)                    │
+│                   Every morning @ 6AM Pacific                     │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│                    TREND DISCOVERY ENGINE                         │
+│  Sources: News APIs, RSS, Reddit, HN, GitHub, Google Trends,     │
+│           arXiv, Stack Overflow, Twitter/X, seasonal calendar    │
+│  Output: Scored topic list with vertical assignments             │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│                      RESEARCH AGENT                               │
+│  Per topic: multi-source extraction → facts, stats, citations,   │
+│  FAQs, timelines, misconceptions, examples                       │
+│  LLM: GamingPC ornith:latest for deep research                   │
+│  Output: Knowledge Package (JSON) → stored in SQLite             │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│                 MULTI-AGENT WRITING PIPELINE                      │
+│  Outline Agent → Technical Writer → SEO Writer → Copy Editor     │
+│  → Fact Checker → Quality Reviewer                               │
+│  LLMs: qwen3.5:4b (fast gate) + ornith:latest (verify)           │
+│  Output: Polished Markdown article + frontmatter                 │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│                    IMAGE GENERATION                               │
+│  FLUX via FAL.ai for hero images, diagrams, charts               │
+│  Output: WebP images in site assets/                             │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│                    SEO PROCESSOR                                  │
+│  Meta titles, descriptions, OG tags, JSON-LD, schema.org,        │
+│  canonical URLs, breadcrumbs, XML sitemap, robots.txt            │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│                    SITE BUILDER (per vertical)                    │
+│  Static site generator: Markdown → HTML, rebuilds on new content │
+│  Updates: homepage, category pages, RSS, sitemap, related links  │
+│  Built-in search via pagefind                                    │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│                    DEPLOYMENT                                     │
+│  1. Build static site locally                                     │
+│  2. scp to Proxmox CT                                            │
+│  3. Restart nginx/service                                        │
+│  4. Ping sitemap to Google/Bing                                  │
+│  5. Verify live                                                   │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│                    ANALYTICS + LEARNING LOOP                      │
+│  Nightly: analyze pageviews, time-on-page, bounce rate            │
+│  Feed back into topic scoring → improve selection over time      │
+│  Plausible-style privacy-first analytics, self-hosted            │
+└──────────────────────────────────────────────────────────────────┘
+```
+
+## Service APIs (REST)
+
+Every component is an independent API service:
+
+### Trend Discovery API (port 5100)
+- `POST /api/trends/discover` — Run full discovery scan
+- `GET /api/trends/scored` — Get scored topic list
+- `GET /api/trends/sources` — List active data sources
+
+### Research API (port 5101)
+- `POST /api/research/topic` — Research a topic, return knowledge package
+- `GET /api/research/package/{id}` — Get stored knowledge package
+- `GET /api/research/sources/{topic}` — Raw sources for a topic
+
+### Writing API (port 5102)
+- `POST /api/write/article` — Generate article from knowledge package
+- `GET /api/write/draft/{id}` — Get draft
+- `POST /api/write/review/{id}` — Request quality review
+
+### SEO API (port 5103)
+- `POST /api/seo/optimize` — SEO-optimize an article
+- `POST /api/seo/sitemap` — Generate sitemap for a site
+- `POST /api/seo/structured-data` — Generate JSON-LD
+
+### Image API (port 5104)
+- `POST /api/images/generate` — Generate image for article
+- `POST /api/images/optimize` — Optimize existing image
+
+### Deploy API (port 5105)
+- `POST /api/deploy/site/{name}` — Build and deploy a site
+- `POST /api/deploy/all` — Deploy all sites
+- `GET /api/deploy/status/{name}` — Deployment status
+
+### Admin Dashboard (port 5106)
+- Full control panel UI
+- Topic approval, manual triggers, analytics views
+
+## Database Schema
+
+### topics
+```sql
+CREATE TABLE topics (
+    id INTEGER PRIMARY KEY,
+    title TEXT NOT NULL,
+    vertical TEXT NOT NULL,  -- ai, tech, science, crypto, linux, gaming, diy, guides
+    trend_score REAL,
+    search_volume INTEGER,
+    competition_score REAL,
+    freshness_score REAL,
+    evergreen_score REAL,
+    composite_score REAL,
+    sources TEXT,  -- JSON array
+    status TEXT DEFAULT 'discovered',  -- discovered, approved, researching, writing, reviewing, published, rejected
+    knowledge_package_id INTEGER,
+    article_id INTEGER,
+    published_url TEXT,
+    published_at TIMESTAMP,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+```
+
+### knowledge_packages
+```sql
+CREATE TABLE knowledge_packages (
+    id INTEGER PRIMARY KEY,
+    topic_id INTEGER,
+    facts TEXT,  -- JSON
+    stats TEXT,  -- JSON
+    definitions TEXT,  -- JSON
+    faqs TEXT,  -- JSON
+    misconceptions TEXT,  -- JSON
+    timeline TEXT,  -- JSON
+    citations TEXT,  -- JSON
+    examples TEXT,  -- JSON
+    related_concepts TEXT,  -- JSON
+    raw_sources TEXT,  -- JSON
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+```
+
+### articles
+```sql
+CREATE TABLE articles (
+    id INTEGER PRIMARY KEY,
+    topic_id INTEGER,
+    vertical TEXT,
+    title TEXT,
+    slug TEXT UNIQUE,
+    content_md TEXT,
+    content_html TEXT,
+    seo_title TEXT,
+    seo_description TEXT,
+    og_image TEXT,
+    json_ld TEXT,
+    word_count INTEGER,
+    reading_time_minutes INTEGER,
+    status TEXT DEFAULT 'draft',
+    published_at TIMESTAMP,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+```
+
+### analytics
+```sql
+CREATE TABLE analytics (
+    id INTEGER PRIMARY KEY,
+    article_id INTEGER,
+    vertical TEXT,
+    pageviews INTEGER DEFAULT 0,
+    unique_visitors INTEGER DEFAULT 0,
+    avg_time_on_page REAL,
+    bounce_rate REAL,
+    referrers TEXT,  -- JSON
+    recorded_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+```
+
+### performance_learning
+```sql
+CREATE TABLE performance_learning (
+    id INTEGER PRIMARY KEY,
+    vertical TEXT,
+    top_performing_patterns TEXT,  -- JSON
+    headline_formats TEXT,  -- JSON
+    optimal_word_count INTEGER,
+    best_publish_times TEXT,  -- JSON
+    keyword_insights TEXT,  -- JSON
+    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+```
+
+## LLM Strategy
+
+| Task | Model | Host | Reasoning |
+|------|-------|------|-----------|
+| Topic scoring | qwen3.5:4b | MacBook | Fast, cheap |
+| Research extraction | ornith:latest | GamingPC | Deep reasoning |
+| Outline generation | qwen3.5:4b | MacBook | Structural, fast |
+| Technical writing | ornith:latest | GamingPC | Quality matters |
+| SEO optimization | qwen3.5:4b | MacBook | Pattern matching |
+| Copy editing | qwen3.5:4b | MacBook | Fast iteration |
+| Fact checking | ornith:latest | GamingPC | Accuracy critical |
+| Quality review | ornith:latest | GamingPC | Final gate |
+
+## Deployment Strategy
+
+Each site is a standalone Proxmox CT running:
+- Python Flask/FastAPI (static site generator)
+- nginx (serving static files)
+- pagefind (search index)
+- cloudflared (tunnel to CF)
+
+Core orchestrator runs on MacBook via cron (launchd-supervised).
+
+## Security Model
+
+- All APIs internal-only (10.30.20.0/24)
+- Admin dashboard: localhost only, auth via Hermes
+- Cloudflare tunnels: only expose nginx on :80
+- No secrets in code — env vars only
+- Gitea private repos for all sites
+
+## Backup Strategy
+
+- All article content in Gitea (git history = backup)
+- SQLite DBs synced to iCloud daily
+- Proxmox CT snapshots weekly
+- Knowledge packages exported to JSON nightly
+
+## Monitoring
+
+- Health checks on all 8 site CTs
+- Orchestrator pipeline status dashboard
+- LLM usage/cost tracking
+- Publish success/failure alerts via Telegram

shared/assets/style.css

@@ -0,0 +1,259 @@
+/* ==========================================================================
+   Autonomous Publishing System — Shared Site CSS
+   Clean, readable, fast-loading design system for all vertical sites.
+   ========================================================================== */
+
+:root {
+    --bg: #ffffff;
+    --bg-secondary: #f8f9fa;
+    --text: #1a1a2e;
+    --text-secondary: #555;
+    --primary: #2563eb;
+    --primary-hover: #1d4ed8;
+    --border: #e5e7eb;
+    --accent: #10b981;
+    --card-bg: #ffffff;
+    --code-bg: #1e293b;
+    --code-text: #e2e8f0;
+    --max-width: 800px;
+    --font-sans: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
+    --font-mono: 'SF Mono', 'Fira Code', 'Fira Mono', 'Roboto Mono', monospace;
+}
+
+@media (prefers-color-scheme: dark) {
+    :root {
+        --bg: #0f172a;
+        --bg-secondary: #1e293b;
+        --text: #e2e8f0;
+        --text-secondary: #94a3b8;
+        --primary: #3b82f6;
+        --primary-hover: #60a5fa;
+        --border: #334155;
+        --card-bg: #1e293b;
+        --code-bg: #0f172a;
+    }
+}
+
+*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
+
+html { font-size: 18px; line-height: 1.7; }
+
+body {
+    font-family: var(--font-sans);
+    background: var(--bg);
+    color: var(--text);
+    max-width: var(--max-width);
+    margin: 0 auto;
+    padding: 0 1.5rem;
+    -webkit-font-smoothing: antialiased;
+}
+
+/* ─── Header ─── */
+header {
+    padding: 1.5rem 0;
+    border-bottom: 1px solid var(--border);
+    margin-bottom: 2rem;
+}
+
+nav {
+    display: flex;
+    justify-content: space-between;
+    align-items: center;
+    flex-wrap: wrap;
+    gap: 1rem;
+}
+
+.logo {
+    font-weight: 700;
+    font-size: 1.2rem;
+    color: var(--primary);
+    text-decoration: none;
+    letter-spacing: -0.02em;
+}
+
+.nav-links { display: flex; gap: 1.5rem; }
+.nav-links a {
+    color: var(--text-secondary);
+    text-decoration: none;
+    font-size: 0.9rem;
+    transition: color 0.2s;
+}
+.nav-links a:hover { color: var(--primary); }
+
+.rss-link {
+    background: var(--bg-secondary);
+    padding: 0.25rem 0.75rem;
+    border-radius: 4px;
+    font-weight: 500;
+}
+
+/* ─── Hero ─── */
+.hero {
+    padding: 3rem 0;
+    text-align: center;
+}
+.hero h1 { font-size: 2.2rem; margin-bottom: 0.5rem; letter-spacing: -0.03em; }
+.hero p { color: var(--text-secondary); font-size: 1.1rem; }
+
+/* ─── Article Cards ─── */
+.articles-grid {
+    display: flex;
+    flex-direction: column;
+    gap: 1.5rem;
+    margin-bottom: 3rem;
+}
+
+.card {
+    background: var(--card-bg);
+    border: 1px solid var(--border);
+    border-radius: 8px;
+    padding: 1.5rem;
+    transition: border-color 0.2s, box-shadow 0.2s;
+}
+.card:hover {
+    border-color: var(--primary);
+    box-shadow: 0 2px 12px rgba(37, 99, 235, 0.1);
+}
+.card h2 {
+    font-size: 1.3rem;
+    margin-bottom: 0.25rem;
+    line-height: 1.3;
+}
+.card h2 a { color: var(--text); text-decoration: none; }
+.card h2 a:hover { color: var(--primary); }
+.card .meta {
+    color: var(--text-secondary);
+    font-size: 0.85rem;
+    margin-bottom: 0.5rem;
+}
+.card p { color: var(--text-secondary); font-size: 0.95rem; }
+
+/* ─── Article Page ─── */
+article { margin-bottom: 3rem; }
+
+.article-header {
+    margin-bottom: 2rem;
+    padding-bottom: 1.5rem;
+    border-bottom: 1px solid var(--border);
+}
+.article-header h1 {
+    font-size: 2rem;
+    line-height: 1.2;
+    margin-bottom: 0.5rem;
+    letter-spacing: -0.03em;
+}
+.article-header .meta {
+    color: var(--text-secondary);
+    font-size: 0.9rem;
+    display: flex;
+    gap: 1rem;
+    flex-wrap: wrap;
+}
+
+.article-content { margin-bottom: 2rem; }
+.article-content h2 {
+    font-size: 1.5rem;
+    margin: 2.5rem 0 0.75rem;
+    padding-bottom: 0.25rem;
+    border-bottom: 2px solid var(--primary);
+    display: inline-block;
+}
+.article-content h3 {
+    font-size: 1.2rem;
+    margin: 1.5rem 0 0.5rem;
+}
+.article-content p { margin-bottom: 1.2rem; }
+.article-content ul, .article-content ol {
+    margin: 0 0 1.2rem 1.5rem;
+}
+.article-content li { margin-bottom: 0.4rem; }
+.article-content a { color: var(--primary); text-decoration: underline; }
+.article-content strong { font-weight: 600; }
+.article-content blockquote {
+    border-left: 4px solid var(--primary);
+    padding: 0.75rem 1.5rem;
+    margin: 1.5rem 0;
+    background: var(--bg-secondary);
+    border-radius: 0 6px 6px 0;
+    font-style: italic;
+}
+.article-content blockquote p { margin-bottom: 0; }
+
+.article-content pre {
+    background: var(--code-bg);
+    color: var(--code-text);
+    padding: 1.25rem;
+    border-radius: 8px;
+    overflow-x: auto;
+    margin: 1.5rem 0;
+    font-family: var(--font-mono);
+    font-size: 0.85rem;
+    line-height: 1.6;
+}
+.article-content code {
+    font-family: var(--font-mono);
+    font-size: 0.85em;
+    background: var(--bg-secondary);
+    padding: 0.15em 0.4em;
+    border-radius: 3px;
+}
+.article-content pre code {
+    background: none;
+    padding: 0;
+}
+
+.article-content hr {
+    border: none;
+    border-top: 1px solid var(--border);
+    margin: 2rem 0;
+}
+
+/* ─── Tags ─── */
+.tags { display: flex; gap: 0.5rem; flex-wrap: wrap; margin-bottom: 1.5rem; }
+.tag {
+    background: var(--bg-secondary);
+    color: var(--text-secondary);
+    padding: 0.25rem 0.75rem;
+    border-radius: 20px;
+    font-size: 0.8rem;
+    text-decoration: none;
+    transition: background 0.2s, color 0.2s;
+}
+.tag:hover { background: var(--primary); color: white; }
+
+/* ─── Sources ─── */
+.sources { margin-top: 2rem; padding-top: 1rem; border-top: 1px solid var(--border); }
+.sources h3 { font-size: 1rem; color: var(--text-secondary); margin-bottom: 0.5rem; }
+
+/* ─── Related ─── */
+.related {
+    margin-top: 3rem;
+    padding-top: 1.5rem;
+    border-top: 1px solid var(--border);
+}
+
+/* ─── Footer ─── */
+.site-footer {
+    margin-top: 4rem;
+    padding: 1.5rem 0;
+    border-top: 1px solid var(--border);
+    text-align: center;
+    color: var(--text-secondary);
+    font-size: 0.85rem;
+    display: flex;
+    justify-content: space-between;
+    flex-wrap: wrap;
+    gap: 1rem;
+}
+.site-footer nav { display: flex; gap: 1.5rem; }
+.site-footer a { color: var(--text-secondary); text-decoration: none; }
+.site-footer a:hover { color: var(--primary); }
+
+/* ─── Responsive ─── */
+@media (max-width: 600px) {
+    html { font-size: 16px; }
+    body { padding: 0 1rem; }
+    .hero h1 { font-size: 1.6rem; }
+    .article-header h1 { font-size: 1.5rem; }
+    nav { flex-direction: column; align-items: flex-start; }
+}