diff --git a/README.md b/README.md index 6dca56f..44355e6 100644 --- a/README.md +++ b/README.md @@ -1,187 +1,144 @@ # DRACO — The Book-Forged Code Oracle -> The smartest coding AI you can **prove**. DRACO answers coding, Linux, systems, and -> security questions using retrieval (BM25 RAG) over **hundreds of real technical books** -> — the coding canon and the hacker's shelf — and cites the exact book behind every -> answer. Talk to it in the browser, call the REST API, or summon it as an **MCP tool** -> from Claude, GPT, or any agent. +> **Other AIs guess. DRACO knows — and proves it.** -- **Live:** https://draco.thetempleofdoom.com -- **GitHub:** https://github.com/drjonesxxx1/draco -- **Repo:** http://10.30.20.149:3000/drjones/draco -- **Host:** Proxmox CT 174 `draco` @ `10.30.20.12` (Debian 13, 4GB RAM / 4 cores) -- **LLM:** `ornith-1.5:9b-64k` on bare-metal Ollama @ `10.30.20.29` (nightmare, 4090S) — abliterated/heretic lineage, zero-spill resident -- **Stack:** Flask + BM25 (rank_bm25 + Snowball stem) + SSE streaming + BTCPay + MCP streamable-http +DRACO answers coding, systems, and security questions using retrieval over a library of +**1,855 real technical books** (354M characters, 116,558 indexed passages) — and cites the +exact book behind every claim. No knowledge cutoff. No hallucination laundering. Receipts attached. + +**Try it now:** https://draco.thetempleofdoom.com — 10 free questions a day, no signup. --- -## Why "book-forged" +## Why it's different -Most assistants guess. DRACO retrieves. Every question is embedded-scanned against the -full library (BM25 over stemmed tokens), the top passages are stuffed into the prompt, -and the model must cite them `[n]`. The response includes a `sources` array — -**check the receipts yourself**. That's the proof behind the homepage brag. - -## Architecture - -``` - ┌────────────────────── CT 174 "draco" (10.30.20.12) ─────────────────────┐ - │ nginx :80 ──► gunicorn :8012 (app.py + FastMCP mounted at /mcp) │ -browser ── CF tunnel│ │ │ -agent ──► /mcp ────┤ /api/chat (SSE) /api/ask /api/search /api/signup /webhook/btcpay │ - │ │ │ - │ draco_core: BM25 index (~N chunks) │ sqlite: users/credits/payments│ - │ │ │ │ - │ /opt/books: original pdf/chm/djvu + text/ + manifest.json + index/ │ - └────────────────────────────────────────┼────────────────────────────────┘ - ▼ - Ollama ornith-1.5:9b-64k @ 10.30.20.29:11434 (bare metal) -``` - -## The library - -| Collection | Contents | Source | -|---|---|---| -| `library_linux` | The coding canon: C/C++, Python, Perl, PHP, Java, assembly, SQL, web, Unix/Linux internals, networks & security, math, Apache | `/Volumes/sanD/library linux` | -| `hackerpack` | Hacker Pro Pack: exploitation, malware, network attacks, crypto, wireless, scene classics | `/Volumes/sanD/dw stuff 3/hacking/HACKER PRO PACK books` | - -**Conversion pipeline** (`ingest.py`, runs on the CT): - -``` -pdf → pdftotext -layout (poppler-utils) -chm → extract_chmLib → walk html/txt → strip tags → unescape -djvu → djvutxt (djvulibre-bin) -txt → passthrough -then: null-strip, whitespace collapse, <200 chars = scanned junk → dropped -→ /opt/books/text//_.txt + manifest.json (id/title/category/chars/…) -``` - -**Index** (`build_index.py`): 300-token chunks, 60 overlap → Snowball-stem → `BM25Okapi` -→ `index/index.pkl` + `chunks.json`. Rebuild = re-run both scripts; app picks up on restart. - ---- - -## Web UI - -| Route | What | +| Generic chatbot | DRACO | |---|---| -| `/` | Hero (the brag, with live library stats) + **chat** — 10 free questions/day/IP, streaming, sources bar under every answer | -| `/library` | The whole hoard, filterable, every book **downloadable** | -| `/pricing` | Credit packs, Bitcoin-only | -| `/api` | Key signup + copy-paste curl examples | -| `/health` | `{"status":"ok","llm":true,"model":...,"books":...,"chunks":...}` | +| Compressed training-data recall | Live retrieval from 1,855 books on the shelf | +| "Trust me" answers | Every claim carries `[n]` citations + a `sources` array | +| Knowledge cutoff: yes | The canon is resident — K&R, Tanenbaum, the shellcoders' handbook | +| Sells your prompts | Self-hosted, zero telemetry, questions never leave the house | -## REST API +## Quickstart -Auth: `X-API-Key` header (or `api_key` JSON field). Free tier = 30 credits/month (email signup). Credits never expire. +### Chat in the browser +https://draco.thetempleofdoom.com — streaming answers, visible reasoning, sources bar +under every response. 10 free/day, no account. +### REST API ```bash -# get a key +# 1. Get a free key (30 credits/month, email only, no card) curl -s -X POST https://draco.thetempleofdoom.com/api/signup \ + -H 'Content-Type: application/json' \ -d '{"email":"you@domain.tld"}' # → {"api_key":"sk-draco-..."} -# ask (RAG + citations) — 1 credit +# 2. Ask — RAG-grounded answer with citations (1 credit) curl -s -X POST https://draco.thetempleofdoom.com/api/ask \ -H 'X-API-Key: sk-draco-...' -H 'Content-Type: application/json' \ -d '{"q":"explain the ret2libc technique with a minimal poc"}' -# → {"answer":"...[1]...[2]","sources":[{"title":"...","category":"hackerpack",...}]} +# → {"answer":"...[1]...[2]","sources":[{"title":"...","category":"..."}]} -# raw BM25 passage search — 1 credit +# 3. Raw passage search across all 1,855 books (1 credit) curl -s "https://draco.thetempleofdoom.com/api/search?q=tcp%20syn%20flood&k=5" \ -H 'X-API-Key: sk-draco-...' - -# usage / credits -curl -s https://draco.thetempleofdoom.com/api/my-usage -H 'X-API-Key: sk-draco-...' - -# buy credits (Bitcoin via BTCPay) → checkout_url -curl -s -X POST https://draco.thetempleofdoom.com/api/create-invoice \ - -H 'X-API-Key: sk-draco-...' -d '{"plan":"crate"}' ``` -| Code | Meaning | -|---|---| -| 200 | answer + sources | -| 401 | missing/invalid key | -| 402 | out of credits (free tier exhausted) | -| 429 | web anon limit (10/day) hit | -| 503 | Ollama unreachable | - -## MCP — summon as a tool call - -**Remote (any MCP client, zero install):** - +### MCP — use it from Claude, GPT, or any agent ```json -{ "mcpServers": { "draco": { "url": "https://draco.thetempleofdoom.com/mcp" } } } +{ + "mcpServers": { + "draco": { "url": "https://draco.thetempleofdoom.com/mcp" } + } +} ``` -Tools: `draco_ask(question)` → cited answer · `draco_search(query, k)` → raw passages · -`draco_status()` → health + library stats. +**Tools:** +- `draco_ask(question)` → cited, book-grounded answer +- `draco_search(query, k)` → raw BM25 passages with titles + scores +- `draco_status()` → live library stats -**Published:** official MCP registry — `com.thetempleofdoom.draco/draco` **v1.1.0** (active, with repo metadata): -https://registry.modelcontextprotocol.io/servers/com.thetempleofdoom.draco/draco -Domain proof: TXT `v=MCPv1` on draco.thetempleofdoom.com (shared fleet ed25519 identity). - -Streamable-http at `/mcp`. Smoke test by hand: +**Officially listed on the MCP registry:** +[`com.thetempleofdoom.draco/draco`](https://registry.modelcontextprotocol.io/servers/com.thetempleofdoom.draco/draco) +Manual MCP handshake: ```bash curl -s https://draco.thetempleofdoom.com/mcp \ -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \ - -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"x","version":"1"}}}' + -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"smoke","version":"1"}}}' ``` -**Local stdio client for your own machine** — `mcp_client.py` in this repo (talks to the -remote HTTP MCP; usable from Claude Desktop via `mcp-remote` or any stdio bridge). +## How it works -Machine-readable discovery: `/llms.txt`, `/llms-full.txt`, `/openapi.json`, -`/.well-known/ai-plugin.json`, `/.well-known/mcp-server.json`, `robots.txt` (all AI crawlers allowed), `/sitemap.xml`. +``` +question ──► BM25 retrieval over 116k book passages + │ + ▼ + top passages + question ──► local LLM (self-hosted GPU) + │ + ▼ + answer with inline [n] citations + sources array +``` -## Pricing (Bitcoin only, BTCPay, no KYC) +1. **Ingest** (`ingest.py`) — PDF/CHM/DJVU → clean plain text. Junk scans dropped automatically. +2. **Index** (`build_index.py`) — 300-token overlapping chunks → Snowball-stemmed BM25 index. +3. **Serve** (Flask + gunicorn) — retrieval-augmented generation with SSE streaming, + separate reasoning and answer channels, and automatic degenerate-output detection/retry. +4. **Expose** — REST API, streamable-http MCP server, and a full agent-discovery kit: + `/llms.txt`, `/llms-full.txt`, `/openapi.json`, `/.well-known/ai-plugin.json`, + `/.well-known/mcp-server.json`, `/sitemap.xml`. -| Tier | Price | Gets | +## Pricing + +Bitcoin only. No subscription traps, no KYC, no card processor. + +| Tier | Price | What you get | |---|---|---| -| Web anon | free | 10 questions/day in browser | -| Free API key | free | 30 credits/month | -| **satchel** | $3 | 60 credits | -| **crate** | $10 | 250 credits | -| **hoard** | $25 | 750 credits | +| Web (no key) | **free** | 10 questions/day in the browser | +| API key | **free** | 30 credits/month, forever | +| satchel | $3 | 60 credits | +| crate | $10 | 250 credits | +| hoard pack | $25 | 750 credits | +| **HOARD** | **$50/mo** | **unlimited** — every ask, every search, 30 days | -1 credit = 1 ask or 1 search. Credits never expire. Invoice webhook (`/webhook/btcpay?wh=`) -credits the key on `InvoiceSettled`. Books are always free to download — we sell answers, not files. +1 credit = 1 ask or 1 search. Credits never expire. Books are free to download at +[/library](https://draco.thetempleofdoom.com/library) — we sell answers, not files. -## Deploy runbook (CT 174) +## Self-hosting ```bash -# code push — the golden path -cd ~/draco && tar czf /tmp/d.tar.gz *.py config.json requirements.txt -scp /tmp/d.tar.gz root@10.30.20.85:/tmp/ -ssh root@10.30.20.85 "pct push 174 /tmp/d.tar.gz /tmp/d.tar.gz" -ssh root@10.30.20.85 "pct exec 174 -- bash -c 'cd /opt/draco && tar xzf /tmp/d.tar.gz && systemctl restart draco'" -curl -s https://draco.thetempleofdoom.com/health +git clone https://github.com/drjonesxxx1/draco.git +cd draco +python3 -m venv venv && ./venv/bin/pip install -r requirements.txt +cp config.example.json config.json # set model, ollama URL, limits, payments -# rebuild the corpus (books → text → index) on the CT -ssh root@10.30.20.85 "pct exec 174 -- systemd-run --unit=draco-ingest \ - bash -c 'cd /opt/draco && ./venv/bin/python3 ingest.py && ./venv/bin/python3 build_index.py && systemctl restart draco'" +# point ingest.py at YOUR book directory, then: +./venv/bin/python3 ingest.py # books → text + manifest +./venv/bin/python3 build_index.py # text → BM25 index + +./venv/bin/gunicorn -w 2 --threads 8 -b 127.0.0.1:8012 app:app # API + web +./venv/bin/python3 -m uvicorn mcp_server:app --port 8013 # MCP ``` -Layout on CT: `/opt/draco` (code, venv, draco.db) · `/opt/books` (books + text/ + index/) · -services `draco.service` + `nginx` · gunicorn binds 127.0.0.1:8012. +Any local Ollama model works. Put nginx in front (see `deploy/nginx-draco.conf` for +the reference config incl. SSE + MCP proxying). Payments are optional — leave the +`btcpay` block empty and the free tiers carry the whole thing. -## Verify everything (the checklist that was actually run) +## Repository layout -```bash -curl -s https://draco.thetempleofdoom.com/health # llm:true, books>0 -curl -s https://draco.thetempleofdoom.com/llms.txt # starts with "# DRACO" -curl -s https://draco.thetempleofdoom.com/robots.txt # GPTBot + Allow: / -curl -s https://draco.thetempleofdoom.com/api/library | jq .count -curl -s https://draco.thetempleofdoom.com/api/search?q=nmap -H 'X-API-Key: ' # hits -# MCP handshake + tools/call against /mcp -``` +| File | Purpose | +|---|---| +| `app.py` | Flask: web UI, REST API, SSE chat, payments webhook, agent-discovery kit | +| `draco_core.py` | RAG engine, retrieval, LLM calls, metering, DB | +| `pages.py` / `pages2.py` | Theme | +| `mcp_server.py` | Standalone streamable-http MCP server | +| `mcp_client.py` | stdio bridge for Claude Desktop / local clients | +| `ingest.py` / `build_index.py` | Corpus pipeline | +| `test_splitter.py` | Unit tests for the reasoning/answer channel splitter | +| `deploy/` | systemd units + nginx config | ## Notes - Security content is for **education and authorized testing only**. -- `config.json` carries runtime knobs (model, k, limits, plans, BTCPay) — admin key + - BTCPay store creds live there on the CT, **never** in the repo (config.json is - git-ignored; a sanitized `config.example.json` is tracked). - Books retain their original copyrights; the library is shared for personal use. +- No telemetry. No logs sold. Ever.