# 🧠 AI Research Engine

**Self-hosted agentic AI search infrastructure** — your private research cloud.

AI agents connect via MCP to discover, crawl, index, and synthesize knowledge from the web — all running on your own Proxmox hardware.

## Architecture

```
┌──────────────────────────────────────────────────┐
│  MacBook (thin MCP proxy)                        │
│  server.py → forwards to CT 145 backend          │
└──────────────┬───────────────────────────────────┘
               │ HTTP
┌──────────────▼───────────────────────────────────┐
│  CT 145 (10.30.20.249) — Docker Host             │
│  ┌─────────┐ ┌──────────┐ ┌───────┐ ┌─────────┐ │
│  │  YaCy   │ │OpenSearch│ │ Redis │ │ Backend │ │
│  │  :8090  │ │  :9200   │ │ :6379 │ │  :8000  │ │
│  │ crawl   │ │  index   │ │ cache │ │  API+UI │ │
│  └─────────┘ └──────────┘ └───────┘ └─────────┘ │
└──────────────────────────────────────────────────┘
               │                    │
┌──────────────▼─────┐  ┌──────────▼──────────────┐
│  CT 509 (.68)      │  │  GamingPC (.186)        │
│  Qdrant :6333      │  │  Ollama :11434          │
│  semantic search   │  │  ornith:latest (9B)     │
└────────────────────┘  └─────────────────────────┘
```

## Endpoints

| Service | URL | Purpose |
|---------|-----|---------|
| Dashboard | http://10.30.20.249:8000 | Web UI |
| Backend API | http://10.30.20.249:8000/api/* | REST API |
| OpenSearch | http://10.30.20.249:9200 | Full-text index |
| YaCy | http://10.30.20.249:8090 | Web crawler |
| Qdrant | http://10.30.20.68:6333 | Vector DB |
| Ollama | http://10.30.20.186:11434 | LLM inference |

## MCP Tools (10)

| Tool | Description |
|------|-------------|
| `search_web(query)` | Full-text search across indexed documents |
| `semantic_search(query)` | Vector search by meaning (Qdrant) |
| `crawl_url(url)` | Crawl a URL into the index |
| `crawl_topic(topic)` | Discover + crawl sources for a topic |
| `research_topic(topic)` | Full pipeline: search → crawl → summarize |
| `retrieve_document(url)` | Get full indexed document content |
| `summarize_sources(urls)` | AI summary of multiple sources |
| `extract_information(url, schema)` | Structured data extraction |
| `create_report(topic)` | Comprehensive research report |
| `index_status()` | System health check |

## Quick Start

```bash
# Check status
curl http://10.30.20.249:8000/api/status

# Search
curl "http://10.30.20.249:8000/api/search?q=knowledge+graphs"

# Crawl a URL
curl "http://10.30.20.249:8000/api/crawl?url=https://example.com&depth=1"

# Deep research
curl "http://10.30.20.249:8000/api/research?topic=LED+grow+lights"

# Generate report
curl "http://10.30.20.249:8000/api/report?topic=AI+agents"
```

## Deployment

On CT 145 (10.30.20.249):

```bash
# Services
docker run -d --name redis --restart unless-stopped -p 6379:6379 redis:7-alpine
docker run -d --name yacy --restart unless-stopped -p 8090:8090 yacy/yacy_search_server:latest
docker run -d --name opensearch --restart unless-stopped -p 9200:9200 \
  -e "discovery.type=single-node" -e "DISABLE_SECURITY_PLUGIN=true" \
  -e "OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx2g" opensearchproject/opensearch:2.17.0

# Backend
docker build -t research-backend .
docker run -d --name research-backend --restart unless-stopped -p 8000:8000 \
  --add-host=host.docker.internal:host-gateway \
  -e YACY_URL=http://host.docker.internal:8090 \
  -e OPENSEARCH_URL=http://host.docker.internal:9200 \
  -e QDRANT_URL=http://10.30.20.68:6333 \
  -e OLLAMA_URL=http://10.30.20.186:11434 \
  research-backend
```

## Files

- `server.py` — Thin MCP proxy (runs on MacBook)
- `backend.py` — REST API + Dashboard (runs on CT 145)
- `docker-compose.yml` — Reference compose file
- `Dockerfile` — Backend container build
- `.env` — Service endpoints config