8d3ffef920
MCP Nexus is a self-hosted control plane for MCP servers — "Kubernetes for
MCP." Agents connect to one endpoint; Nexus discovers services, installs the
right MCP servers, aggregates them behind a namespaced router, secures access
with auth/RBAC, and heals/updates them via a reconcile loop.
This first commit is design-phase only (no runnable code yet):
- README.md project front door + module map
- docs/ARCHITECTURE.md target design: tenets, two-plane split, reconcile
loop, domain types, storage, security, deployment
- docs/ROADMAP.md phased delivery (foundations -> walking skeleton ->
discover+install -> secure -> operate -> extend/scale)
- docs/modules/01-20 one design doc per module, all cross-linked
Backend stack decision: Go (single static binary, embedded SQLite + embedded
React dashboard). Repo initialized in /root with a whitelist .gitignore so
only project files are tracked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
123 lines
7.5 KiB
Markdown
123 lines
7.5 KiB
Markdown
# MCP Nexus — Roadmap
|
||
|
||
> This roadmap sequences the [architecture](./ARCHITECTURE.md) into shippable phases.
|
||
> Ordering principle: **prove the core loop end-to-end early, then widen and harden.**
|
||
> Each phase ends in something demonstrable, not just more scaffolding.
|
||
|
||
## Guiding constraints
|
||
|
||
- Every phase must leave `main` runnable (after Phase 1) — no long-lived broken states.
|
||
- Data plane and control plane evolve on separate tracks; the data plane must never depend on an in-progress control-plane feature to serve traffic.
|
||
- Security is not a phase you bolt on. Auth/secrets/audit hooks are stubbed from Phase 0 and filled in Phase 3, but the *seams* exist from the start.
|
||
|
||
---
|
||
|
||
## Phase 0 — Foundations
|
||
|
||
**Goal:** a Go project you can build, test, and configure. No behavior yet.
|
||
|
||
- Repo layout (`cmd/nexus`, `internal/…`, `pkg/…`), Makefile, CI (build + test + lint), `CONTRIBUTING.md`, license.
|
||
- Core domain types (§7 of Architecture): `DiscoveredResource`, `Package`, `Recipe`, `MCPInstance`, `Tool`, `AgentProfile`, `Role`, `Secret`.
|
||
- Storage layer: embedded SQLite with migrations; the six logical stores as interfaces.
|
||
- Config loader: declarative YAML → desired state; hot reload.
|
||
- Event bus + reconciler skeleton (register controllers, no controllers yet).
|
||
- Structured logging + audit-log sink interface.
|
||
|
||
**Exit criteria:** `nexus serve` boots, loads config, exposes `/healthz` and `/metrics`, persists to SQLite. Nothing MCP yet.
|
||
|
||
---
|
||
|
||
## Phase 1 — Walking skeleton (data plane)
|
||
|
||
**Goal:** an agent connects to Nexus and calls a tool on a **manually registered** upstream MCP server.
|
||
|
||
- MCP server implementation (agent edge): Streamable HTTP transport, JSON-RPC 2.0, `initialize`/`tools/list`/`tools/call`.
|
||
- Upstream MCP **client** (stdio + Streamable HTTP) with a connection pool per instance.
|
||
- **Router + Aggregator:** multiplex N upstream servers, namespace tools (`ns.tool`), route `tools/call` to the owning instance, stream results back.
|
||
- Static instance registration via config (skip discovery/installer for now).
|
||
- `nexus connect` stdio↔HTTP bridge so local agents (Claude Desktop, Cursor) can attach.
|
||
|
||
**Exit criteria:** register a filesystem MCP + one HTTP MCP in YAML; connect Claude; see `filesystem.*` and `other.*` tools; call one successfully. **This is the core value proposition proven.**
|
||
|
||
Modules: [Gateway](./modules/04-gateway.md), [Dynamic Tool Registry](./modules/05-dynamic-tool-registry.md).
|
||
|
||
---
|
||
|
||
## Phase 2 — Discover → install (control plane)
|
||
|
||
**Goal:** close the reconcile loop. Discover a service, auto-install its MCP, and it shows up at the Gateway with no manual step.
|
||
|
||
- **Discovery Engine** v1: Docker API + mDNS/DNS-SD + HTTP(S) probing; emits `DiscoveredResource` with confidence.
|
||
- **Fingerprinting** for a starter set (Home Assistant, Postgres, Ollama, Grafana, Portainer…).
|
||
- **Package Registry** v1: local + GitHub/OCI package index; config schemas.
|
||
- **Smart Recipes** v1: match→install→config rules; a starter recipe pack.
|
||
- **Auto Installer** + **Runtime (Docker)**: pull, create sandboxed container, template config, register instance.
|
||
- **Reconciler** wires it together: discovered→matched→installed→routed, idempotently.
|
||
- **Inventory** persistence + first dashboard-less status API.
|
||
|
||
**Exit criteria:** start a Postgres container on the LAN → within one reconcile cycle, `postgres.query` is live at the Gateway, no human action.
|
||
|
||
Modules: [Discovery](./modules/01-discovery-engine.md), [Registry](./modules/02-package-registry.md), [Recipes](./modules/15-smart-recipes.md), [Installer](./modules/03-auto-installer.md), [Infra Discovery](./modules/16-infrastructure-discovery.md).
|
||
|
||
---
|
||
|
||
## Phase 3 — Secure (data plane hardening)
|
||
|
||
**Goal:** multi-agent, least-privilege access. An agent sees only what its role allows.
|
||
|
||
- **Authentication:** API keys first; then OIDC/OAuth; JWT sessions. Pluggable backends (LDAP/AD/SAML later).
|
||
- **RBAC:** roles → allowed namespaces/tools; enforced in the Router at both `tools/list` and `tools/call`.
|
||
- **AI Agent Profiles:** identity → roles → resolved visible tool set.
|
||
- **Secrets Manager:** encrypted-at-rest vault; envelope encryption; runtime injection into MCP containers; never returned to agents or logged.
|
||
- **Audit:** every tool call + mutating action recorded with principal/target/outcome.
|
||
- **Security baseline:** container sandboxing defaults, TLS termination, rate limiting, signed-package verification.
|
||
|
||
**Exit criteria:** two agents with different roles connect; each sees a different tool set; a secret-backed MCP works without the secret ever appearing in any agent-visible payload or log.
|
||
|
||
Modules: [Auth](./modules/06-authentication.md), [RBAC](./modules/08-rbac.md), [Profiles](./modules/14-agent-profiles.md), [Secrets](./modules/07-secrets-manager.md), [Security](./modules/19-security.md).
|
||
|
||
---
|
||
|
||
## Phase 4 — Operate (day-2)
|
||
|
||
**Goal:** it runs unattended and you can see what it's doing.
|
||
|
||
- **Health Monitoring:** liveness/readiness per instance; auto-restart; backoff; quarantine.
|
||
- **Update Manager:** watch image/release/registry updates; auto or approval-gated; version pinning; rollback.
|
||
- **Metrics:** Prometheus exposition (requests, latency, errors, token usage, discovery counts, instance states); Grafana dashboards.
|
||
- **Notifications:** Discord/Slack/Email/ntfy/Telegram/Pushover/Signal on discovery, updates, failures, offline, security alerts.
|
||
- **Web Dashboard:** React UI embedded in the binary — services, instances, containers, logs, health, updates, secrets, users, roles, discovery, agents.
|
||
|
||
**Exit criteria:** kill a managed MCP container → it's auto-restarted, a Discord alert fires, and the dashboard reflects the transition in real time.
|
||
|
||
Modules: [Health](./modules/09-health-monitoring.md), [Updates](./modules/10-update-manager.md), [Metrics](./modules/18-metrics.md), [Notifications](./modules/13-notifications.md), [Dashboard](./modules/12-web-dashboard.md).
|
||
|
||
---
|
||
|
||
## Phase 5 — Extend & scale
|
||
|
||
**Goal:** third parties extend Nexus; it runs in production topologies.
|
||
|
||
- **Plugin System:** stable interfaces + out-of-process plugin transport for discovery, fingerprinters, installers, package providers, notifiers, auth, dashboards, tool transformers.
|
||
- **Service Graph:** live dependency graph (agent→gateway→MCP→service→device) with interactive visualization.
|
||
- **Runtime adapters:** containerd, Kubernetes operator, LXC.
|
||
- **HA:** multi-replica Nexus, shared Postgres, leader-elected reconciler.
|
||
- **Community registry & recipe governance:** trust/signing model for shared recipes and packages.
|
||
|
||
**Exit criteria:** a community-authored discovery plugin + recipe installs a new MCP with no core changes; Nexus runs 3-replica HA against Postgres.
|
||
|
||
Modules: [Plugins](./modules/11-plugin-system.md), [Service Graph](./modules/17-service-graph.md), [Future Vision](./modules/20-future-vision.md).
|
||
|
||
---
|
||
|
||
## What is intentionally *not* early
|
||
|
||
- Full auth backend matrix (SAML/AD) — API keys + OIDC cover the 90% first.
|
||
- K8s runtime — Docker proves the model; K8s is an adapter, not a rewrite.
|
||
- Multi-tenancy above RBAC — deferred until there's demand (see Architecture §12).
|
||
- Windows/Hyper-V discovery — Linux/Docker homelab is the beachhead.
|
||
|
||
## Sequencing rationale
|
||
|
||
The riskiest, most differentiating claim is **"discover a service and its MCP just appears behind one endpoint."** Phases 1–2 prove exactly that as fast as possible. Everything after widens (more discovery methods, more recipes), hardens (security, ops), or scales (plugins, HA) — none of which is worth building before the core loop is real.
|