mcp-gateway-nexus/docs/ROADMAP.md

MCP Nexus — Roadmap

This roadmap sequences the architecture into shippable phases. Ordering principle: prove the core loop end-to-end early, then widen and harden. Each phase ends in something demonstrable, not just more scaffolding.

Guiding constraints

• Every phase must leave main runnable (after Phase 1) — no long-lived broken states.
• Data plane and control plane evolve on separate tracks; the data plane must never depend on an in-progress control-plane feature to serve traffic.
• Security is not a phase you bolt on. Auth/secrets/audit hooks are stubbed from Phase 0 and filled in Phase 3, but the seams exist from the start.

---

Phase 0 — Foundations

Goal: a Go project you can build, test, and configure. No behavior yet.

• Repo layout (cmd/nexus, internal/…, pkg/…), Makefile, CI (build + test + lint), CONTRIBUTING.md, license.
• Core domain types (§7 of Architecture): DiscoveredResource, Package, Recipe, MCPInstance, Tool, AgentProfile, Role, Secret.
• Storage layer: embedded SQLite with migrations; the six logical stores as interfaces.
• Config loader: declarative YAML → desired state; hot reload.
• Event bus + reconciler skeleton (register controllers, no controllers yet).
• Structured logging + audit-log sink interface.

Exit criteria: nexus serve boots, loads config, exposes /healthz and /metrics, persists to SQLite. Nothing MCP yet.

---

Phase 1 — Walking skeleton (data plane)

Goal: an agent connects to Nexus and calls a tool on a manually registered upstream MCP server.

• MCP server implementation (agent edge): Streamable HTTP transport, JSON-RPC 2.0, initialize/tools/list/tools/call.
• Upstream MCP client (stdio + Streamable HTTP) with a connection pool per instance.
• Router + Aggregator: multiplex N upstream servers, namespace tools (ns.tool), route tools/call to the owning instance, stream results back.
• Static instance registration via config (skip discovery/installer for now).
• nexus connect stdio↔HTTP bridge so local agents (Claude Desktop, Cursor) can attach.

Exit criteria: register a filesystem MCP + one HTTP MCP in YAML; connect Claude; see filesystem.* and other.* tools; call one successfully. This is the core value proposition proven.

Modules: Gateway, Dynamic Tool Registry.

---

Phase 2 — Discover → install (control plane)

Goal: close the reconcile loop. Discover a service, auto-install its MCP, and it shows up at the Gateway with no manual step.

• Discovery Engine v1: Docker API + mDNS/DNS-SD + HTTP(S) probing; emits DiscoveredResource with confidence.
• Fingerprinting for a starter set (Home Assistant, Postgres, Ollama, Grafana, Portainer…).
• Package Registry v1: local + GitHub/OCI package index; config schemas.
• Smart Recipes v1: match→install→config rules; a starter recipe pack.
• Auto Installer + Runtime (Docker): pull, create sandboxed container, template config, register instance.
• Reconciler wires it together: discovered→matched→installed→routed, idempotently.
• Inventory persistence + first dashboard-less status API.

Exit criteria: start a Postgres container on the LAN → within one reconcile cycle, postgres.query is live at the Gateway, no human action.

Modules: Discovery, Registry, Recipes, Installer, Infra Discovery.

---

Phase 3 — Secure (data plane hardening)

Goal: multi-agent, least-privilege access. An agent sees only what its role allows.

• Authentication: API keys first; then OIDC/OAuth; JWT sessions. Pluggable backends (LDAP/AD/SAML later).
• RBAC: roles → allowed namespaces/tools; enforced in the Router at both tools/list and tools/call.
• AI Agent Profiles: identity → roles → resolved visible tool set.
• Secrets Manager: encrypted-at-rest vault; envelope encryption; runtime injection into MCP containers; never returned to agents or logged.
• Audit: every tool call + mutating action recorded with principal/target/outcome.
• Security baseline: container sandboxing defaults, TLS termination, rate limiting, signed-package verification.

Exit criteria: two agents with different roles connect; each sees a different tool set; a secret-backed MCP works without the secret ever appearing in any agent-visible payload or log.

Modules: Auth, RBAC, Profiles, Secrets, Security.

---

Phase 4 — Operate (day-2)

Goal: it runs unattended and you can see what it's doing.

• Health Monitoring: liveness/readiness per instance; auto-restart; backoff; quarantine.
• Update Manager: watch image/release/registry updates; auto or approval-gated; version pinning; rollback.
• Metrics: Prometheus exposition (requests, latency, errors, token usage, discovery counts, instance states); Grafana dashboards.
• Notifications: Discord/Slack/Email/ntfy/Telegram/Pushover/Signal on discovery, updates, failures, offline, security alerts.
• Web Dashboard: React UI embedded in the binary — services, instances, containers, logs, health, updates, secrets, users, roles, discovery, agents.

Exit criteria: kill a managed MCP container → it's auto-restarted, a Discord alert fires, and the dashboard reflects the transition in real time.

Modules: Health, Updates, Metrics, Notifications, Dashboard.

---

Phase 5 — Extend & scale

Goal: third parties extend Nexus; it runs in production topologies.

• Plugin System: stable interfaces + out-of-process plugin transport for discovery, fingerprinters, installers, package providers, notifiers, auth, dashboards, tool transformers.
• Service Graph: live dependency graph (agent→gateway→MCP→service→device) with interactive visualization.
• Runtime adapters: containerd, Kubernetes operator, LXC.
• HA: multi-replica Nexus, shared Postgres, leader-elected reconciler.
• Community registry & recipe governance: trust/signing model for shared recipes and packages.

Exit criteria: a community-authored discovery plugin + recipe installs a new MCP with no core changes; Nexus runs 3-replica HA against Postgres.

Modules: Plugins, Service Graph, Future Vision.

---

What is intentionally not early

• Full auth backend matrix (SAML/AD) — API keys + OIDC cover the 90% first.
• K8s runtime — Docker proves the model; K8s is an adapter, not a rewrite.
• Multi-tenancy above RBAC — deferred until there's demand (see Architecture §12).
• Windows/Hyper-V discovery — Linux/Docker homelab is the beachhead.

Sequencing rationale

The riskiest, most differentiating claim is "discover a service and its MCP just appears behind one endpoint." Phases 1–2 prove exactly that as fast as possible. Everything after widens (more discovery methods, more recipes), hardens (security, ops), or scales (plugins, HA) — none of which is worth building before the core loop is real.