docs: initial architecture and design for MCP Nexus
MCP Nexus is a self-hosted control plane for MCP servers — "Kubernetes for
MCP." Agents connect to one endpoint; Nexus discovers services, installs the
right MCP servers, aggregates them behind a namespaced router, secures access
with auth/RBAC, and heals/updates them via a reconcile loop.
This first commit is design-phase only (no runnable code yet):
- README.md project front door + module map
- docs/ARCHITECTURE.md target design: tenets, two-plane split, reconcile
loop, domain types, storage, security, deployment
- docs/ROADMAP.md phased delivery (foundations -> walking skeleton ->
discover+install -> secure -> operate -> extend/scale)
- docs/modules/01-20 one design doc per module, all cross-linked
Backend stack decision: Go (single static binary, embedded SQLite + embedded
React dashboard). Repo initialized in /root with a whitelist .gitignore so
only project files are tracked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
drjones
169
docs/modules/04-gateway.md
Normal file
169
docs/modules/04-gateway.md
Normal file
@@ -0,0 +1,169 @@
|
||||
# Gateway
|
||||
|
||||
> Module 04 · Plane: Data · Roadmap phase: 1
|
||||
> Part of [MCP Nexus architecture](../ARCHITECTURE.md).
|
||||
|
||||
## Purpose
|
||||
|
||||
The Gateway is **the single MCP endpoint every AI agent connects to** —
|
||||
`https://mcp.company.lan`. It embodies tenet #1 ("one endpoint, many
|
||||
agents"): to an agent the Gateway *is* an MCP server; upstream, together with
|
||||
the **Router** (the "MCP Router + Aggregator" of Architecture §4), it acts as
|
||||
an MCP client to every managed
|
||||
[`MCPInstance`](../ARCHITECTURE.md). It dynamically **aggregates** all healthy
|
||||
upstream MCP servers behind one address and **namespaces** their tools so
|
||||
`homeassistant.turn_on`, `postgres.query`, and `github.create_issue` never
|
||||
collide.
|
||||
|
||||
The Gateway owns the *edge and transport*. It does not own tool ownership or
|
||||
dispatch (that is the Router) and it does not own the tool
|
||||
catalog (that is the [Dynamic Tool Registry](05-dynamic-tool-registry.md)).
|
||||
Keeping these three concerns separate is deliberate: the Gateway can be
|
||||
restarted, load-balanced, and hardened without touching routing logic or the
|
||||
catalog.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
- Terminate TLS and accept agent connections over **Streamable HTTP** (the
|
||||
current MCP HTTP transport) speaking **JSON-RPC 2.0**.
|
||||
- Run the MCP server side of the `initialize` handshake: negotiate protocol
|
||||
version, advertise server capabilities (`tools`, and later `resources` /
|
||||
`prompts`), and open a session.
|
||||
- Manage the per-session request lifecycle: parse JSON-RPC frames, correlate
|
||||
ids, enforce timeouts, and stream partial/chunked results back to the agent.
|
||||
- Delegate every request to [Auth](06-authentication.md) (who is calling?) and
|
||||
[RBAC](08-rbac.md) (what may they see/call?) *before* it reaches the Router.
|
||||
- Hand `tools/list` and `tools/call` to the Router, which resolves the owning
|
||||
instance from the Dynamic Tool Registry and dispatches over the upstream
|
||||
client pool.
|
||||
- Maintain the **connection pool** to upstream MCP servers (stdio and
|
||||
Streamable HTTP), reusing warm connections across agent requests.
|
||||
- Ship the `nexus connect` **stdio↔HTTP bridge** so local agents (Claude
|
||||
Desktop, Cursor) that only speak stdio can attach to the HTTP endpoint.
|
||||
- Emit per-request audit and metrics events.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Tool ownership / dispatch decisions** — owned by the Router.
|
||||
- **The tool catalog and its freshness** — owned by the Dynamic Tool Registry.
|
||||
- **AuthN/AuthZ policy** — the Gateway *calls* Auth/RBAC; it stores no policy.
|
||||
- **Installing, healing, or updating instances** — control-plane concern; the
|
||||
Gateway only consumes the healthy upstream set.
|
||||
- **Secret storage or injection** — secrets reach MCP containers via the
|
||||
[Secrets Manager](07-secrets-manager.md), never through the Gateway.
|
||||
|
||||
## Interfaces
|
||||
|
||||
```go
|
||||
// Server is the agent-facing MCP server. One per Nexus process.
|
||||
type Server interface {
|
||||
// Serve binds the Streamable HTTP listener (TLS terminated here).
|
||||
Serve(ctx context.Context, addr string, tls *tls.Config) error
|
||||
// Handle processes one JSON-RPC request within an authenticated session
|
||||
// and writes the response (possibly streamed) to w.
|
||||
Handle(ctx context.Context, s *Session, req *jsonrpc.Request, w StreamWriter) error
|
||||
}
|
||||
|
||||
// Session is a live agent connection after a successful initialize.
|
||||
type Session struct {
|
||||
ID string
|
||||
Principal auth.Principal // resolved by Auth
|
||||
Roles []rbac.Role // resolved by RBAC → visible tool set
|
||||
Protocol string // negotiated MCP version
|
||||
Caps ClientCaps
|
||||
}
|
||||
|
||||
// StreamWriter delivers incremental JSON-RPC results / progress notifications.
|
||||
type StreamWriter interface {
|
||||
WriteChunk(any) error
|
||||
Close(err error) error
|
||||
}
|
||||
|
||||
// UpstreamPool hands the Router warm MCP client connections per instance.
|
||||
type UpstreamPool interface {
|
||||
Client(ctx context.Context, instanceID string) (mcpclient.Conn, release func(), error)
|
||||
Drain(instanceID string) // called when Health marks an instance gone
|
||||
}
|
||||
```
|
||||
|
||||
HTTP / MCP endpoints exposed at the edge:
|
||||
|
||||
| Method / path | Purpose |
|
||||
|---|---|
|
||||
| `POST /mcp` | Streamable HTTP MCP endpoint; carries JSON-RPC `initialize`, `tools/list`, `tools/call`, notifications. |
|
||||
| `GET /mcp` | Server→client stream channel (SSE-style) for long-lived sessions. |
|
||||
| `GET /healthz` | Liveness of the edge (does not gate on upstream health). |
|
||||
|
||||
`nexus connect --url https://mcp.company.lan --token …` runs a local process
|
||||
that presents stdio to the agent and proxies frames to `POST/GET /mcp`.
|
||||
|
||||
## Data
|
||||
|
||||
The Gateway is **mostly stateless** — it holds only in-memory session and pool
|
||||
state, so replicas behind a load balancer are interchangeable (HA topology,
|
||||
Architecture §10).
|
||||
|
||||
- **Session table (in-memory):** `sessionID → Session`, TTL-expired.
|
||||
- **Connection pool (in-memory):** per-`MCPInstance` warm `mcpclient.Conn`s,
|
||||
keyed by `instanceID`, with idle eviction and per-instance max.
|
||||
- **Persisted:** nothing of its own. It *reads* the healthy instance set and
|
||||
the RBAC-filtered tool view via the Registry/Router, and *writes* audit +
|
||||
metrics events to the shared Audit/Metrics stores.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- [Authentication](06-authentication.md) — resolves the calling principal.
|
||||
- [RBAC](08-rbac.md) — resolves the visible/callable tool set per session.
|
||||
- [Dynamic Tool Registry](05-dynamic-tool-registry.md) — source of the live
|
||||
`{namespace}.{tool}` catalog and tool→instance mapping.
|
||||
- [AI Agent Profiles](14-agent-profiles.md) — maps agent identity to roles.
|
||||
- [Health Monitoring](09-health-monitoring.md) — signals which instances are
|
||||
poolable; drives `UpstreamPool.Drain`.
|
||||
- [Secrets Manager](07-secrets-manager.md) — ensures no secret transits the
|
||||
edge in an agent-visible payload.
|
||||
- [Metrics](18-metrics.md) / [Security](19-security.md) — audit, TLS, limits.
|
||||
|
||||
## Failure modes & handling
|
||||
|
||||
- **Upstream instance down mid-call:** the pool returns a transport error; the
|
||||
Router surfaces a JSON-RPC error to the agent. The Gateway never hangs on a
|
||||
dead upstream — every dispatch is deadline-bounded.
|
||||
- **Upstream slow:** per-request deadline + circuit breaker per instance;
|
||||
streamed progress keeps the agent connection alive until the deadline.
|
||||
- **Auth/RBAC unavailable:** fail **closed** — reject with a JSON-RPC error;
|
||||
never fall back to unauthenticated access (tenet #5).
|
||||
- **Pool exhaustion:** bounded per-instance connections + a wait queue with a
|
||||
timeout; over-limit calls get a retryable error, not unbounded fan-out.
|
||||
- **Malformed JSON-RPC:** respond with the spec error code; drop the frame,
|
||||
keep the session.
|
||||
- **Bridge disconnect (`nexus connect`):** local process retries with backoff;
|
||||
the HTTP session is TTL-reaped if the bridge does not resume.
|
||||
|
||||
## Security notes
|
||||
|
||||
- **TLS terminates here** (Architecture §9); internal calls to Router/Auth are
|
||||
over localhost/socket.
|
||||
- The Gateway enforces that a session may only ever see and invoke its
|
||||
RBAC-visible tool set — requests are scoped *before* the Router dispatches,
|
||||
so an agent cannot call a tool outside its role even by guessing its name.
|
||||
- Per-session and per-principal **rate limiting** at the edge.
|
||||
- No secret material is ever echoed back to an agent; error messages from
|
||||
upstreams are sanitized.
|
||||
- Every `tools/call` is audited with principal, resolved instance, and outcome.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Session affinity vs. stateless replicas: do we need sticky sessions for
|
||||
streamed calls behind the HA load balancer, or is a shared session store
|
||||
sufficient?
|
||||
- Should the `nexus connect` bridge be a subcommand of the main binary or a
|
||||
separately distributed micro-binary (relates to Architecture §12)?
|
||||
- Streamable HTTP resumability: how far do we implement MCP resumable streams
|
||||
vs. requiring the agent to re-issue on reconnect?
|
||||
|
||||
## Milestone
|
||||
|
||||
Phase 1 (Walking skeleton). Ships with the MCP server edge, upstream client
|
||||
pool, and `nexus connect`. **Exit:** register a filesystem MCP + one HTTP MCP
|
||||
in YAML, connect Claude, see `filesystem.*` and `other.*` tools, and call one
|
||||
successfully — the core value proposition proven end-to-end.
|
||||
Reference in New Issue
Block a user