mcp-gateway-nexus/docs/modules/04-gateway.md

Gateway

Module 04 · Plane: Data · Roadmap phase: 1 Part of MCP Nexus architecture.

Purpose

The Gateway is the single MCP endpoint every AI agent connects to — https://mcp.company.lan. It embodies tenet #1 ("one endpoint, many agents"): to an agent the Gateway is an MCP server; upstream, together with the Router (the "MCP Router + Aggregator" of Architecture §4), it acts as an MCP client to every managed MCPInstance. It dynamically aggregates all healthy upstream MCP servers behind one address and namespaces their tools so homeassistant.turn_on, postgres.query, and github.create_issue never collide.

The Gateway owns the edge and transport. It does not own tool ownership or dispatch (that is the Router) and it does not own the tool catalog (that is the Dynamic Tool Registry). Keeping these three concerns separate is deliberate: the Gateway can be restarted, load-balanced, and hardened without touching routing logic or the catalog.

Responsibilities

• Terminate TLS and accept agent connections over Streamable HTTP (the current MCP HTTP transport) speaking JSON-RPC 2.0.
• Run the MCP server side of the initialize handshake: negotiate protocol version, advertise server capabilities (tools, and later resources / prompts), and open a session.
• Manage the per-session request lifecycle: parse JSON-RPC frames, correlate ids, enforce timeouts, and stream partial/chunked results back to the agent.
• Delegate every request to Auth (who is calling?) and RBAC (what may they see/call?) before it reaches the Router.
• Hand tools/list and tools/call to the Router, which resolves the owning instance from the Dynamic Tool Registry and dispatches over the upstream client pool.
• Maintain the connection pool to upstream MCP servers (stdio and Streamable HTTP), reusing warm connections across agent requests.
• Ship the nexus connect stdio↔HTTP bridge so local agents (Claude Desktop, Cursor) that only speak stdio can attach to the HTTP endpoint.
• Emit per-request audit and metrics events.

Non-goals

• Tool ownership / dispatch decisions — owned by the Router.
• The tool catalog and its freshness — owned by the Dynamic Tool Registry.
• AuthN/AuthZ policy — the Gateway calls Auth/RBAC; it stores no policy.
• Installing, healing, or updating instances — control-plane concern; the Gateway only consumes the healthy upstream set.
• Secret storage or injection — secrets reach MCP containers via the Secrets Manager, never through the Gateway.

Interfaces

// Server is the agent-facing MCP server. One per Nexus process.
type Server interface {
	// Serve binds the Streamable HTTP listener (TLS terminated here).
	Serve(ctx context.Context, addr string, tls *tls.Config) error
	// Handle processes one JSON-RPC request within an authenticated session
	// and writes the response (possibly streamed) to w.
	Handle(ctx context.Context, s *Session, req *jsonrpc.Request, w StreamWriter) error
}

// Session is a live agent connection after a successful initialize.
type Session struct {
	ID        string
	Principal auth.Principal // resolved by Auth
	Roles     []rbac.Role    // resolved by RBAC → visible tool set
	Protocol  string         // negotiated MCP version
	Caps      ClientCaps
}

// StreamWriter delivers incremental JSON-RPC results / progress notifications.
type StreamWriter interface {
	WriteChunk(any) error
	Close(err error) error
}

// UpstreamPool hands the Router warm MCP client connections per instance.
type UpstreamPool interface {
	Client(ctx context.Context, instanceID string) (mcpclient.Conn, release func(), error)
	Drain(instanceID string) // called when Health marks an instance gone
}

HTTP / MCP endpoints exposed at the edge:

Method / path	Purpose
POST /mcp	Streamable HTTP MCP endpoint; carries JSON-RPC initialize, tools/list, tools/call, notifications.
GET /mcp	Server→client stream channel (SSE-style) for long-lived sessions.
GET /healthz	Liveness of the edge (does not gate on upstream health).

nexus connect --url https://mcp.company.lan --token … runs a local process that presents stdio to the agent and proxies frames to POST/GET /mcp.

Data

The Gateway is mostly stateless — it holds only in-memory session and pool state, so replicas behind a load balancer are interchangeable (HA topology, Architecture §10).

• Session table (in-memory): sessionID → Session, TTL-expired.
• Connection pool (in-memory): per-MCPInstance warm mcpclient.Conns, keyed by instanceID, with idle eviction and per-instance max.
• Persisted: nothing of its own. It reads the healthy instance set and the RBAC-filtered tool view via the Registry/Router, and writes audit + metrics events to the shared Audit/Metrics stores.

Dependencies

• Authentication — resolves the calling principal.
• RBAC — resolves the visible/callable tool set per session.
• Dynamic Tool Registry — source of the live {namespace}.{tool} catalog and tool→instance mapping.
• AI Agent Profiles — maps agent identity to roles.
• Health Monitoring — signals which instances are poolable; drives UpstreamPool.Drain.
• Secrets Manager — ensures no secret transits the edge in an agent-visible payload.
• Metrics / Security — audit, TLS, limits.

Failure modes & handling

• Upstream instance down mid-call: the pool returns a transport error; the Router surfaces a JSON-RPC error to the agent. The Gateway never hangs on a dead upstream — every dispatch is deadline-bounded.
• Upstream slow: per-request deadline + circuit breaker per instance; streamed progress keeps the agent connection alive until the deadline.
• Auth/RBAC unavailable: fail closed — reject with a JSON-RPC error; never fall back to unauthenticated access (tenet #5).
• Pool exhaustion: bounded per-instance connections + a wait queue with a timeout; over-limit calls get a retryable error, not unbounded fan-out.
• Malformed JSON-RPC: respond with the spec error code; drop the frame, keep the session.
• Bridge disconnect (nexus connect): local process retries with backoff; the HTTP session is TTL-reaped if the bridge does not resume.

Security notes

• TLS terminates here (Architecture §9); internal calls to Router/Auth are over localhost/socket.
• The Gateway enforces that a session may only ever see and invoke its RBAC-visible tool set — requests are scoped before the Router dispatches, so an agent cannot call a tool outside its role even by guessing its name.
• Per-session and per-principal rate limiting at the edge.
• No secret material is ever echoed back to an agent; error messages from upstreams are sanitized.
• Every tools/call is audited with principal, resolved instance, and outcome.

Open questions

• Session affinity vs. stateless replicas: do we need sticky sessions for streamed calls behind the HA load balancer, or is a shared session store sufficient?
• Should the nexus connect bridge be a subcommand of the main binary or a separately distributed micro-binary (relates to Architecture §12)?
• Streamable HTTP resumability: how far do we implement MCP resumable streams vs. requiring the agent to re-issue on reconnect?

Milestone

Phase 1 (Walking skeleton). Ships with the MCP server edge, upstream client pool, and nexus connect. Exit: register a filesystem MCP + one HTTP MCP in YAML, connect Claude, see filesystem.* and other.* tools, and call one successfully — the core value proposition proven end-to-end.