docs: initial architecture and design for MCP Nexus
MCP Nexus is a self-hosted control plane for MCP servers — "Kubernetes for
MCP." Agents connect to one endpoint; Nexus discovers services, installs the
right MCP servers, aggregates them behind a namespaced router, secures access
with auth/RBAC, and heals/updates them via a reconcile loop.
This first commit is design-phase only (no runnable code yet):
- README.md project front door + module map
- docs/ARCHITECTURE.md target design: tenets, two-plane split, reconcile
loop, domain types, storage, security, deployment
- docs/ROADMAP.md phased delivery (foundations -> walking skeleton ->
discover+install -> secure -> operate -> extend/scale)
- docs/modules/01-20 one design doc per module, all cross-linked
Backend stack decision: Go (single static binary, embedded SQLite + embedded
React dashboard). Repo initialized in /root with a whitelist .gitignore so
only project files are tracked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
drjones
182
docs/modules/17-service-graph.md
Normal file
182
docs/modules/17-service-graph.md
Normal file
@@ -0,0 +1,182 @@
|
||||
# Service Graph
|
||||
|
||||
> Module 17 · Plane: Cross-cutting · Roadmap phase: 5
|
||||
> Part of [MCP Nexus architecture](../ARCHITECTURE.md).
|
||||
|
||||
## Purpose
|
||||
|
||||
The Service Graph builds and exposes a **live dependency graph** of everything
|
||||
Nexus mediates, from the agent all the way down to the physical device:
|
||||
|
||||
```
|
||||
Agent → Gateway → MCP server → real service → device
|
||||
Claude → Gateway → Home Assistant MCP → Home Assistant → ESPHome → Light
|
||||
```
|
||||
|
||||
It answers questions no single module can on its own: *"If Home Assistant goes
|
||||
down, which agents and tools break?"*, *"What does `github.create_issue`
|
||||
actually talk to?"*, *"Show me the full path from this agent to that light."*
|
||||
|
||||
The graph is **derived and observational** — it is assembled from
|
||||
[Inventory](../ARCHITECTURE.md), routing data (the
|
||||
[Dynamic Tool Registry](05-dynamic-tool-registry.md) tool→instance mapping),
|
||||
and [Discovery](01-discovery-engine.md) relationships. It is explicitly **not
|
||||
in the request hot path**: it never gates or slows a `tools/call`. It is a
|
||||
read-only lens over data other modules already own.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
- Assemble a typed node/edge graph spanning agents, the Gateway,
|
||||
`MCPInstance`s, discovered real services, and downstream devices.
|
||||
- **Infer edges** from existing data: routing (tool→instance), instance
|
||||
binding (`bound_resource_uuid`), and Discovery-observed relationships
|
||||
(service→device, service→dependency).
|
||||
- Keep the graph current by subscribing to the same lifecycle/health/routing
|
||||
events other data-plane modules emit.
|
||||
- Expose a **query API** (subgraph, neighbors, paths, blast-radius) for the
|
||||
Dashboard and external tooling.
|
||||
- Feed the [Dashboard](12-web-dashboard.md) an interactive, real-time
|
||||
visualization with health overlays.
|
||||
- Annotate nodes/edges with health, confidence, and last-seen so the graph
|
||||
doubles as an impact/topology map.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Serving requests or routing** — that is the [Gateway](04-gateway.md) and
|
||||
Router; the Service Graph observes them, never intercepts.
|
||||
- **Being a source of truth** — it derives from Inventory / Registry /
|
||||
Discovery and holds no independent authoritative state.
|
||||
- **Discovering infrastructure** — that is [Discovery](01-discovery-engine.md)
|
||||
and [Infra Discovery](16-infrastructure-discovery.md); the graph consumes
|
||||
their output.
|
||||
- **Alerting/notifying** — health transitions are owned by
|
||||
[Health](09-health-monitoring.md) and [Notifications](13-notifications.md);
|
||||
the graph only visualizes state.
|
||||
|
||||
## Interfaces
|
||||
|
||||
```go
|
||||
type NodeType string
|
||||
|
||||
const (
|
||||
NodeAgent NodeType = "agent" // AgentProfile
|
||||
NodeGateway NodeType = "gateway" // the single edge
|
||||
NodeInstance NodeType = "instance" // MCPInstance (the MCP server)
|
||||
NodeService NodeType = "service" // DiscoveredResource (real service)
|
||||
NodeDevice NodeType = "device" // leaf device (e.g. a light)
|
||||
)
|
||||
|
||||
type EdgeType string
|
||||
|
||||
const (
|
||||
EdgeConnects EdgeType = "connects" // agent → gateway (session)
|
||||
EdgeRoutes EdgeType = "routes" // gateway → instance (tool dispatch)
|
||||
EdgeBinds EdgeType = "binds" // instance → service (bound_resource_uuid)
|
||||
EdgeDependsOn EdgeType = "depends_on" // service → service/device (discovery)
|
||||
)
|
||||
|
||||
type Node struct {
|
||||
ID string
|
||||
Type NodeType
|
||||
Label string
|
||||
Health string // healthy | degraded | down | unknown
|
||||
Confidence float64 // for discovery-derived nodes/edges
|
||||
Ref string // uuid/instanceID/profileID it mirrors
|
||||
}
|
||||
|
||||
type Edge struct {
|
||||
From, To string
|
||||
Type EdgeType
|
||||
Health string
|
||||
Since time.Time
|
||||
}
|
||||
|
||||
// Graph is the read-only query surface. Rebuilt from events, never on the
|
||||
// request path.
|
||||
type Graph interface {
|
||||
Snapshot(ctx context.Context, f Filter) (Nodes []Node, Edges []Edge, err error)
|
||||
Neighbors(ctx context.Context, id string, depth int) ([]Node, []Edge, error)
|
||||
Path(ctx context.Context, from, to string) ([]Edge, error) // e.g. agent→device
|
||||
BlastRadius(ctx context.Context, id string) ([]Node, error) // what breaks if id fails
|
||||
Subscribe(ctx context.Context) (<-chan Delta, func()) // live updates
|
||||
}
|
||||
```
|
||||
|
||||
HTTP / API surface (all RBAC-guarded, control-plane):
|
||||
|
||||
| Method / path | Purpose |
|
||||
|---|---|
|
||||
| `GET /graph` | Full or filtered graph snapshot (JSON nodes+edges). |
|
||||
| `GET /graph/nodes/{id}/neighbors?depth=n` | Local subgraph around a node. |
|
||||
| `GET /graph/path?from=&to=` | Concrete dependency path, e.g. agent→light. |
|
||||
| `GET /graph/nodes/{id}/blast-radius` | Downstream/upstream impact set. |
|
||||
| `GET /graph/stream` | SSE stream of graph deltas for live rendering. |
|
||||
|
||||
## Data
|
||||
|
||||
- **Derived graph (in-memory, cached):** nodes and typed edges, rebuilt from
|
||||
source modules and updated incrementally from their event streams.
|
||||
- **Edge inference sources:**
|
||||
- `connects` — from active Gateway sessions (`AgentProfile` → Gateway).
|
||||
- `routes` — from the Dynamic Tool Registry `{namespace}.{tool} → instanceID`
|
||||
map (Gateway → `MCPInstance`).
|
||||
- `binds` — from `MCPInstance.bound_resource_uuid` (instance →
|
||||
`DiscoveredResource`).
|
||||
- `depends_on` — from Discovery relationships and capability probing
|
||||
(service → service, service → device); carries a **confidence** score
|
||||
(tenet #6).
|
||||
- **Persistence:** optional snapshotting to the Config/Inventory store for
|
||||
historical topology; the runtime graph is a rebuildable projection, not a
|
||||
primary store.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- [Dynamic Tool Registry](05-dynamic-tool-registry.md) — `routes` edges.
|
||||
- [Gateway](04-gateway.md) — `connects` edges (live sessions).
|
||||
- [Discovery Engine](01-discovery-engine.md) /
|
||||
[Infra Discovery](16-infrastructure-discovery.md) — services, devices, and
|
||||
`depends_on` relationships with confidence.
|
||||
- [Health Monitoring](09-health-monitoring.md) — node/edge health overlays.
|
||||
- [AI Agent Profiles](14-agent-profiles.md) — agent nodes.
|
||||
- [RBAC](08-rbac.md) — filters the graph to what the viewer may see.
|
||||
- [Web Dashboard](12-web-dashboard.md) — consumer of the visualization.
|
||||
|
||||
## Failure modes & handling
|
||||
|
||||
- **Stale source data:** nodes/edges carry `last-seen`; the graph marks
|
||||
entries `unknown` rather than deleting them on a single missed event, and
|
||||
reconciles on periodic resync.
|
||||
- **Source module unavailable:** the graph degrades gracefully — it renders the
|
||||
partial graph it can derive and flags missing regions, never blocking.
|
||||
- **Low-confidence inferred edges:** `depends_on` edges below the confidence
|
||||
threshold are rendered distinctly (dashed) and excluded from
|
||||
`BlastRadius` unless explicitly requested (tenet #6).
|
||||
- **Large graphs:** the query API paginates/filters and supports depth-bounded
|
||||
neighbor queries so the Dashboard never fetches the whole graph at once.
|
||||
- **Event lag:** because it is off the hot path, transient inconsistency is
|
||||
acceptable and self-heals on the next resync.
|
||||
|
||||
## Security notes
|
||||
|
||||
- The graph is **RBAC-filtered per viewer**: a user only sees agents,
|
||||
instances, and services their role permits (Architecture §9). Blast-radius
|
||||
and path queries respect the same filter.
|
||||
- No secrets or credentials appear on nodes/edges — the graph references
|
||||
services and devices by identity, never by connection secret.
|
||||
- Graph queries are audited like other control-plane reads.
|
||||
|
||||
## Open questions
|
||||
|
||||
- How deep does device-level (`NodeDevice`) resolution go — do we model
|
||||
ESPHome→individual-entity, or stop at the integration boundary?
|
||||
- Do we retain historical topology snapshots for time-travel/diff views, and if
|
||||
so where (Inventory vs. a dedicated store)?
|
||||
- Should `depends_on` inference incorporate observed call traffic (from
|
||||
Metrics) in addition to static discovery relationships?
|
||||
|
||||
## Milestone
|
||||
|
||||
Phase 5 (Extend & scale), alongside the [Plugin System](11-plugin-system.md)
|
||||
and HA work. **Exit contribution:** the Dashboard renders a live
|
||||
agent→gateway→MCP→service→device graph with health overlays and supports
|
||||
path and blast-radius queries over the running fleet.
|
||||
Reference in New Issue
Block a user