docs: initial architecture and design for MCP Nexus
MCP Nexus is a self-hosted control plane for MCP servers — "Kubernetes for
MCP." Agents connect to one endpoint; Nexus discovers services, installs the
right MCP servers, aggregates them behind a namespaced router, secures access
with auth/RBAC, and heals/updates them via a reconcile loop.
This first commit is design-phase only (no runnable code yet):
- README.md project front door + module map
- docs/ARCHITECTURE.md target design: tenets, two-plane split, reconcile
loop, domain types, storage, security, deployment
- docs/ROADMAP.md phased delivery (foundations -> walking skeleton ->
discover+install -> secure -> operate -> extend/scale)
- docs/modules/01-20 one design doc per module, all cross-linked
Backend stack decision: Go (single static binary, embedded SQLite + embedded
React dashboard). Repo initialized in /root with a whitelist .gitignore so
only project files are tracked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
drjones
123
docs/modules/15-smart-recipes.md
Normal file
123
docs/modules/15-smart-recipes.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# Smart Recipes
|
||||
|
||||
> Module 15 · Plane: Control · Roadmap phase: 2
|
||||
> Part of [MCP Nexus architecture](../ARCHITECTURE.md).
|
||||
|
||||
## Purpose
|
||||
Provide declarative **YAML rules that map a fingerprint to an MCP install** — no code required to support a new service. A recipe binds `match` → `install` → `config`, and is the **desired-state input** to the reconciler: given a `DiscoveredResource` (from [Discovery Engine](01-discovery-engine.md)) and a `Package` (from the [Registry](02-package-registry.md)), a matching recipe resolves into a concrete `MCPInstance` spec for the [Auto Installer](03-auto-installer.md).
|
||||
|
||||
## Responsibilities
|
||||
- Define the **Recipe schema** (Architecture §7 `Recipe`): `match`, `install`, `config`, plus metadata (`name`, `version`, `priority`).
|
||||
- **Match** a `DiscoveredResource` against recipe predicates (fingerprint `type`, open `ports`, `http_title`, capabilities, version constraints, evidence fields) and produce a **match score**.
|
||||
- **Resolve precedence** when multiple recipes match a single resource, deterministically.
|
||||
- **Render config templates** — substitute variables (`{{ip}}`, `{{port}}`, `{{hostname}}`, `{{version}}`) and **secret references** into the config a `Package`'s `config_schema` expects.
|
||||
- Reference **Registry packages** (by name + optional version constraint) and **Secrets** (by ref, never inline).
|
||||
- Ship a **starter recipe pack** and load user/community recipes from the Config store.
|
||||
- Validate recipes against package `config_schema` and report actionable errors.
|
||||
|
||||
## Non-goals
|
||||
- **Does not discover or fingerprint** — it consumes `DiscoveredResource`s from [Discovery Engine](01-discovery-engine.md).
|
||||
- **Does not resolve or verify packages** — that is the [Registry](02-package-registry.md) (signatures, digests, versions).
|
||||
- **Does not install or run** the resulting spec — the [Auto Installer](03-auto-installer.md) + Runtime do.
|
||||
- **Does not store or decrypt secrets** — it emits secret *refs*; [Secrets](07-secrets-manager.md) injects values at container runtime.
|
||||
- **Does not own the reconcile loop** — it is a pure(ish) resolver the reconciler calls.
|
||||
|
||||
## Interfaces
|
||||
```go
|
||||
// A Recipe is a declarative match+install+config rule (Architecture §7).
|
||||
type Recipe struct {
|
||||
Name string
|
||||
Version string
|
||||
Priority int // tie-breaker; higher wins
|
||||
Match MatchSpec // predicates over a DiscoveredResource
|
||||
Install InstallSpec // package ref (docker_image / package name)
|
||||
Config map[string]any // templated values -> Package.config_schema
|
||||
}
|
||||
|
||||
// The engine matches, scores, and resolves recipes into instance specs.
|
||||
type Engine interface {
|
||||
Load(ctx context.Context) error // from Config store
|
||||
Match(r DiscoveredResource) []Scored // sorted, best first
|
||||
Resolve(r DiscoveredResource, pick Recipe) (MCPInstanceSpec, error)
|
||||
Validate(rec Recipe, pkg Package) error // against config_schema
|
||||
}
|
||||
|
||||
type Scored struct {
|
||||
Recipe Recipe
|
||||
Score float64 // match strength, 0.0–1.0
|
||||
}
|
||||
|
||||
// Resolve renders templates + secret refs into a spec the Installer consumes.
|
||||
type MCPInstanceSpec struct {
|
||||
Package PackageRef // resolved via Registry
|
||||
Config map[string]any // templates rendered; secrets as refs
|
||||
SecretRefs []SecretRef // injected at runtime, never inlined
|
||||
BoundResource string // DiscoveredResource.uuid
|
||||
}
|
||||
```
|
||||
|
||||
## Matching, scoring & precedence
|
||||
- **Matching:** every predicate in `match` must hold (AND semantics). Predicates: `fingerprint` (type), `ports`, `http_title` (regex/substring), `capabilities`, `version` (semver constraint), and arbitrary `evidence.*` equality.
|
||||
- **Scoring:** a matched recipe's score combines predicate **specificity** (more/stricter predicates → higher) with the resource's own discovery **confidence**. This favors precise recipes over broad catch-alls.
|
||||
- **Precedence** when several recipes match one resource, in order:
|
||||
1. Highest explicit `priority`.
|
||||
2. Then highest match score (specificity × confidence).
|
||||
3. Then most specific version constraint.
|
||||
4. Then recipe `name` (stable, deterministic) as final tie-break.
|
||||
- The reconciler acts on the winner only if `confidence ≥ threshold` (tenet #6); below threshold it queues for human approval rather than auto-installing.
|
||||
|
||||
## Example recipe — Home Assistant
|
||||
```yaml
|
||||
name: home-assistant
|
||||
version: 1.2.0
|
||||
priority: 50
|
||||
match:
|
||||
fingerprint: home-assistant # DiscoveredResource.type
|
||||
ports: [8123]
|
||||
http_title: "Home Assistant" # substring/regex over evidence.http_title
|
||||
capabilities: [rest-api]
|
||||
install:
|
||||
package: mcp-home-assistant # resolved via Registry (name + constraint)
|
||||
version: ">=0.4 <1.0"
|
||||
config:
|
||||
base_url: "http://{{ip}}:{{port}}" # {{port}} -> 8123 from the resource
|
||||
token: "{{secret:home-assistant/llat}}" # secret ref, injected at runtime
|
||||
verify_tls: false
|
||||
```
|
||||
Given a discovered Home Assistant at `192.168.1.20:8123`, this resolves to an `MCPInstanceSpec` for package `mcp-home-assistant`, config `base_url=http://192.168.1.20:8123`, and a `SecretRef` to `home-assistant/llat` — the token value is never written into the spec or shown to agents.
|
||||
|
||||
## Data
|
||||
- **Reads** `Recipe`s from the **Registry**/**Config** stores (§8; recipes are syncable from remote indexes like packages).
|
||||
- **Reads** `DiscoveredResource`s from **Inventory** and `Package` metadata (incl. `config_schema`) from **Registry**.
|
||||
- **Produces** an `MCPInstanceSpec` consumed by the reconciler/Installer to create an `MCPInstance` (§7). It does not itself persist instances.
|
||||
- References `Secret`s by ref only.
|
||||
|
||||
## Dependencies
|
||||
- Input from [Discovery Engine](01-discovery-engine.md) (`DiscoveredResource` + confidence).
|
||||
- Package resolution via [MCP Package Registry](02-package-registry.md).
|
||||
- Secret refs resolved at runtime by [Secrets](07-secrets-manager.md).
|
||||
- Output consumed by the [Auto Installer](03-auto-installer.md).
|
||||
- User-authored/community recipes distributed under the [Plugin System](11-plugin-system.md) / community registry governance (Phase 5).
|
||||
|
||||
## Failure modes & handling
|
||||
| Failure | Behavior |
|
||||
|---|---|
|
||||
| No recipe matches a resource | Resource stays visible in Inventory; no install; optionally surfaced as "no recipe" for a human to author one. |
|
||||
| Multiple recipes match | Deterministic precedence (priority → score → version → name) picks one; the discarded matches are recorded for transparency. |
|
||||
| Referenced package not found in Registry | Resolution error; spec not produced; flagged for operator; reconciler retries after next Registry sync. |
|
||||
| Config template references a missing variable/secret | Validation fails **before** install; recipe rejected with the exact missing key; no partial install. |
|
||||
| Recipe config violates package `config_schema` | Rejected at `Validate`; never handed to the Installer. |
|
||||
| Confidence below threshold | Match computed but held for human-in-the-loop approval (tenet #6). |
|
||||
|
||||
## Security notes
|
||||
Honors Architecture §9: secrets appear in recipes **only as refs** (`{{secret:...}}`) and are injected into the MCP container at runtime by [Secrets](07-secrets-manager.md) — never inlined into config, never persisted in the rendered spec, never returned to agents or logged. Community/user recipes are untrusted input: they are schema-validated and their referenced packages are signature-verified by the Registry before any install (supply-chain rule). Applying a recipe is an audited, mutating action. Recipes cannot grant a container more privilege than the package/runtime policy allows.
|
||||
|
||||
## Open questions
|
||||
- Do we allow OR/`anyOf` predicate groups, or keep strict AND for predictability?
|
||||
- Should scoring weights (specificity vs confidence) be tunable per deployment?
|
||||
- Community recipe trust model: signing, review, namespacing (ties to Architecture §12).
|
||||
- Multi-instance: when one host exposes several matching services, how do recipes express "one instance per resource" vs "one shared instance"?
|
||||
- Template language scope — keep it to safe variable substitution, or allow limited expressions?
|
||||
|
||||
## Milestone
|
||||
Delivered in **Phase 2** (Discover → install), as **Smart Recipes v1: match→install→config rules + a starter recipe pack**. Thin slice that lands first: strict-AND matching with the precedence rules above and safe `{{variable}}`/`{{secret:...}}` substitution, proven by the Postgres recipe that turns a discovered Postgres into a live `postgres.query` at the Gateway within one reconcile cycle. OR-predicates, tunable scoring, and community distribution follow later.
|
||||
Reference in New Issue
Block a user