Files
mcp-gateway-nexus/docs/modules/15-smart-recipes.md
drjones 8d3ffef920 docs: initial architecture and design for MCP Nexus
MCP Nexus is a self-hosted control plane for MCP servers — "Kubernetes for
MCP." Agents connect to one endpoint; Nexus discovers services, installs the
right MCP servers, aggregates them behind a namespaced router, secures access
with auth/RBAC, and heals/updates them via a reconcile loop.

This first commit is design-phase only (no runnable code yet):
- README.md            project front door + module map
- docs/ARCHITECTURE.md target design: tenets, two-plane split, reconcile
                       loop, domain types, storage, security, deployment
- docs/ROADMAP.md      phased delivery (foundations -> walking skeleton ->
                       discover+install -> secure -> operate -> extend/scale)
- docs/modules/01-20   one design doc per module, all cross-linked

Backend stack decision: Go (single static binary, embedded SQLite + embedded
React dashboard). Repo initialized in /root with a whitelist .gitignore so
only project files are tracked.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 04:38:27 +00:00

124 lines
8.4 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Smart Recipes
> Module 15 · Plane: Control · Roadmap phase: 2
> Part of [MCP Nexus architecture](../ARCHITECTURE.md).
## Purpose
Provide declarative **YAML rules that map a fingerprint to an MCP install** — no code required to support a new service. A recipe binds `match` → `install` → `config`, and is the **desired-state input** to the reconciler: given a `DiscoveredResource` (from [Discovery Engine](01-discovery-engine.md)) and a `Package` (from the [Registry](02-package-registry.md)), a matching recipe resolves into a concrete `MCPInstance` spec for the [Auto Installer](03-auto-installer.md).
## Responsibilities
- Define the **Recipe schema** (Architecture §7 `Recipe`): `match`, `install`, `config`, plus metadata (`name`, `version`, `priority`).
- **Match** a `DiscoveredResource` against recipe predicates (fingerprint `type`, open `ports`, `http_title`, capabilities, version constraints, evidence fields) and produce a **match score**.
- **Resolve precedence** when multiple recipes match a single resource, deterministically.
- **Render config templates** — substitute variables (`{{ip}}`, `{{port}}`, `{{hostname}}`, `{{version}}`) and **secret references** into the config a `Package`'s `config_schema` expects.
- Reference **Registry packages** (by name + optional version constraint) and **Secrets** (by ref, never inline).
- Ship a **starter recipe pack** and load user/community recipes from the Config store.
- Validate recipes against package `config_schema` and report actionable errors.
## Non-goals
- **Does not discover or fingerprint** — it consumes `DiscoveredResource`s from [Discovery Engine](01-discovery-engine.md).
- **Does not resolve or verify packages** — that is the [Registry](02-package-registry.md) (signatures, digests, versions).
- **Does not install or run** the resulting spec — the [Auto Installer](03-auto-installer.md) + Runtime do.
- **Does not store or decrypt secrets** — it emits secret *refs*; [Secrets](07-secrets-manager.md) injects values at container runtime.
- **Does not own the reconcile loop** — it is a pure(ish) resolver the reconciler calls.
## Interfaces
```go
// A Recipe is a declarative match+install+config rule (Architecture §7).
type Recipe struct {
Name string
Version string
Priority int // tie-breaker; higher wins
Match MatchSpec // predicates over a DiscoveredResource
Install InstallSpec // package ref (docker_image / package name)
Config map[string]any // templated values -> Package.config_schema
}
// The engine matches, scores, and resolves recipes into instance specs.
type Engine interface {
Load(ctx context.Context) error // from Config store
Match(r DiscoveredResource) []Scored // sorted, best first
Resolve(r DiscoveredResource, pick Recipe) (MCPInstanceSpec, error)
Validate(rec Recipe, pkg Package) error // against config_schema
}
type Scored struct {
Recipe Recipe
Score float64 // match strength, 0.0–1.0
}
// Resolve renders templates + secret refs into a spec the Installer consumes.
type MCPInstanceSpec struct {
Package PackageRef // resolved via Registry
Config map[string]any // templates rendered; secrets as refs
SecretRefs []SecretRef // injected at runtime, never inlined
BoundResource string // DiscoveredResource.uuid
}
```
## Matching, scoring & precedence
- **Matching:** every predicate in `match` must hold (AND semantics). Predicates: `fingerprint` (type), `ports`, `http_title` (regex/substring), `capabilities`, `version` (semver constraint), and arbitrary `evidence.*` equality.
- **Scoring:** a matched recipe's score combines predicate **specificity** (more/stricter predicates → higher) with the resource's own discovery **confidence**. This favors precise recipes over broad catch-alls.
- **Precedence** when several recipes match one resource, in order:
1. Highest explicit `priority`.
2. Then highest match score (specificity × confidence).
3. Then most specific version constraint.
4. Then recipe `name` (stable, deterministic) as final tie-break.
- The reconciler acts on the winner only if `confidence ≥ threshold` (tenet #6); below threshold it queues for human approval rather than auto-installing.
## Example recipe — Home Assistant
```yaml
name: home-assistant
version: 1.2.0
priority: 50
match:
fingerprint: home-assistant # DiscoveredResource.type
ports: [8123]
http_title: "Home Assistant" # substring/regex over evidence.http_title
capabilities: [rest-api]
install:
package: mcp-home-assistant # resolved via Registry (name + constraint)
version: ">=0.4 <1.0"
config:
base_url: "http://{{ip}}:{{port}}" # {{port}} -> 8123 from the resource
token: "{{secret:home-assistant/llat}}" # secret ref, injected at runtime
verify_tls: false
```
Given a discovered Home Assistant at `192.168.1.20:8123`, this resolves to an `MCPInstanceSpec` for package `mcp-home-assistant`, config `base_url=http://192.168.1.20:8123`, and a `SecretRef` to `home-assistant/llat` — the token value is never written into the spec or shown to agents.
## Data
- **Reads** `Recipe`s from the **Registry**/**Config** stores (§8; recipes are syncable from remote indexes like packages).
- **Reads** `DiscoveredResource`s from **Inventory** and `Package` metadata (incl. `config_schema`) from **Registry**.
- **Produces** an `MCPInstanceSpec` consumed by the reconciler/Installer to create an `MCPInstance` (§7). It does not itself persist instances.
- References `Secret`s by ref only.
## Dependencies
- Input from [Discovery Engine](01-discovery-engine.md) (`DiscoveredResource` + confidence).
- Package resolution via [MCP Package Registry](02-package-registry.md).
- Secret refs resolved at runtime by [Secrets](07-secrets-manager.md).
- Output consumed by the [Auto Installer](03-auto-installer.md).
- User-authored/community recipes distributed under the [Plugin System](11-plugin-system.md) / community registry governance (Phase 5).
## Failure modes & handling
| Failure | Behavior |
|---|---|
| No recipe matches a resource | Resource stays visible in Inventory; no install; optionally surfaced as "no recipe" for a human to author one. |
| Multiple recipes match | Deterministic precedence (priority → score → version → name) picks one; the discarded matches are recorded for transparency. |
| Referenced package not found in Registry | Resolution error; spec not produced; flagged for operator; reconciler retries after next Registry sync. |
| Config template references a missing variable/secret | Validation fails **before** install; recipe rejected with the exact missing key; no partial install. |
| Recipe config violates package `config_schema` | Rejected at `Validate`; never handed to the Installer. |
| Confidence below threshold | Match computed but held for human-in-the-loop approval (tenet #6). |
## Security notes
Honors Architecture §9: secrets appear in recipes **only as refs** (`{{secret:...}}`) and are injected into the MCP container at runtime by [Secrets](07-secrets-manager.md) — never inlined into config, never persisted in the rendered spec, never returned to agents or logged. Community/user recipes are untrusted input: they are schema-validated and their referenced packages are signature-verified by the Registry before any install (supply-chain rule). Applying a recipe is an audited, mutating action. Recipes cannot grant a container more privilege than the package/runtime policy allows.
## Open questions
- Do we allow OR/`anyOf` predicate groups, or keep strict AND for predictability?
- Should scoring weights (specificity vs confidence) be tunable per deployment?
- Community recipe trust model: signing, review, namespacing (ties to Architecture §12).
- Multi-instance: when one host exposes several matching services, how do recipes express "one instance per resource" vs "one shared instance"?
- Template language scope — keep it to safe variable substitution, or allow limited expressions?
## Milestone
Delivered in **Phase 2** (Discover → install), as **Smart Recipes v1: match→install→config rules + a starter recipe pack**. Thin slice that lands first: strict-AND matching with the precedence rules above and safe `{{variable}}`/`{{secret:...}}` substitution, proven by the Postgres recipe that turns a discovered Postgres into a live `postgres.query` at the Gateway within one reconcile cycle. OR-predicates, tunable scoring, and community distribution follow later.