Files
mcp-gateway-nexus/docs/modules/15-smart-recipes.md
drjones 8d3ffef920 docs: initial architecture and design for MCP Nexus
MCP Nexus is a self-hosted control plane for MCP servers — "Kubernetes for
MCP." Agents connect to one endpoint; Nexus discovers services, installs the
right MCP servers, aggregates them behind a namespaced router, secures access
with auth/RBAC, and heals/updates them via a reconcile loop.

This first commit is design-phase only (no runnable code yet):
- README.md            project front door + module map
- docs/ARCHITECTURE.md target design: tenets, two-plane split, reconcile
                       loop, domain types, storage, security, deployment
- docs/ROADMAP.md      phased delivery (foundations -> walking skeleton ->
                       discover+install -> secure -> operate -> extend/scale)
- docs/modules/01-20   one design doc per module, all cross-linked

Backend stack decision: Go (single static binary, embedded SQLite + embedded
React dashboard). Repo initialized in /root with a whitelist .gitignore so
only project files are tracked.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 04:38:27 +00:00

8.4 KiB
Raw Permalink Blame History

Smart Recipes

Module 15 · Plane: Control · Roadmap phase: 2 Part of MCP Nexus architecture.

Purpose

Provide declarative YAML rules that map a fingerprint to an MCP install — no code required to support a new service. A recipe binds match → install → config, and is the desired-state input to the reconciler: given a DiscoveredResource (from Discovery Engine) and a Package (from the Registry), a matching recipe resolves into a concrete MCPInstance spec for the Auto Installer.

Responsibilities

  • Define the Recipe schema (Architecture §7 Recipe): match, install, config, plus metadata (name, version, priority).
  • Match a DiscoveredResource against recipe predicates (fingerprint type, open ports, http_title, capabilities, version constraints, evidence fields) and produce a match score.
  • Resolve precedence when multiple recipes match a single resource, deterministically.
  • Render config templates — substitute variables ({{ip}}, {{port}}, {{hostname}}, {{version}}) and secret references into the config a Package's config_schema expects.
  • Reference Registry packages (by name + optional version constraint) and Secrets (by ref, never inline).
  • Ship a starter recipe pack and load user/community recipes from the Config store.
  • Validate recipes against package config_schema and report actionable errors.

Non-goals

  • Does not discover or fingerprint — it consumes DiscoveredResources from Discovery Engine.
  • Does not resolve or verify packages — that is the Registry (signatures, digests, versions).
  • Does not install or run the resulting spec — the Auto Installer + Runtime do.
  • Does not store or decrypt secrets — it emits secret refs; Secrets injects values at container runtime.
  • Does not own the reconcile loop — it is a pure(ish) resolver the reconciler calls.

Interfaces

// A Recipe is a declarative match+install+config rule (Architecture §7).
type Recipe struct {
    Name     string
    Version  string
    Priority int              // tie-breaker; higher wins
    Match    MatchSpec        // predicates over a DiscoveredResource
    Install  InstallSpec      // package ref (docker_image / package name)
    Config   map[string]any   // templated values -> Package.config_schema
}

// The engine matches, scores, and resolves recipes into instance specs.
type Engine interface {
    Load(ctx context.Context) error                       // from Config store
    Match(r DiscoveredResource) []Scored                  // sorted, best first
    Resolve(r DiscoveredResource, pick Recipe) (MCPInstanceSpec, error)
    Validate(rec Recipe, pkg Package) error               // against config_schema
}

type Scored struct {
    Recipe Recipe
    Score  float64   // match strength, 0.0–1.0
}

// Resolve renders templates + secret refs into a spec the Installer consumes.
type MCPInstanceSpec struct {
    Package       PackageRef        // resolved via Registry
    Config        map[string]any    // templates rendered; secrets as refs
    SecretRefs    []SecretRef       // injected at runtime, never inlined
    BoundResource string            // DiscoveredResource.uuid
}

Matching, scoring & precedence

  • Matching: every predicate in match must hold (AND semantics). Predicates: fingerprint (type), ports, http_title (regex/substring), capabilities, version (semver constraint), and arbitrary evidence.* equality.
  • Scoring: a matched recipe's score combines predicate specificity (more/stricter predicates → higher) with the resource's own discovery confidence. This favors precise recipes over broad catch-alls.
  • Precedence when several recipes match one resource, in order:
    1. Highest explicit priority.
    2. Then highest match score (specificity × confidence).
    3. Then most specific version constraint.
    4. Then recipe name (stable, deterministic) as final tie-break.
  • The reconciler acts on the winner only if confidence ≥ threshold (tenet #6); below threshold it queues for human approval rather than auto-installing.

Example recipe — Home Assistant

name: home-assistant
version: 1.2.0
priority: 50
match:
  fingerprint: home-assistant      # DiscoveredResource.type
  ports: [8123]
  http_title: "Home Assistant"     # substring/regex over evidence.http_title
  capabilities: [rest-api]
install:
  package: mcp-home-assistant      # resolved via Registry (name + constraint)
  version: ">=0.4 <1.0"
config:
  base_url: "http://{{ip}}:{{port}}"   # {{port}} -> 8123 from the resource
  token: "{{secret:home-assistant/llat}}"  # secret ref, injected at runtime
  verify_tls: false

Given a discovered Home Assistant at 192.168.1.20:8123, this resolves to an MCPInstanceSpec for package mcp-home-assistant, config base_url=http://192.168.1.20:8123, and a SecretRef to home-assistant/llat — the token value is never written into the spec or shown to agents.

Data

  • Reads Recipes from the Registry/Config stores (§8; recipes are syncable from remote indexes like packages).
  • Reads DiscoveredResources from Inventory and Package metadata (incl. config_schema) from Registry.
  • Produces an MCPInstanceSpec consumed by the reconciler/Installer to create an MCPInstance (§7). It does not itself persist instances.
  • References Secrets by ref only.

Dependencies

Failure modes & handling

Failure Behavior
No recipe matches a resource Resource stays visible in Inventory; no install; optionally surfaced as "no recipe" for a human to author one.
Multiple recipes match Deterministic precedence (priority → score → version → name) picks one; the discarded matches are recorded for transparency.
Referenced package not found in Registry Resolution error; spec not produced; flagged for operator; reconciler retries after next Registry sync.
Config template references a missing variable/secret Validation fails before install; recipe rejected with the exact missing key; no partial install.
Recipe config violates package config_schema Rejected at Validate; never handed to the Installer.
Confidence below threshold Match computed but held for human-in-the-loop approval (tenet #6).

Security notes

Honors Architecture §9: secrets appear in recipes only as refs ({{secret:...}}) and are injected into the MCP container at runtime by Secrets — never inlined into config, never persisted in the rendered spec, never returned to agents or logged. Community/user recipes are untrusted input: they are schema-validated and their referenced packages are signature-verified by the Registry before any install (supply-chain rule). Applying a recipe is an audited, mutating action. Recipes cannot grant a container more privilege than the package/runtime policy allows.

Open questions

  • Do we allow OR/anyOf predicate groups, or keep strict AND for predictability?
  • Should scoring weights (specificity vs confidence) be tunable per deployment?
  • Community recipe trust model: signing, review, namespacing (ties to Architecture §12).
  • Multi-instance: when one host exposes several matching services, how do recipes express "one instance per resource" vs "one shared instance"?
  • Template language scope — keep it to safe variable substitution, or allow limited expressions?

Milestone

Delivered in Phase 2 (Discover → install), as Smart Recipes v1: match→install→config rules + a starter recipe pack. Thin slice that lands first: strict-AND matching with the precedence rules above and safe {{variable}}/{{secret:...}} substitution, proven by the Postgres recipe that turns a discovered Postgres into a live postgres.query at the Gateway within one reconcile cycle. OR-predicates, tunable scoring, and community distribution follow later.