mcp-gateway-nexus/docs/modules/11-plugin-system.md

Plugin System

Module 11 · Plane: Cross-cutting · Roadmap phase: 5 Part of MCP Nexus architecture.

Purpose

The extensibility backbone that makes tenet #3 real — "everything is a plugin." The Nexus core knows only interfaces; concrete discovery methods, fingerprinters, installers, package providers, notifiers, auth backends, dashboards, and tool transformers are swappable implementations. This module does not invent those interfaces — each owning module already defines its own (discovery.Method, notify.Notifier, health.Probe, …). Module 11 generalizes them into one plugin contract, lifecycle, discovery/registration mechanism, and a recommended out-of-process transport so third parties can extend Nexus, in any language, without forking or recompiling the core.

Responsibilities

• Define the common plugin contract: a lifecycle (register → configure → start → health → stop) and metadata (name, type, version, capabilities) every plugin implements regardless of its type.
• Enumerate and load the supported plugin types, each mapping to an interface already owned by a core module: discovery methods, fingerprinters, installers, package providers, notifiers, auth backends, dashboards, and tool transformers.
• Provide the out-of-process transport (gRPC over a handshake-negotiated local socket, in the style of hashicorp/go-plugin) so plugins run as isolated subprocesses — a crashing or malicious plugin cannot take down the core.
• Support in-process (compiled-in) plugins too, for first-party built-ins and performance-sensitive paths — the transport is an implementation detail behind the same contract.
• Own versioning & compatibility: a semver'd plugin API, capability negotiation on the handshake, and refusal to load incompatible plugins.
• Manage plugin discovery, install, config, and health: a plugin registry, per-plugin config (secrets by ref), and health surfacing to the Dashboard.

Non-goals

• Does not define the per-type interfaces' semantics. What a Method or Notifier means is owned by Discovery, Notifications, etc. Module 11 provides the wrapper, transport, and lifecycle.
• Not the MCP tool plugins agents call. Those are upstream MCPInstances managed by the control loop. This is about extending Nexus itself.
• Not a general sandbox runtime — container sandboxing of MCP servers is Security + Runtime; plugin isolation here is process-level (separate proc, dropped privileges, scoped host interface).
• Not a package registry. Distributing/signing plugin artifacts leans on Package Registry + Security supply-chain rules.

Interfaces

// Plugin is the universal contract every plugin implements, regardless of type.
type Plugin interface {
    // Info returns identity + declared capabilities used for compatibility checks.
    Info() PluginInfo
    // Configure applies validated config; secrets arrive as refs, resolved by host.
    Configure(ctx context.Context, cfg map[string]any) error
    // Health reports readiness so the dashboard/plugin manager can surface it.
    Health(ctx context.Context) error
    // Stop releases resources; the host may kill the subprocess after a deadline.
    Stop(ctx context.Context) error
}

type PluginInfo struct {
    Name    string
    Type    PluginType // Discovery, Fingerprinter, Installer, PackageProvider,
                       // Notifier, AuthBackend, Dashboard, ToolTransformer
    Version string     // plugin's own version
    APIVer  string     // Nexus plugin-API semver it was built against
    Caps    []string
}

// Host is the surface the core exposes back to a plugin (least-privilege).
type Host interface {
    Recorder() metrics.Recorder          // scoped metrics
    Secret(ref string) (string, error)   // resolve a Secret by ref only
    Logger() Logger                       // structured, plugin-namespaced
    EventBus() EventEmitter               // emit typed events (RBAC-scoped)
}

// Manager loads, negotiates, and supervises plugins (in- or out-of-process).
type Manager interface {
    Load(ctx context.Context, spec PluginSpec) (handle PluginHandle, err error)
    // TypedDispatch adapts a loaded plugin to the owning module's interface,
    // e.g. as a discovery.Method or notify.Notifier, over the transport.
    Bind(handle PluginHandle) (any, error)
    List() []PluginStatus
    Unload(ctx context.Context, name string) error
}

Transport: a go-plugin-style handshake starts the subprocess, negotiates the API version + a shared secret, and multiplexes typed gRPC services (one service definition per plugin type) over a local Unix socket. Each core module ships the gRPC ⇆ Go-interface adapter for its type.

Lifecycle: Load (spawn + handshake + version check) → Configure (validated config, secrets by ref) → Bind (adapt to the owning module's typed interface) → serving (health-polled) → Unload/Stop (graceful, then kill after a deadline). A plugin that fails Health is restarted with backoff; the owning module keeps functioning on its built-in implementations while a plugin is down, so an extension is never a single point of failure for a core capability.

Internal HTTP (control API, RBAC-guarded):

• GET /api/v1/plugins — installed plugins, type, version, health, config status.
• POST /api/v1/plugins — install/register a plugin (artifact ref + config).
• POST /api/v1/plugins/{name}/reload · DELETE /api/v1/plugins/{name}.

Data

• Writes the plugin registry (installed plugins, type, version, artifact digest, enabled state) to the Config/Registry store (§8).
• Reads plugin config from Config and plugin credentials by ref from Secrets — never handed the raw vault.
• Plugin subprocess state (PID, socket, handshake) is module-local runtime state, rebuilt on restart from the registry.

Dependencies

• Generalizes interfaces owned by: Discovery (Method, Fingerprinter), Auto Installer (installer/runtime adapters), Package Registry (package providers), Notifications (Notifier), Authentication (auth backends), Web Dashboard (panels), Health (Probe), and the Router (tool transformers).
• Secrets Manager — resolves plugin credential refs.
• Security — signature verification + isolation policy for plugin artifacts.
• Metrics — plugins emit through a scoped Recorder.

Failure modes & handling

Failure	Behavior
Plugin subprocess crashes	Isolated by the transport; the Manager marks it unhealthy, restarts with backoff, and the owning module falls back to built-ins — the core stays up.
Version incompatibility	Handshake rejects a plugin built against an incompatible API semver; it is not loaded; surfaced in dashboard.
Plugin hangs / slow RPC	Every host↔plugin call is deadline-bounded; a hung plugin is killed after a grace period, not awaited.
Malicious/greedy plugin	Runs out-of-process with dropped privileges and only the least-privilege Host surface (no direct DB, secrets only by ref); resource limits applied.
Bad plugin config	Configure validates and rejects atomically; the plugin stays in its prior good state or disabled.
Unsigned/tampered artifact	Security supply-chain check refuses to install (signature + digest pin).

Security notes

Honors Architecture §9 and tenet #5. Out-of-process isolation is the core safety property: plugins get no ambient authority — only the narrow Host interface (scoped metrics, secrets by ref, namespaced logging, RBAC-scoped event emit), never the database, the raw vault, or the host network unless policy grants it. Plugin artifacts are signature-verified and digest-pinned before load (supply chain). Installing/reloading/removing a plugin is a mutating, RBAC-gated, audited action. Plugin logs and emitted events are sanitized so a plugin cannot exfiltrate secrets or forge another principal's identity.

Open questions

• Transport: standardize on hashicorp/go-plugin directly, or a thin in-house gRPC handshake to avoid the dependency surface?
• Do we support WASM plugins (sandboxed, portable) as a third mode alongside in-process and subprocess?
• How rich should the Host surface be before it becomes an attack surface — where is the line between capability and least privilege?
• Compatibility policy: strict semver gate, or capability-negotiation that allows partial feature sets?
• Plugin distribution/governance — reuse the community registry + recipe trust model (Architecture §12)?

Milestone

Delivered in Phase 5 (Extend & scale). The per-type interfaces exist earlier (each module defines its own from its phase); Phase 5 promotes them to a stable, semver'd plugin API with the out-of-process gRPC transport, plugin manager, and registry. Exit proof (shared with the phase goal): a community-authored discovery plugin + recipe installs a new MCP with no core changes, running as an isolated subprocess.