mcp-gateway-nexus/docs/modules/05-dynamic-tool-registry.md

Dynamic Tool Registry

Module 05 · Plane: Data · Roadmap phase: 1 Part of MCP Nexus architecture.

Purpose

The Dynamic Tool Registry is the live, continuously-updated catalog of every tool exposed by every healthy MCPInstance, each namespaced as {namespace}.{tool} (Architecture §7). It is the authoritative answer to two questions on the data-plane hot path:

1. "What tools exist?" — it serves tools/list to agents, filtered to the caller's RBAC-visible set.
2. "Who owns this tool?" — it maintains the tool → owning instance mapping the Router uses to dispatch tools/call.

It is live because upstream MCP servers appear, disappear, and change their tool sets at runtime. The Registry reacts to instances coming and going (from the Installer / Health) and to upstream tools/list results and notifications/tools/list_changed notifications, keeping the aggregated catalog current without a restart.

Responsibilities

• Aggregate the tool lists of all healthy instances into one namespaced catalog; drop tools of unhealthy/absent instances immediately.
• Assign and enforce the {namespace}.{tool} naming and collision rules (see Data).
• Maintain the {namespace}.{tool} → instanceID dispatch map consumed by the Router.
• Serve tools/list results and the GET /tools view, always filtered by the caller's RBAC-resolved visibility.
• Subscribe to lifecycle events (instance added/removed, health transitions) and to upstream list_changed notifications; refresh incrementally.
• Cache upstream tool schemas (input/output JSON Schema) so tools/list does not fan out to every upstream on each agent call.
• Emit change events so the Gateway can forward list_changed to connected agents whose visible set changed.

Non-goals

• Transport / sessions / the agent edge — owned by the Gateway.
• Dispatching tools/call and streaming results — owned by the Router; the Registry only supplies the ownership mapping.
• Deciding what an agent may see — the Registry applies an RBAC filter it is given; policy lives in RBAC.
• Installing or health-checking instances — control-plane modules feed the Registry; it does not manage instance lifecycle.

Interfaces

// Registry is the live namespaced tool catalog. Concurrency-safe; reads are
// hot-path, writes are event-driven.
type Registry interface {
	// List returns the tools visible to a principal, already RBAC-filtered.
	List(ctx context.Context, view rbac.ToolView) ([]Tool, error)
	// Resolve maps a namespaced tool name to its owning instance for dispatch.
	Resolve(name string) (instanceID string, t Tool, ok bool)
	// Subscribe streams catalog change events (tool added/removed/schema-changed).
	Subscribe(ctx context.Context) (<-chan ChangeEvent, func())
}

// Ingest is the write side, driven by lifecycle + upstream events.
type Ingest interface {
	// UpsertInstance (re)loads an instance's tools under its namespace.
	UpsertInstance(ctx context.Context, inst MCPInstance, tools []UpstreamTool) error
	// RemoveInstance drops all tools owned by an instance.
	RemoveInstance(ctx context.Context, instanceID string) error
	// OnListChanged handles an upstream notifications/tools/list_changed.
	OnListChanged(ctx context.Context, instanceID string) error
}

type Tool struct {
	Name        string // "{namespace}.{tool}", e.g. "postgres.query"
	Namespace   string
	Bare        string // upstream tool name, e.g. "query"
	InstanceID  string
	InputSchema json.RawMessage
	Title, Description string
}

HTTP / MCP surface:

Method / path	Purpose
MCP tools/list	Served via the Gateway; RBAC-filtered namespaced catalog.
MCP notifications/tools/list_changed	Forwarded to agents when their visible set changes.
GET /tools	Dashboard/API view of the full catalog with owning instance, namespace, health, and schema (control-plane, RBAC-guarded).

Data

• Catalog (in-memory, source of truth at runtime): map[name]Tool, plus a namespace → instanceID index and reverse instanceID → []name.
• Namespacing rule: the namespace is the instance's stable, human-readable name (from its Recipe/config, e.g. homeassistant, postgres, github). The fully-qualified tool name is {namespace}.{tool}.
• Collision rules:
  • Two instances may never share a namespace — enforced at instance registration; a duplicate namespace is a config/reconcile error surfaced to the operator, not silently merged.
  • Within a namespace, upstream tool names are already unique per MCP server, so {namespace}.{tool} is globally unique by construction.
  • Bare tool names never reach the agent; only fully-qualified names are exposed, so unrelated servers with the same tool name (query) never clash.
• Persistence: the catalog is derived and rebuilt from Inventory + upstream tools/list on boot; it is a cache, not a store. Namespace assignments live with the MCPInstance in Inventory.

Dependencies

• Gateway — consumes List for tools/list, forwards list_changed.
• RBAC — supplies the ToolView filter applied to every list.
• Health Monitoring — health transitions add/remove instances from the catalog.
• Auto Installer — instance create/destroy events.
• AI Agent Profiles — identity → roles → tool view.
• Service Graph — consumes the tool→instance mapping as graph edges.

Failure modes & handling

• Upstream tools/list fails on ingest: keep the last-known good tool set for that instance, mark it stale, and retry with backoff; do not blank the namespace on a transient error.
• list_changed storm: debounce/coalesce refreshes per instance so a flapping upstream cannot thrash the hot path.
• Instance vanishes: RemoveInstance drops its tools atomically; in-flight Resolve for a removed tool returns ok=false, and the Router returns a clean JSON-RPC "unknown tool" error.
• Namespace collision at registration: reject the newer instance's tools, emit an operator notification, and keep the existing namespace intact.
• Cold start: serve an empty catalog that fills as instances report; never block the Gateway waiting for a full rebuild.

Security notes

• Every tools/list and GET /tools response is RBAC-filtered — an agent cannot even see a tool outside its role (Architecture §9, request scoping).
• Cached schemas contain no secret material; secrets live only in the running MCP container (see Secrets Manager).
• Namespaces are validated (charset/length) to prevent spoofing a well-known namespace like github.

Open questions

• Namespace assignment when two instances legitimately wrap the same service type (two Postgres servers): auto-suffix (postgres, postgres-2) vs. operator-chosen aliases?
• How aggressively to pre-fetch/refresh schemas vs. lazy-load on first tools/call.
• Whether to surface tool deprecation/version metadata in the catalog for the Update Manager and Dashboard.

Milestone

Phase 1 (Walking skeleton), alongside the Gateway. Exit: with a filesystem MCP and one HTTP MCP registered in YAML, an agent's tools/list shows filesystem.* and other.*, and Resolve maps a chosen tool to its owning instance for a successful tools/call.