mcp-gateway-nexus/docs/modules/05-dynamic-tool-registry.md

# Dynamic Tool Registry

> Module 05 · Plane: Data · Roadmap phase: 1
> Part of [MCP Nexus architecture](../ARCHITECTURE.md).

## Purpose

The Dynamic Tool Registry is the **live, continuously-updated catalog of every
tool exposed by every healthy [`MCPInstance`](../ARCHITECTURE.md)**, each
namespaced as `{namespace}.{tool}` (Architecture §7). It is the authoritative
answer to two questions on the data-plane hot path:

1. *"What tools exist?"* — it serves `tools/list` to agents, filtered to the
   caller's RBAC-visible set.
2. *"Who owns this tool?"* — it maintains the `tool → owning instance` mapping
   the Router uses to dispatch `tools/call`.

It is **live** because upstream MCP servers appear, disappear, and change their
tool sets at runtime. The Registry reacts to instances coming and going (from
the [Installer](03-auto-installer.md) / [Health](09-health-monitoring.md)) and
to upstream `tools/list` results and `notifications/tools/list_changed`
notifications, keeping the aggregated catalog current without a restart.

## Responsibilities

- Aggregate the tool lists of all **healthy** instances into one namespaced
  catalog; drop tools of unhealthy/absent instances immediately.
- Assign and enforce the `{namespace}.{tool}` naming and **collision rules**
  (see [Data](#data)).
- Maintain the `{namespace}.{tool} → instanceID` dispatch map consumed by the
  Router.
- Serve `tools/list` results and the `GET /tools` view, always filtered by the
  caller's RBAC-resolved visibility.
- Subscribe to lifecycle events (instance added/removed, health transitions)
  and to upstream `list_changed` notifications; refresh incrementally.
- Cache upstream tool schemas (input/output JSON Schema) so `tools/list` does
  not fan out to every upstream on each agent call.
- Emit change events so the Gateway can forward `list_changed` to connected
  agents whose visible set changed.

## Non-goals

- **Transport / sessions / the agent edge** — owned by the
  [Gateway](04-gateway.md).
- **Dispatching `tools/call` and streaming results** — owned by the Router; the
  Registry only supplies the ownership mapping.
- **Deciding what an agent may see** — the Registry *applies* an RBAC filter it
  is given; policy lives in [RBAC](08-rbac.md).
- **Installing or health-checking instances** — control-plane modules feed the
  Registry; it does not manage instance lifecycle.

## Interfaces

```go
// Registry is the live namespaced tool catalog. Concurrency-safe; reads are
// hot-path, writes are event-driven.
type Registry interface {
	// List returns the tools visible to a principal, already RBAC-filtered.
	List(ctx context.Context, view rbac.ToolView) ([]Tool, error)
	// Resolve maps a namespaced tool name to its owning instance for dispatch.
	Resolve(name string) (instanceID string, t Tool, ok bool)
	// Subscribe streams catalog change events (tool added/removed/schema-changed).
	Subscribe(ctx context.Context) (<-chan ChangeEvent, func())
}

// Ingest is the write side, driven by lifecycle + upstream events.
type Ingest interface {
	// UpsertInstance (re)loads an instance's tools under its namespace.
	UpsertInstance(ctx context.Context, inst MCPInstance, tools []UpstreamTool) error
	// RemoveInstance drops all tools owned by an instance.
	RemoveInstance(ctx context.Context, instanceID string) error
	// OnListChanged handles an upstream notifications/tools/list_changed.
	OnListChanged(ctx context.Context, instanceID string) error
}

type Tool struct {
	Name        string // "{namespace}.{tool}", e.g. "postgres.query"
	Namespace   string
	Bare        string // upstream tool name, e.g. "query"
	InstanceID  string
	InputSchema json.RawMessage
	Title, Description string
}
```

HTTP / MCP surface:

| Method / path | Purpose |
|---|---|
| MCP `tools/list` | Served via the Gateway; RBAC-filtered namespaced catalog. |
| MCP `notifications/tools/list_changed` | Forwarded to agents when their visible set changes. |
| `GET /tools` | Dashboard/API view of the full catalog with owning instance, namespace, health, and schema (control-plane, RBAC-guarded). |

## Data

- **Catalog (in-memory, source of truth at runtime):** `map[name]Tool`, plus a
  `namespace → instanceID` index and reverse `instanceID → []name`.
- **Namespacing rule:** the namespace is the instance's stable, human-readable
  name (from its Recipe/config, e.g. `homeassistant`, `postgres`, `github`).
  The fully-qualified tool name is `{namespace}.{tool}`.
- **Collision rules:**
  - Two instances may never share a namespace — enforced at instance
    registration; a duplicate namespace is a config/reconcile error surfaced to
    the operator, not silently merged.
  - Within a namespace, upstream tool names are already unique per MCP server,
    so `{namespace}.{tool}` is globally unique by construction.
  - Bare tool names never reach the agent; only fully-qualified names are
    exposed, so unrelated servers with the same tool name (`query`) never clash.
- **Persistence:** the catalog is *derived* and rebuilt from Inventory +
  upstream `tools/list` on boot; it is a cache, not a store. Namespace
  assignments live with the `MCPInstance` in Inventory.

## Dependencies

- [Gateway](04-gateway.md) — consumes `List` for `tools/list`, forwards
  `list_changed`.
- [RBAC](08-rbac.md) — supplies the `ToolView` filter applied to every list.
- [Health Monitoring](09-health-monitoring.md) — health transitions add/remove
  instances from the catalog.
- [Auto Installer](03-auto-installer.md) — instance create/destroy events.
- [AI Agent Profiles](14-agent-profiles.md) — identity → roles → tool view.
- [Service Graph](17-service-graph.md) — consumes the tool→instance mapping as
  graph edges.

## Failure modes & handling

- **Upstream `tools/list` fails on ingest:** keep the last-known good tool set
  for that instance, mark it stale, and retry with backoff; do not blank the
  namespace on a transient error.
- **`list_changed` storm:** debounce/coalesce refreshes per instance so a
  flapping upstream cannot thrash the hot path.
- **Instance vanishes:** `RemoveInstance` drops its tools atomically; in-flight
  `Resolve` for a removed tool returns `ok=false`, and the Router returns a
  clean JSON-RPC "unknown tool" error.
- **Namespace collision at registration:** reject the newer instance's tools,
  emit an operator notification, and keep the existing namespace intact.
- **Cold start:** serve an empty catalog that fills as instances report; never
  block the Gateway waiting for a full rebuild.

## Security notes

- Every `tools/list` and `GET /tools` response is RBAC-filtered — an agent
  cannot even *see* a tool outside its role (Architecture §9, request scoping).
- Cached schemas contain no secret material; secrets live only in the running
  MCP container (see [Secrets Manager](07-secrets-manager.md)).
- Namespaces are validated (charset/length) to prevent spoofing a well-known
  namespace like `github`.

## Open questions

- Namespace assignment when two instances legitimately wrap the *same* service
  type (two Postgres servers): auto-suffix (`postgres`, `postgres-2`) vs.
  operator-chosen aliases?
- How aggressively to pre-fetch/refresh schemas vs. lazy-load on first
  `tools/call`.
- Whether to surface tool *deprecation*/version metadata in the catalog for the
  Update Manager and Dashboard.

## Milestone

Phase 1 (Walking skeleton), alongside the [Gateway](04-gateway.md). **Exit:**
with a filesystem MCP and one HTTP MCP registered in YAML, an agent's
`tools/list` shows `filesystem.*` and `other.*`, and `Resolve` maps a chosen
tool to its owning instance for a successful `tools/call`.