Hot-Reload MCP tools

[aaronburtle]

Why make this change?

Closes #3066

DAB previously constructed the MCP tool registry only once during startup. Runtime configuration hot-reload could therefore leave MCP discovery stale when custom tools were added, removed, renamed, enabled, disabled, or given updated descriptions or stored-procedure parameters.

This change keeps MCP discovery and execution aligned with the latest successfully refreshed configuration and metadata generation.

What is this change?

• Replaces the mutable startup-only MCP registry with atomically published immutable snapshots.
  • tools/list and tools/call observe either the complete previous generation or the complete new generation—never a partially rebuilt registry.
  • Mutable MCP SDK metadata is defensively cloned before publication and when returned to callers.
• Adds an ordered MCP registry refresh after metadata, query, mutation, and authorization dependencies have refreshed.
  • Configuration-generated custom tools are recreated for each generation.
  • Database-enriched stored-procedure schemas are refreshed with a logged configuration-schema fallback when database metadata is unavailable.
  • Invalid names, duplicate names, or construction failures reject the entire candidate and retain the previous valid snapshot.
  • A stale-generation guard prevents an older candidate from replacing a newer registry.
• Uses the same serialized initialization path for HTTP and stdio.
  • Initial metadata initialization and registry publication cannot overlap a file-triggered reload.
  • Independently DI-registered IMcpTool implementations remain available across generations.
• Adds stdio notifications/tools/list_changed support.
  • Notifications are sent only after the initialization handshake and only for semantic discovery changes.
  • Notifications are coalesced and written asynchronously through the shared stdout writer so blocked transport I/O cannot block hot-reload.
  • HTTP handlers immediately read the latest snapshot, but HTTP push notification support remains deferred because it requires experimental MCP SDK session tracking.
• Adds coordinated hot-reload shutdown.
  • Shutdown stops new reload admission, cancels queued and active DAB-owned work, and drains cooperative work before dependency disposal.
  • The drain respects HostOptions.ShutdownTimeout; synchronous disposal remains nonblocking.
  • Loader-owned synchronization resources are released after all admitted operations and cancellation callbacks have exited.
• Propagates cancellation through metadata discovery, query retries, connection opening, commands, and access-token acquisition.
  • Explicit query cancellation is linked with HttpContext.RequestAborted, so either source can stop database work.
  • Existing public Core interface implementations remain compatible through default interface methods and retained legacy overloads.
  • Two undocumented protected metadata-provider implementation hooks intentionally move to token-bearing signatures so their database I/O cannot escape bounded shutdown. This limited compatibility exception and its migration path are documented.
• Startup-bound MCP settings such as endpoint enablement and path changes still require a restart.
• Detailed architecture, lifecycle guarantees, compatibility decisions, rejected alternatives, and limitations:
  • MCP Tool Registry Hot-Reload Design

How was this tested?

• Integration Tests
  • HTTP configuration-file reload updates tool membership, descriptions, visibility, and stored-procedure schemas.
  • Failed or duplicate candidates retain the previous snapshot, and a corrected configuration recovers on the next reload.
  • Stdio configuration-file reload publishes the latest snapshot and emits one valid notifications/tools/list_changed notification.
  • Initial metadata/registry construction cannot overlap a concurrent reload generation.
  • Hosted shutdown drains cooperative reload work before hosted services and singleton dependencies are disposed.
• Unit Tests
  • Atomic publication, deterministic ordering, case-insensitive lookup, collision handling, defensive metadata cloning, and concurrent readers.
  • Semantic metadata comparison while preserving client-visible schema property order.
  • Stdio initialization ordering, notification coalescing, blocked stdout behavior, scheduling fallback, and nonblocking disposal.
  • Active and queued reload cancellation, bounded shutdown, watcher cleanup, and synchronization-resource disposal.
  • Cancellation propagation through metadata providers and query execution.
  • Explicit query cancellation and HttpContext.RequestAborted both cancel the same operation.
  • Existing interface implementations, legacy overloads, and virtual dispatch remain compatible.
  • Full solution build completed successfully.

Sample Request(s)

No REST or GraphQL wire shape changes are introduced.

Start DAB in MCP stdio mode using a development-mode configuration:

dab start --config dab-config.json --mcp-stdio

Initialize the MCP session and request the initial tool list:

{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"sample-client","version":"1.0.0"}}}
{"jsonrpc":"2.0","method":"notifications/initialized","params":{}}
{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}

While DAB remains running, enable a stored-procedure entity as a custom MCP tool:

{
  "entities": {
    "GetBook": {
      "source": {
        "object": "dbo.GetBook",
        "type": "stored-procedure"
      },
      "mcp": {
        "custom-tool": true
      }
    }
  }
}

After the configuration reload succeeds, an initialized stdio client receives:

{"jsonrpc":"2.0","method":"notifications/tools/list_changed","params":{}}

Request the tool list again to retrieve the newly published snapshot:

{"jsonrpc":"2.0","id":3,"method":"tools/list","params":{}}

[Copilot]

Pull request overview

This PR adds robust hot-reload support for MCP tools by rebuilding and atomically publishing tool-registry snapshots whenever runtime configuration/metadata refreshes, keeping tools/list and tools/call aligned with the latest successfully applied generation across both HTTP and stdio transports.

Changes:

• Introduces an immutable, versioned MCP tool-registry snapshot model with ordered refresh tied into the hot-reload pipeline.
• Serializes initial runtime initialization and file-triggered hot reloads through a shared gate, and adds coordinated shutdown draining/cancellation.
• Propagates cancellation across metadata discovery and query execution paths, and adds stdio notifications/tools/list_changed.

Reviewed changes

Copilot reviewed 58 out of 58 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/Service/Utilities/RuntimeInitializationHelper.cs	Centralizes serialized startup init path.
src/Service/Utilities/RuntimeConfigLoaderShutdownService.cs	Drains loader work on host shutdown.
src/Service/Utilities/McpStdioHelper.cs	Runs serialized init + loader drain.
src/Service/Telemetry/LogLevelInitializer.cs	Adds cancellation checks on reload.
src/Service/Startup.cs	Uses new init helper + shutdown service.
src/Service/Program.cs	Registers stdio tool-list notifier services.
src/Service.Tests/UnitTests/SqlQueryExecutorUnitTests.cs	Adds linked cancellation unit test.
src/Service.Tests/UnitTests/McpStdoutWriterTests.cs	Tests nonblocking dispose with blocked stdout.
src/Service.Tests/UnitTests/McpStdioToolListChangedNotifierTests.cs	Adds notifier behavior/unit coverage.
src/Service.Tests/UnitTests/McpStdioServerRunAsyncTests.cs	Verifies notifier initialization handshake behavior.
src/Service.Tests/UnitTests/McpStdioServerInitializeTests.cs	Ensures listChanged advertised only when supported.
src/Service.Tests/UnitTests/McpStdioHelperTests.cs	Ensures stdio init order: metadata then registry.
src/Service.Tests/UnitTests/McpServerConfigurationTests.cs	Verifies HTTP capabilities + snapshot discovery filtering.
src/Service.Tests/UnitTests/ConfigFileWatcherUnitTests.cs	Adds serialization/drain/shutdown tests for loader.
src/Service.Tests/Mcp/McpToolRegistryTests.cs	Updates tests for snapshot/ordering/semantic comparison.
src/Service.Tests/Mcp/McpStdioToolRegistryHotReloadIntegrationTests.cs	Adds stdio hot-reload notification integration test.
src/Service.Tests/Mcp/McpMetadataHelperTests.cs	Updates metadata helper tests + new overload coverage.
src/Service.Tests/Mcp/McpInitialHotReloadSerializationTests.cs	Tests startup vs reload serialization correctness.
src/Service.Tests/Mcp/McpHttpToolRegistryHotReloadIntegrationTests.cs	Adds HTTP hot-reload integration coverage.
src/Service.Tests/Mcp/DynamicCustomToolTests.cs	Updates tool metadata init to explicit factory/config.
src/Service.Tests/Mcp/DynamicCustomToolMsSqlIntegrationTests.cs	Updates integration tests for new init signature.
src/Core/Services/OpenAPI/OpenApiDocumentor.cs	Adds cancellation checks on config change.
src/Core/Services/MetadataProviders/SqlMetadataProvider.cs	Adds cancellation-aware metadata discovery APIs.
src/Core/Services/MetadataProviders/MySqlMetadataProvider.cs	Propagates cancellation to schema discovery.
src/Core/Services/MetadataProviders/MsSqlMetadataProvider.cs	Propagates cancellation to metadata + autoentities.
src/Core/Services/MetadataProviders/MetadataProviderFactory.cs	Runs init with cancellation during reload.
src/Core/Services/MetadataProviders/ISqlMetadataProvider.cs	Adds default cancellation-aware interface method.
src/Core/Services/MetadataProviders/IMetadataProviderFactory.cs	Adds default cancellation-aware interface method.
src/Core/Services/MetadataProviders/CosmosSqlMetadataProvider.cs	Adds cancellation-aware no-op init.
src/Core/Services/GraphQLSchemaCreator.cs	Adds cancellation checks on config change.
src/Core/Resolvers/QueryExecutor.cs	Adds linked cancellation + token-aware overloads.
src/Core/Resolvers/PostgreSqlExecutor.cs	Adds cancellation to MI token acquisition.
src/Core/Resolvers/MySqlQueryExecutor.cs	Adds cancellation to MI token acquisition.
src/Core/Resolvers/MsSqlQueryExecutor.cs	Adds cancellation to OBO/MI token acquisition.
src/Core/Resolvers/IQueryExecutor.cs	Adds default cancellation-aware interface methods.
src/Core/Resolvers/Factories/QueryManagerFactory.cs	Adds cancellation checks on config change.
src/Core/Resolvers/Factories/QueryEngineFactory.cs	Adds cancellation checks on config change.
src/Core/Resolvers/Factories/MutationEngineFactory.cs	Adds cancellation checks on config change.
src/Core/Authorization/AuthorizationResolver.cs	Adds cancellation checks on config change.
src/Config/RuntimeConfigLoader.cs	Adds cancellation-aware ordered event signaling.
src/Config/HotReloadEventHandler.cs	Adds MCP registry event + docs.
src/Config/HotReloadEventArgs.cs	Adds CancellationToken to hot-reload args.
src/Config/FileSystemRuntimeConfigLoader.cs	Serializes reload + adds StopAsync drain/cancel.
src/Config/DabConfigEvents.cs	Adds MCP tool registry ordered event name.
src/Config/ConfigFileWatcher.cs	Adds stoppable watcher abstraction + lifecycle fixes.
src/Azure.DataApiBuilder.Mcp/Utils/McpMetadataHelper.cs	Adds overload accepting explicit metadata factory.
src/Azure.DataApiBuilder.Mcp/Core/McpToolRegistryRefreshService.cs	Adds refresh service + notifier integration.
src/Azure.DataApiBuilder.Mcp/Core/McpToolRegistryInitializer.cs	Removes startup-only hosted initializer.
src/Azure.DataApiBuilder.Mcp/Core/McpToolRegistry.cs	Implements immutable snapshot registry + canonical compare.
src/Azure.DataApiBuilder.Mcp/Core/McpStdoutWriter.cs	Makes dispose nonblocking under blocked stdout.
src/Azure.DataApiBuilder.Mcp/Core/McpStdioToolListChangedNotifier.cs	Implements stdio list-changed notifications.
src/Azure.DataApiBuilder.Mcp/Core/McpStdioServer.cs	Adds handshake gating + uses advertised snapshot.
src/Azure.DataApiBuilder.Mcp/Core/McpServiceCollectionExtensions.cs	Registers refresh service + removes config tool DI.
src/Azure.DataApiBuilder.Mcp/Core/McpServerConfiguration.cs	Uses advertised snapshot; disables HTTP listChanged.
src/Azure.DataApiBuilder.Mcp/Core/DynamicCustomTool.cs	Regenerates per snapshot + explicit metadata init.
src/Azure.DataApiBuilder.Mcp/Core/CustomMcpToolFactory.cs	Builds dynamic tools per generation (fail-fast).