nl-mcp-tool-contract-gates

An educational local C# project showing how to gate a real MCP server profile by launching it over STDIO, discovering its live tools and JSON schemas, probing those tools through the MCP client, and blocking risky tool surface drift before promotion.

This version uses a support operations scenario with one baseline MCP profile, one candidate MCP profile, a live MCP client, frozen role cases, and file-backed JSON gate reports.

Overview

This repository demonstrates a practical live MCP gate:

1. Launch a real MCP server profile over STDIO.
2. Discover the live MCP tools exposed by that profile.
3. Extract actual tool names, JSON schemas, and tool annotations from the MCP protocol response.
4. Compare a baseline profile and a candidate profile for added tools, removed tools, schema breakage, read-only hint regressions, and newly added destructive tools.
5. Replay frozen role cases by calling the live MCP tools with structured arguments.
6. Apply deterministic promotion rules to decide Promote, Hold, or Rollback.
7. Persist the report as JSON for later review.

The result is a real MCP control loop rather than a simulated contract comparison over static snapshot files.

What This Project Demonstrates

• How to host a real MCP server in .NET with STDIO transport
• How to expose tools with [McpServerTool] names and annotations
• How to launch that server from a C# MCP client
• How to discover live MCP tools and inspect their actual JSON schemas
• How to compare baseline and candidate MCP profiles using live discovery
• How to probe discovered tools through MCP calls before promotion
• How to block newly exposed destructive tools from leaking into a narrower role surface

Prerequisites

• .NET 10 SDK or later

No external model endpoint or cloud MCP host is required. The sample uses a local MCP server process started by the gate client itself.

Quick Start

From the project root:

dotnet run --project McpToolContractGates

The app:

• launches the baseline MCP profile
• discovers its live tools
• launches the candidate MCP profile
• discovers its live tools
• compares both live MCP surfaces
• probes the required tools through real MCP calls
• prints the promotion decision
• saves a JSON report under McpToolContractGates/bin/Debug/net10.0/data/reports/

Configuration

Default settings are in McpToolContractGates/appsettings.json under Experiment and Promotion.

Example:

{
  "Experiment": {
    "ServerCommand": "dotnet",
    "BaselineProfile": "baseline",
    "CandidateProfile": "candidate",
    "DatasetPath": "data/mcp_tool_gate_eval.json",
    "ReportDirectory": "data/reports"
  },
  "Promotion": {
    "BlockOnRemovedTools": true,
    "BlockOnSchemaBreakage": true,
    "BlockOnReadOnlyHintRegression": true,
    "BlockOnNewDestructiveTools": true,
    "MaxNewDestructiveTools": 0,
    "MaxBrokenCases": 0
  }
}

Environment variable overrides use prefix MCPGATE_.

How It Works

1. SupportOpsMcpServer is a real MCP server host using WithStdioServerTransport().
2. The server accepts one profile argument: baseline or candidate.
3. ActualMcpSession launches that server with dotnet <server.dll> <profile>.
4. The MCP client discovers live tools with ListToolsAsync().
5. DiscoveredToolSurface extracts real tool names, JSON schemas, and ReadOnlyHint / DestructiveHint metadata from the MCP response.
6. LiveToolSurfaceComparer compares the baseline and candidate profiles.
7. LiveToolProbeRunner calls the required MCP tools with structured arguments from the frozen role dataset.
8. PromotionGate turns the findings into an explicit Promote, Hold, or Rollback decision.

Project Structure

.
+-- McpToolContractGates.slnx
+-- SupportOpsMcpServer/
|   +-- SupportOpsMcpServer.csproj
|   +-- Program.cs
|   +-- ServerAssemblyMarker.cs
|   +-- BaselineSupportOpsTools.cs
|   +-- CandidateSupportOpsTools.cs
+-- McpToolContractGates/
|   +-- McpToolContractGates.csproj
|   +-- Program.cs
|   +-- appsettings.json
|   +-- App/
|   |   +-- ExperimentConfig.cs
|   |   +-- PromotionGateConfig.cs
|   +-- Contracts/
|   |   +-- DiscoveredToolContract.cs
|   |   +-- RequiredToolExpectation.cs
|   |   +-- ToolGateEvalCase.cs
|   +-- Data/
|   |   +-- ToolGateDatasetLoader.cs
|   |   +-- mcp_tool_gate_eval.json
|   +-- Evaluation/
|   |   +-- Actual discovery + diff + probe + gate types
|   +-- Mcp/
|   |   +-- ActualMcpSession.cs
|   +-- Persistence/
|       +-- JsonGateReportStore.cs
+-- McpToolContractGates.Tests/
|   +-- McpToolContractGates.Tests.csproj
|   +-- ActualMcpDiscoveryTests.cs
|   +-- LiveToolSurfaceComparerTests.cs
|   +-- EndToEndGateTests.cs

Control Guarantees

• The server is a real MCP process, not a simulated adapter
• Tool discovery comes from actual MCP ListToolsAsync() results
• Required arguments come from the live MCP JSON schemas
• Tool hints come from live MCP tool annotations
• Role-case probes call real MCP tools over STDIO
• New destructive tools can be blocked before promotion
• Full reports are persisted so decisions remain inspectable

Tests

Run the test suite from the repo root:

dotnet test McpToolContractGates.slnx

The tests cover:

• live MCP discovery of the baseline profile
• detection of live schema breakage and added destructive tools
• end-to-end rollback of the candidate profile against the real MCP server

License

See the LICENSE file for details.

Contributing

Contributions are welcome for improvements within current project scope.

Suggested areas:

• Add streamable HTTP transport support alongside STDIO so the same gate can evaluate remote MCP deployments
• Extend the sample from one support role to multiple role surfaces with different allowed and forbidden tools
• Persist longitudinal diff history so tool-surface drift can be inspected across multiple promotion attempts
• Add policy exceptions for explicitly approved tool additions while keeping the rest of the surface gated
• Expand the probe dataset to cover more schema edge cases, hint regressions, and destructive-tool scenarios