@yawlabs/mcp-compliance

Test any MCP server for spec compliance. 43-test suite covering transport, lifecycle, tools, resources, prompts, error handling, and schema validation against the MCP specification. CLI, MCP server, and programmatic API.

Built and maintained by YawLabs.

Quick Start

1. Run against any MCP server

npx @yawlabs/mcp-compliance test https://my-server.com/mcp

2. Or install globally

npm install -g @yawlabs/mcp-compliance
mcp-compliance test https://my-server.com/mcp

That's it. You'll get a colored terminal report with a letter grade (A-F), per-test pass/fail, and a compliance score.

CLI Usage

# Terminal output with colors and grade
mcp-compliance test https://my-server.com/mcp

# JSON output (for scripting)
mcp-compliance test https://my-server.com/mcp --format json

# Strict mode — exits with code 1 on required test failure (for CI)
mcp-compliance test https://my-server.com/mcp --strict

# With authentication
mcp-compliance test https://my-server.com/mcp --auth "Bearer tok123"

# Custom headers (repeatable)
mcp-compliance test https://my-server.com/mcp -H "Authorization: Bearer tok123" -H "X-Api-Key: abc"

# Custom timeout (default: 15000ms)
mcp-compliance test https://my-server.com/mcp --timeout 30000

# Retry failed tests
mcp-compliance test https://my-server.com/mcp --retries 2

# Only run specific categories or test IDs
mcp-compliance test https://my-server.com/mcp --only transport,lifecycle
mcp-compliance test https://my-server.com/mcp --only lifecycle-init,tools-list

# Skip specific categories or test IDs
mcp-compliance test https://my-server.com/mcp --skip prompts,resources

# Verbose mode — print each test result as it runs
mcp-compliance test https://my-server.com/mcp --verbose

Options

| Option | Description | |--------|-------------| | --format <format> | Output format: terminal or json (default: terminal) | | --strict | Exit with code 1 on any required test failure (for CI) | | -H, --header <header> | Add header to all requests, format "Key: Value" (repeatable) | | --auth <token> | Shorthand for -H "Authorization: <token>" | | --timeout <ms> | Request timeout in milliseconds (default: 15000) | | --retries <n> | Number of retries for failed tests (default: 0) | | --only <items> | Only run tests matching these categories or test IDs (comma-separated) | | --skip <items> | Skip tests matching these categories or test IDs (comma-separated) | | --verbose | Print each test result as it runs |

Get badge markdown

mcp-compliance badge https://my-server.com/mcp

Outputs the markdown embed for a compliance badge hosted at mcp.hosting.

What the 43 tests check

Grading

| Grade | Score | |-------|--------| | A | 90-100 | | B | 75-89 | | C | 60-74 | | D | 40-59 | | F | 0-39 |

Required tests are worth 70% of the score, optional tests 30%.

CI Integration

# GitHub Actions example
- name: MCP Compliance Check
  run: npx @yawlabs/mcp-compliance test ${{ env.MCP_SERVER_URL }} --strict

# With JSON output for parsing
- name: MCP Compliance Check
  run: |
    npx @yawlabs/mcp-compliance test ${{ env.MCP_SERVER_URL }} --format json > compliance.json
    cat compliance.json | jq '.grade'

# With retries for flaky network conditions
- name: MCP Compliance Check
  run: npx @yawlabs/mcp-compliance test ${{ env.MCP_SERVER_URL }} --strict --retries 2 --timeout 30000

MCP Server (for Claude Code, Cursor, etc.)

This package also exposes an MCP server with 3 tools that can be used from Claude Code, Cursor, or any MCP client.

Setup

Create .mcp.json in your project root:

macOS / Linux / WSL:

{
  "mcpServers": {
    "mcp-compliance": {
      "command": "npx",
      "args": ["-y", "@yawlabs/mcp-compliance", "mcp"]
    }
  }
}

Windows:

{
  "mcpServers": {
    "mcp-compliance": {
      "command": "cmd",
      "args": ["/c", "npx", "-y", "@yawlabs/mcp-compliance", "mcp"]
    }
  }
}

Tip: This file is safe to commit — it contains no secrets.

Restart your MCP client and approve the server when prompted.

Tools

• mcp_compliance_test — Run the full 43-test suite against a URL. Supports auth, custom headers, timeout, retries, and category/test filtering. Returns grade, score, and detailed results.
• mcp_compliance_badge — Get the badge markdown/HTML for a server. Supports auth and custom headers.
• mcp_compliance_explain — Explain what a specific test ID checks and why it matters.

All tools have MCP tool annotations (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) so MCP clients can skip confirmation dialogs for safe operations.

Programmatic Usage

import { runComplianceSuite } from '@yawlabs/mcp-compliance';

const report = await runComplianceSuite('https://my-server.com/mcp');
console.log(`Grade: ${report.grade} (${report.score}%)`);

// With options
const report2 = await runComplianceSuite('https://my-server.com/mcp', {
  headers: { 'Authorization': 'Bearer tok123' },
  timeout: 30000,
  retries: 1,
  only: ['transport', 'lifecycle'],
});

Requirements

• Node.js 18+

Links

• mcp.hosting — Hosted MCP server infrastructure
• MCP Specification
• Yaw Labs

License

MIT