MCP Skill Telemetry Server

A lightweight Model Context Protocol (MCP) server that tracks AI skill usage through Azure Application Insights. Built for tracking custom skills in GitHub Copilot agent workflows.

Repository Structure

mcp-skill-telemetry/
├── README.md                           # You are here
├── LICENSE                             # MIT License
├── CONTRIBUTING.md                     # Contribution guidelines
│
├── instructions/                       # 📋 Copy this to your workspace
│   └── skill-telemetry.instructions.md # Global instruction file for agents
│
└── skill-telemetry/                    # 🔧 .NET MCP Server Project
    ├── mcp-skill-telemetry.sln        # Visual Studio solution
    ├── SkillTelemetry.csproj          # .NET 9.0 project
    ├── Program.cs                      # MCP server host
    └── SkillTelemetryTool.cs          # track_skill_usage tool implementation

🚀 Quick Setup

Prerequisites

• .NET 9.0 SDK (download)
• VS Code with MCP support
• (Optional) Azure Application Insights for telemetry

1. Copy Files to Your Workspace

your-workspace/
  .github/
    instructions/
      skill-telemetry.instructions.md   ← Copy from /instructions
    mcp-servers/
      skill-telemetry/                  ← Copy from /skill-telemetry
        SkillTelemetry.csproj
        Program.cs
        SkillTelemetryTool.cs
    skills/
      your-skill-1/                     ← Your existing skills (no changes needed!)
      your-skill-2/

2. Configure MCP Server in VS Code

Add to your .code-workspace file or .vscode/settings.json:

Option A: Reference System Environment Variable (Recommended)

This reads from your system environment variable (set in Step 3):

{
  "mcp.servers": {
    "skill-telemetry": {
      "type": "stdio",
      "command": "dotnet",
      "args": [
        "run",
        "--project",
        "${workspaceFolder}/.github/mcp-servers/skill-telemetry/SkillTelemetry.csproj"
      ],
      "env": {
        "APPINSIGHTS_CONNECTION_STRING": "${env:APPINSIGHTS_CONNECTION_STRING}"
      }
    }
  }
}

Option B: Hardcode Connection String Directly

No need to set a system environment variable with this approach:

{
  "mcp.servers": {
    "skill-telemetry": {
      "type": "stdio",
      "command": "dotnet",
      "args": [
        "run",
        "--project",
        "${workspaceFolder}/.github/mcp-servers/skill-telemetry/SkillTelemetry.csproj"
      ],
      "env": {
        "APPINSIGHTS_CONNECTION_STRING": "InstrumentationKey=xxx;IngestionEndpoint=https://..."
      }
    }
  }
}

Note: Hardcoding is simpler but less secure if you commit your workspace settings to source control.

Option C: Using Published Executable (Faster Startup)

First publish the server:

cd .github/mcp-servers/skill-telemetry
dotnet publish -c Release -o ./publish

Then configure with either approach above, changing the command:

{
  "mcp.servers": {
    "skill-telemetry": {
      "type": "stdio",
      "command": "${workspaceFolder}/.github/mcp-servers/skill-telemetry/publish/SkillTelemetry",
      "env": {
        "APPINSIGHTS_CONNECTION_STRING": "${env:APPINSIGHTS_CONNECTION_STRING}"
      }
    }
  }
}

3. Set System Environment Variable (Only if using Option A)

Windows PowerShell

# Session-level (temporary)
$env:APPINSIGHTS_CONNECTION_STRING = "InstrumentationKey=...;IngestionEndpoint=..."

# User-level (permanent)
[System.Environment]::SetEnvironmentVariable('APPINSIGHTS_CONNECTION_STRING', 'InstrumentationKey=...;IngestionEndpoint=...', 'User')

Linux/macOS

# Session-level (temporary)
export APPINSIGHTS_CONNECTION_STRING="InstrumentationKey=...;IngestionEndpoint=..."

# User-level (permanent) - add to ~/.bashrc or ~/.zshrc
echo 'export APPINSIGHTS_CONNECTION_STRING="..."' >> ~/.bashrc
source ~/.bashrc

4. Reload VS Code

Press Ctrl+Shift+P (or Cmd+Shift+P on Mac) → "Developer: Reload Window"

That's it! Skills are automatically tracked without modification.

Note: If you used Option B (hardcoded connection string), you're done! If you used Option A or want telemetry to be optional, the server will run without sending telemetry when APPINSIGHTS_CONNECTION_STRING is not set (useful for local development).

Repository Structure

mcp-skill-telemetry/
├── README.md                           # You are here
├── LICENSE                             # MIT License
├── CONTRIBUTING.md                     # Contribution guidelines
├── PUBLISHING.md                       # How to publish to GitHub
│
├── instructions/                       # 📋 Copy this to your workspace
│   └── skill-telemetry.instructions.md # Global instruction file for agents
│
└── skill-telemetry/                    # 🔧 .NET MCP Server Project
    ├── mcp-skill-telemetry.sln        # Visual Studio solution
    ├── SkillTelemetry.csproj          # .NET 9.0 project
    ├── Program.cs                      # MCP server host
    └── SkillTelemetryTool.cs          # track_skill_usage tool implementation

How It Works

The Global Instruction Pattern

The skill-telemetry.instructions.md file tells agents to automatically track all skills:

# Skill Telemetry

Whenever you activate a skill from `.github/skills/`, you **MUST**:
1. Before doing anything, call `track_skill_usage` with `action: "started"`
2. When finished, call with `action: "completed"`
3. If failed, call with `action: "failed"`

Your skills stay clean—no telemetry code needed:

---
name: code-discovery
description: Discovers code patterns
---

# Code Discovery Skill

1. Scan workspace for project files
2. Analyze patterns
3. Generate catalog

The agent framework handles telemetry automatically.

MCP Tool Reference

track_skill_usage

| Parameter | Type | Required | Description | |-----------|------|----------|-------------| | skillName | string | ✅ | Name of the skill (e.g. "code-discovery") | | action | string | ✅ | Lifecycle: "started", "completed", or "failed" | | workspaceContext | string | ❌ | Active workspace folder name | | agentName | string | ❌ | Name of the agent that triggered the skill |

Example Call

await mcpClient.callTool("track_skill_usage", {
  skillName: "code-discovery",
  action: "started",
  workspaceContext: "my-project",
  agentName: "research-agent"
});

☁️ Azure Application Insights Setup

Quick Setup (Azure CLI)

# Login to Azure
az login

# Create resource group
az group create --name mcp-telemetry-rg --location eastus

# Create Application Insights
az monitor app-insights component create \
  --app mcp-skill-telemetry \
  --location eastus \
  --resource-group mcp-telemetry-rg

# Get connection string
az monitor app-insights component show \
  --app mcp-skill-telemetry \
  --resource-group mcp-telemetry-rg \
  --query connectionString -o tsv

Azure Portal

1. Go to portal.azure.com
2. Create new resource → "Application Insights"
3. Fill in:
   • Name: mcp-skill-telemetry
   • Region: Choose closest
   • Resource Group: Create new or use existing
4. Click "Review + Create"
5. After creation, go to Overview → copy Connection String

📊 Verifying Telemetry

After running skills, verify events are flowing to Application Insights:

customEvents
| where name == "SkillUsage" and timestamp > ago(10m)
| project timestamp, skillName = customDimensions.skillName, action = customDimensions.action
| order by timestamp desc

Expected: 2 events per skill execution (started, completed)

🔧 Troubleshooting

Server Not Found in VS Code

• Check that the command path is correct and points to the .csproj file
• Verify .NET 9.0 SDK is installed: dotnet --version
• Reload VS Code window: Ctrl+Shift+P → "Developer: Reload Window"
• Check VS Code Output panel for MCP errors

"dotnet: command not found"

Install .NET 9.0 SDK from dotnet.microsoft.com

No Telemetry in Azure

• Verify APPINSIGHTS_CONNECTION_STRING is set correctly
• Wait 2-5 minutes for data ingestion
• Check the connection string format: InstrumentationKey=...;IngestionEndpoint=...
• Verify the Application Insights resource is not disabled
• Try calling the tool manually to test:

  await mcpClient.callTool("track_skill_usage", {
    skillName: "test-skill",
    action: "started"
  });
  

Build Errors

cd skill-telemetry
dotnet clean
dotnet restore
dotnet build

📚 Documentation

• Publishing - How to publish this as a public repository
• Contributing - Contribution guidelines

🔧 .NET Project Details

• Framework: .NET 9.0
• Dependencies:
  • ModelContextProtocol 0.8.0-preview.1
  • Microsoft.ApplicationInsights 2.*
  • Microsoft.Extensions.Hosting 9.*

Build & Run

cd skill-telemetry
dotnet build
dotnet run

Publish Executable

cd skill-telemetry
dotnet publish -c Release -o ./publish

Then use the executable in your MCP configuration:

{
  "mcp.servers": {
    "skill-telemetry": {
      "command": "/path/to/publish/SkillTelemetry"
    }
  }
}

☁️ Azure Application Insights Setup

Quick Setup (Azure CLI)

az monitor app-insights component create \
  --app mcp-skill-telemetry \
  --location eastus \
  --resource-group my-resource-group

# Get connection string
az monitor app-insights component show \
  --app mcp-skill-telemetry \
  --resource-group my-resource-group \
  --query connectionString -o tsv

Azure Portal

1. Create new Application Insights resource
2. Copy the Connection String from Overview page
3. Set as environment variable (see Quick Setup step 3)

⚖️ Reliability & Integration Options

The global instruction file is generally reliable with modern agent frameworks. For critical telemetry, you have options:

Option 1: Instruction File Only (Recommended)

✅ Skills stay completely clean
✅ Test to verify your agents load instructions

Option 2: Add YAML Signal (Self-Documenting)

---
name: my-skill
telemetry: auto  # Uses global instruction
---

Option 3: Add Brief Note (Maximum Visibility)

> **Telemetry:** Tracked via global instruction (start/complete/fail)

Recommendation: Start with Option 1, verify events appear in Application Insights, upgrade if needed.

See SKILL_INTEGRATION.md for detailed comparison.

📈 Example Queries

Most Used Skills

customEvents
| where name == "SkillUsage" and customDimensions.action == "started"
| summarize count() by tostring(customDimensions.skillName)
| order by count_ desc

Success Rate by Skill

customEvents
| where name == "SkillUsage" and customDimensions.action in ("completed", "failed")
| summarize 
    total = count(),
    succeeded = countif(customDimensions.action == "completed")
    by skillName = tostring(customDimensions.skillName)
| extend successRate = round(succeeded * 100.0 / total, 2)
| order by total desc

Agent Activity Timeline

customEvents
| where name == "SkillUsage" and isnotempty(customDimensions.agentName)
| summarize count() by bin(timestamp, 1h), agentName = tostring(customDimensions.agentName)
| render timechart

🔨 Adding Custom Telemetry

Extend SkillTelemetryTool.cs to add new tools or properties:

[McpServerTool(Name = "track_skill_metric"), Description("Track custom skill metrics")]
public string TrackSkillMetric(
    TelemetryClient telemetryClient,
    [Description("Metric name")] string metricName,
    [Description("Metric value")] double value)
{
    telemetryClient.TrackMetric(metricName, value);
    telemetryClient.Flush();
    return $"Tracked: {metricName} = {value}";
}

Then call from agents:

await mcpClient.callTool("track_skill_metric", {
  metricName: "skill_duration_ms", 
  value: 1234.5
});

🧪 Testing Without Azure

The server works without APPINSIGHTS_CONNECTION_STRING set—telemetry calls succeed but data isn't sent. Useful for:

• ✅ Local development
• ✅ CI/CD pipelines
• ✅ Testing integration without cloud dependency

✨ Features

✅ Lightweight .NET 9.0 MCP server
✅ Single tool: track_skill_usage
✅ Graceful degradation (works without telemetry configured)
✅ Zero-config for local development
✅ Global instruction pattern keeps skills clean
✅ Compatible with VS Code MCP integration
✅ Extensible for custom telemetry needs

Architecture

┌─────────────────┐         ┌──────────────────────┐         ┌─────────────────────┐
│  GitHub Copilot │  MCP    │  SkillTelemetry      │  HTTPS  │  Azure Application  │
│  Agent/Skill    │────────▶│  MCP Server          │────────▶│  Insights           │
└─────────────────┘         └──────────────────────┘         └─────────────────────┘
                                      │
                                      │ Uses
                                      ▼
                            ┌──────────────────────┐
                            │  TelemetryClient     │
                            │  (App Insights SDK)  │
                            └──────────────────────┘

Contributing

Contributions welcome! Please see CONTRIBUTING.md.

License

MIT License - see LICENSE file for details.

Support

For issues, questions, or feedback, please open a GitHub issue.

Related Projects

• Model Context Protocol
• Azure Application Insights
• .NET 9.0