TOON-MCP: Token-Optimized Object Notation

TOON-MCP is a Model Context Protocol server that automatically converts verbose JSON structures into Token-Optimized Object Notation (TOON), reducing token consumption in AI-assisted development workflows by up to 60%.

🚀 Features

• Smart Compression: Automatic pattern detection and intelligent JSON optimization
• Token Monitoring: Real-time tracking of conversation token usage with optimization recommendations
• MCP Integration: Seamless integration with Claude and other MCP clients
• Lossless Conversion: Perfect round-trip conversion between JSON and TOON
• Auto-Optimization: Proactive conversion of tool outputs for maximum efficiency
• Pre-commit Hooks: Scan code for JSON and suggest TOON conversions
• Context Management: Monitor and optimize token usage across conversations

📊 Quick Example

Original JSON (142 tokens):

{
  "id": 12345,
  "name": "John Doe",
  "email": "john@example.com",
  "type": "user",
  "status": "active",
  "metadata": {
    "created_at": "2025-01-01T00:00:00Z",
    "updated_at": "2025-01-15T10:30:00Z"
  }
}

TOON Format (68 tokens, 52% reduction):

{"_toon":"1.0","d":{"i":12345,"n":"John Doe","eml":"john@example.com","t":"user","s":"active","meta":{"ca":"2025-01-01T00:00:00Z","ua":"2025-01-15T10:30:00Z"}}}

📦 Installation

Prerequisites

• Python 3.10 or higher
• pip package manager

Quick Install

# Clone the repository
git clone https://github.com/aj-geddes/toon-context-mcp.git
cd toon-context-mcp

# Install TOON-MCP
cd mcp-server-toon
pip install -e .

Docker Installation

TOON-MCP can be run in a Docker container for easy deployment and isolation:

# Clone the repository
git clone https://github.com/aj-geddes/toon-context-mcp.git
cd toon-context-mcp/mcp-server-toon

# Build the Docker image
docker build -t toon-mcp-server:latest .

# Run with Docker
docker run -i toon-mcp-server:latest

# Or use Docker Compose
docker-compose up -d

Note: The Docker image uses Debian-based Python (python:3.10-slim) for optimal performance, not Alpine.

For MCP integration with Docker, update your Claude Desktop config to use the containerized server:

{
  "mcpServers": {
    "toon": {
      "command": "docker",
      "args": ["run", "-i", "toon-mcp-server:latest"]
    }
  }
}

MCP Configuration

Add to your Claude Desktop config (~/.config/Claude/claude_desktop_config.json):

{
  "mcpServers": {
    "toon": {
      "command": "python",
      "args": ["-m", "src.server"],
      "cwd": "/path/to/toon-context-mcp/mcp-server-toon"
    }
  }
}

🎯 Project Structure

toon-context-mcp/
├── mcp-server-toon/          # Phase 1: Core MCP Server
│   ├── src/
│   │   ├── toon_converter.py  # Core TOON logic
│   │   ├── server.py          # MCP server
│   │   └── patterns.py        # Pattern detection
│   ├── tests/
│   │   └── test_conversions.py
│   ├── pyproject.toml
│   └── README.md
├── claude-code-integration/   # Phase 2: Claude Code Integration
│   ├── auto_converter.py      # Auto-conversion system
│   └── example_generator.py   # TOON example generator
├── context-manager/           # Phase 3: Context Manager
│   ├── token_monitor.py       # Token usage monitoring
│   ├── tool_output_optimizer.py # Proactive conversion
│   └── mcp_integration.py     # MCP integration helpers
├── docs/                      # Complete Jekyll documentation
│   ├── index.md
│   ├── guides/
│   │   ├── setup.md
│   │   ├── user-guide.md
│   │   └── troubleshooting.md
│   └── api/
│       └── reference.md
└── .git/hooks/
    └── pre-commit              # JSON scanning hook

🔧 Usage

Basic Conversion

from src.toon_converter import convert_json_to_toon, convert_toon_to_json

# Convert to TOON
toon_format = convert_json_to_toon(your_json_data)

# Convert back to JSON
original = convert_toon_to_json(toon_format)

MCP Tools

Use these tools in Claude:

• convert_to_toon - Convert JSON to TOON format
• convert_to_json - Convert TOON back to JSON
• analyze_patterns - Detect optimization patterns
• get_compression_strategy - Get optimal compression strategy
• calculate_savings - Calculate token savings
• batch_convert - Convert multiple JSON objects

Token Monitoring

from context_manager.token_monitor import TokenMonitor

monitor = TokenMonitor(warn_threshold=50000)

# Analyze messages
monitor.analyze_message(message_content, role='user')

# Get metrics
metrics = monitor.get_metrics()
print(f"Total tokens: {metrics.total_tokens:,}")
print(f"Potential savings: {metrics.savings_percent:.1f}%")

Tool Output Optimization

from context_manager.tool_output_optimizer import ToolOutputOptimizer

optimizer = ToolOutputOptimizer(auto_optimize=True, min_savings=15.0)

# Automatically optimize tool outputs
optimized, metadata = optimizer.optimize_tool_output("api_call", tool_output)

📚 Documentation

Complete documentation available at /docs:

• Setup Guide - Installation and configuration
• User Guide - Usage patterns and examples
• Troubleshooting - Common issues and solutions
• API Reference - Complete API documentation

View Documentation Locally

cd docs
# Install Jekyll (if not already installed)
gem install bundler jekyll

# Create Gemfile
cat > Gemfile << 'EOF'
source "https://rubygems.org"
gem "jekyll", "~> 4.3"
gem "minima", "~> 2.5"
gem "jekyll-seo-tag"
gem "jekyll-sitemap"
gem "jekyll-feed"
EOF

# Install and serve
bundle install
bundle exec jekyll serve

# Open http://localhost:4000

🧪 Testing

cd mcp-server-toon
pytest tests/ -v

🎨 Features by Phase

Phase 1: Core MCP Server ✅

• TOON converter with smart compression
• Pattern detection system
• MCP server with 6 tools
• Comprehensive test suite
• Package configuration

Phase 2: Claude Code Integration ✅

• Pre-commit hook for JSON scanning
• Auto-conversion system for code/docs
• TOON example generator
• Multiple language support

Phase 3: Context Manager ✅

• Token usage monitoring
• Proactive tool output conversion
• MCP server integration helpers
• Optimization recommendations

📈 Performance

Typical token savings by data type:

• API Responses: 40-60% savings
• Database Results: 50-65% savings
• Configuration Files: 35-50% savings
• Arrays of Objects: 55-70% savings

🤝 Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Run tests: pytest tests/ -v
4. Submit a pull request

📄 License

MIT License - see LICENSE file for details

🔗 Links

• Documentation: See /docs directory
• Issues: GitHub Issues
• Discussions: GitHub Discussions

🌟 Why TOON?

In AI-assisted development, token limits can constrain your workflow. TOON helps you:

• Fit more context in conversations
• Reduce API costs with fewer tokens
• Improve response times with smaller payloads
• Work with larger datasets within token limits

📊 Architecture

graph TB
    subgraph "TOON-MCP Server"
        A[MCP Server] --> B[TOON Converter]
        A --> C[Pattern Detector]
        A --> D[Token Monitor]
    end

    subgraph "Claude Code Integration"
        E[Pre-commit Hook] --> F[Auto-Converter]
        F --> G[Example Generator]
    end

    subgraph "Context Manager"
        H[Token Monitor] --> I[Tool Optimizer]
        I --> J[MCP Integration]
    end

    K[Claude / MCP Client] --> A
    A --> K

🎯 Roadmap

• [x] Core TOON converter
• [x] MCP server implementation
• [x] Pattern detection
• [x] Claude Code integration
• [x] Context manager
• [x] Complete documentation
• [ ] TypeScript/JavaScript port
• [ ] VSCode extension
• [ ] Browser extension
• [ ] Performance benchmarks
• [ ] Community templates

💡 Use Cases

1. Large Codebase Analysis: Compress file listings and search results
2. API Integration: Optimize API response storage in conversations
3. Database Queries: Reduce token usage when sharing query results
4. Configuration Management: Store configs in compact format
5. Documentation: Include examples without token overhead

Built with ❤️ for the AI development community

Reduce tokens. Increase productivity.