Layout Detector MCP

An MCP (Model Context Protocol) server that analyzes webpage screenshots to extract layout information. Given a screenshot and image assets, it finds where each asset appears and calculates spatial relationships - enabling AI assistants to rebuild layouts with proper semantic structure.

Quick Start

Install from GitHub

pip install git+https://github.com/katlis/layout-detector-mcp.git

Or clone and install locally

git clone https://github.com/katlis/layout-detector-mcp.git
cd layout-detector-mcp
pip install .

Verify installation:

python3 -c "from layout_detector import server; print('OK')"

Configuration

Add to your Claude Code MCP settings (~/.claude.json or project .claude/settings.json):

{
  "mcpServers": {
    "layout-detector": {
      "command": "layout-detector-mcp"
    }
  }
}

After adding the configuration, restart Claude Code and run /mcp to verify the server is connected.

The Problem

When an AI assistant looks at a screenshot, it can describe what it sees but cannot extract precise pixel measurements. This makes it difficult to accurately recreate layouts without human intervention or extensive trial-and-error.

The Solution

This MCP server uses computer vision (OpenCV template matching) to:

1. Find known assets - Locate images within a screenshot with pixel-perfect coordinates
2. Analyze relationships - Calculate angles, distances, and relative positions
3. Detect patterns - Identify radial, grid, stacked, sidebar, or freeform layouts
4. Enable semantic rebuilds - Provide structured data for modern CSS implementation

Tools

analyze_layout

Performs full layout analysis including pattern detection. This is the main tool you'll use.

Parameters:

• screenshot_path (string, required): Absolute path to the screenshot image
• asset_paths (array of strings, required): Absolute paths to asset images to find
• threshold (number, optional): Match confidence 0-1, default 0.8

Returns:

{
  "viewport": { "width": 900, "height": 650 },
  "pattern": {
    "type": "radial",
    "confidence": 0.90
  },
  "radial": {
    "center_x": 450,
    "center_y": 250,
    "center_element": "logo.gif",
    "average_radius": 196
  },
  "elements": [
    {
      "asset_name": "planet1.gif",
      "x": 628,
      "y": 89,
      "width": 62,
      "height": 62,
      "angle_degrees": 45.0,
      "distance_from_center": 240
    }
  ]
}

find_assets_in_screenshot

Locates image assets within a screenshot without layout analysis.

Parameters:

• screenshot_path (string, required): Path to the screenshot image
• asset_paths (array of strings, required): Paths to asset images to find
• threshold (number, optional): Match confidence 0-1, default 0.8

Returns:

{
  "found": 5,
  "total_assets": 6,
  "matches": [
    {
      "asset_path": "/path/to/logo.png",
      "asset_name": "logo.png",
      "x": 350,
      "y": 200,
      "width": 200,
      "height": 100,
      "center_x": 450,
      "center_y": 250,
      "confidence": 0.95
    }
  ]
}

get_screenshot_info

Get basic screenshot dimensions.

Parameters:

• screenshot_path (string, required): Path to the screenshot image

Returns:

{
  "path": "/path/to/screenshot.png",
  "width": 900,
  "height": 650
}

Supported Layout Patterns

| Pattern | Description | Key Data Returned | |---------|-------------|-------------------| | Radial | Elements arranged around a center point | Center element, angles, distances | | Grid | Elements in rows and columns | Row/column positions, gaps | | Stacked | Vertical sections (header/main/footer) | Section names, Y positions | | Sidebar | Two-column with narrow sidebar | Sidebar side, widths | | Freeform | No clear pattern | Raw X/Y coordinates |

Example Usage

Once configured, Claude Code can use these tools:

User: Rebuild this webpage screenshot using the images in /assets

Claude: I'll analyze the layout first using the layout detector.

[Calls analyze_layout tool]

The analysis shows:
- Viewport: 900x650px
- Pattern: Radial (90% confidence)
- Center element: logo.gif at (450, 250)
- 8 elements arranged around the center
- Average distance from center: 196px

I'll implement this using CSS with the logo centered and
other elements positioned using absolute positioning...

Supported Image Formats

• PNG
• JPEG
• GIF (including animated - uses first frame)
• WebP
• BMP

Troubleshooting

"No module named 'cv2'"

OpenCV isn't installed. Run:

pip install opencv-python-headless

MCP server not showing in /mcp

1. Check your settings file path is correct
2. Ensure the command path is absolute (for source installs)
3. Restart Claude Code after changing settings
4. Run python3 test_install.py to verify the package works

Low confidence matches

Try lowering the threshold parameter (default 0.8). Values between 0.6-0.7 may help with compressed or scaled images.

Development

# Install in editable mode with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Test installation
python3 test_install.py

Requirements

• Python 3.11+
• OpenCV (opencv-python-headless)
• NumPy
• Pillow
• MCP SDK

License

MIT