mcp-ai-research-kit

A powerful AI research agent toolkit built on Claude Code MCP, providing a complete workflow from broad search to deep scraping and interactive execution.

English | 中文

Features

| Tool | Description | |------|-------------| | Tavily Search | Deep web search with advanced search modes | | Web Scraper | Multi-platform web content scraping | | Browser Automation | Browser automation skill execution | | Deep Research | Gemini-powered async deep research | | Free Model Router | Free model summarization and keyword extraction | | Trend Monitor | HackerNews & GitHub trending monitoring |

Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Claude Code                              │
└─────────────────────┬───────────────────────────────────────┘
                      │
        ┌─────────────┴─────────────┐
        │      MCP Protocol        │
        └─────────────┬─────────────┘
                      │
    ┌─────────┬───────┴───────┬─────────┬─────────┐
    │         │               │         │         │
    ▼         ▼               ▼         ▼         ▼
┌───────┐ ┌───────┐    ┌──────────┐ ┌──────┐ ┌───────┐
│Search │ │Scraper│    │ Research │ │ Free │ │Trends │
│ Server│ │ Server│    │  Server  │ │Model │ │ Monitor│
└───────┘ └───────┘    └──────────┘ └──────┘ └───────┘
    │         │               │         │         │
    ▼         ▼               ▼         ▼         ▼
┌───────┐ ┌───────┐    ┌──────────┐ ┌──────┐ ┌───────┐
│Tavily │ │Playwrht│    │ Gemini  │ │Open  │ │ SQLite│
│   API │ │        │    │    API  │ │Router│ │       │
└───────┘ └───────┘    └──────────┘ └──────┘ └───────┘

Prerequisites

Claude Code - Installed and configured
Python 3.9+ - Anaconda or other Python environment
Playwright - Browser driver (for web scraping and automation)

Quick Start

1. Clone the Repository

git clone https://github.com/jinhong219220-lab/mcp-ai-research-kit.git
cd mcp-ai-research-kit

2. Install Dependencies

# Using Anaconda
conda install -c conda-forge playwright
playwright install chromium

# Or using pip
pip install -r requirements.txt

3. Configure API Keys

Edit config/api_config.yaml and add your API keys:

# API Configuration
tavily_api_key: "your_TAVILY_API_KEY"
google_api_key: "your_GOOGLE_API_KEY"
openrouter_api_key: "your_OPENROUTER_API_KEY"

4. Add MCP Servers to Claude Code

Run the following commands in your terminal (adjust the Python path as needed):

# Add MCP servers
claude mcp add -e TAVILY_API_KEY="your_key" tavily-search -- python "path/to/tavily_mcp.py"
claude mcp add -e GOOGLE_API_KEY="your_key" deep-research -- python "path/to/async_research_mcp.py"
claude mcp add -e OPENROUTER_API_KEY="your_key" free-model -- python "path/to/free_model_mcp.py"
claude mcp add web-scraper -- python "path/to/mcp_server.py"
claude mcp add browser-auto -- python "path/to/browser_mcp.py"
claude mcp add trend-monitor -- python "path/to/trend_monitor_mcp.py"

Note: Replace path/to with your actual file path.

Usage Examples

1. Search

Use tavily_search to search for Python async programming best practices

Parameters:

query: Search keywords
search_depth: basic or advanced
max_results: Number of results (1-20)
topic: general, news, or finance

2. Web Scraping

Use scrape_url to scrape https://example.com

Parameters:

url: Target URL
platform: wechat/xiaohongshu/bilibili/generic
wait_for: CSS selector to wait for
extract_text: Extract plain text (default: True)

3. Bilibili Video Info

Use get_bilibili_info to get video info for BV1xx411c7mD

4. Text Summarization

Use summarize_text to summarize: [paste your text]

5. Keyword Extraction

Use extract_keywords to extract keywords from text

6. Trend Monitoring

Use refresh_trends to refresh trend data
Use get_latest_trends to get trends from last 24 hours

API Key Guide

| Service | Website | Free Tier | |---------|---------|-----------| | Tavily | tavily.com | 1000 credits/month | | Google Gemini | ai.google.dev | $0/month (with limits) | | OpenRouter | openrouter.ai | $5/month |

Project Structure

mcp-ai-research-kit/
├── mcp_servers/
│   ├── search/
│   │   └── tavily_mcp.py          # Search MCP
│   ├── scraper/
│   │   ├── mcp_server.py          # Scraper MCP
│   │   └── docker-compose.yml     # Docker config
│   ├── browser/
│   │   ├── browser_mcp.py        # Browser automation MCP
│   │   └── skills/               # Skill scripts
│   ├── research/
│   │   └── async_research_mcp.py # Deep research MCP
│   ├── free_model/
│   │   └── free_model_mcp.py     # Free model MCP
│   └── trends/
│       └── trend_monitor_mcp.py  # Trend monitoring MCP
├── config/
│   ├── settings.yaml
│   └── api_config.yaml
├── .gitignore
├── LICENSE
├── README.md
├── CONTRIBUTING.md
└── requirements.txt

Contributing

Contributions are welcome! Please read our Contributing Guide first.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Disclaimer

This tool is for educational and research purposes only. Ensure compliance with websites' terms of service when scraping content.

Star ⭐ if you find this useful!

MCP Servers