YAML-based Playwright MCP Testing Framework for Windsurf

中文文档 | English Documentation

A powerful automation testing framework that combines Windsurf + Playwright MCP with natural language YAML-based testing, dynamic element targeting, multi-environment configuration, and session persistence.

🚀 About This Project

This is a standalone YAML-based Playwright testing framework that provides complete test automation capabilities. All scripts and commands are included in the project and ready to use without additional installations.

🧠 How Playwright MCP Works - The Core Innovation

🎯 Revolutionary Element Targeting System

Unlike traditional Playwright automation that relies on fragile CSS selectors or XPath expressions, Playwright MCP uses a revolutionary dynamic element identification system:

flowchart TD
    A["🌐 Web Page Loaded"] --> B["🔍 Playwright MCP Scans Page"]
    B --> C["🏷️ Assigns Unique ref_id to Each Element"]
    C --> D["📋 Creates Accessibility Snapshot"]
    D --> E["🤖 Windsurf Receives Element Map"]
    E --> F["🗣️ User Writes Natural Language Step"]
    F --> G["🎯 Windsurf Matches Description to ref_id"]
    G --> H["⚡ Executes Precise Action"]
    
    style A fill:#4CAF50,stroke:#2E7D32,stroke-width:3px,color:#ffffff
    style B fill:#2196F3,stroke:#1565C0,stroke-width:3px,color:#ffffff
    style C fill:#FF9800,stroke:#F57C00,stroke-width:3px,color:#ffffff
    style D fill:#9C27B0,stroke:#6A1B9A,stroke-width:3px,color:#ffffff
    style E fill:#F44336,stroke:#C62828,stroke-width:3px,color:#ffffff
    style F fill:#607D8B,stroke:#37474F,stroke-width:3px,color:#ffffff
    style G fill:#3F51B5,stroke:#283593,stroke-width:3px,color:#ffffff
    style H fill:#4CAF50,stroke:#2E7D32,stroke-width:3px,color:#ffffff

🎯 Dynamic ref_id Generation

When Playwright MCP accesses a web page, it automatically:

Scans all interactive elements on the page (buttons, inputs, links, etc.)
Assigns unique ref_id attributes to each element dynamically
Creates an accessibility snapshot with element descriptions and their corresponding ref_ids
Provides this mapping to Windsurf for intelligent element selection

🎯 Intelligent Element Selection

Instead of guessing selectors, Windsurf can:

See exactly what elements exist on the page with human-readable descriptions
Reference elements by their unique ref_id with 100% accuracy
Avoid brittle selector-based failures that plague traditional automation
Handle dynamic content without manual selector updates

🎯 Natural Language to Precise Actions

# Your YAML test step:
- "Click Add to Cart button for first product"

# What happens behind the scenes:
# 1. Playwright MCP identifies all "Add to Cart" buttons
# 2. Assigns ref_ids: button[ref_id="addcart_123"], button[ref_id="addcart_456"]
# 3. Windsurf intelligently selects the first one: ref_id="addcart_123"
# 4. Executes precise click action without guessing selectors

🎯 Benefits Over Traditional Automation

| Traditional Approach | Playwright MCP Approach | |---------------------|------------------------| | page.click('#add-cart-btn-1') | Windsurf sees "Add to Cart button for Sauce Labs Backpack" with ref_id | | Brittle CSS selectors | Dynamic element identification | | Breaks when HTML changes | Adapts to page structure automatically | | Requires manual maintenance | Self-healing element detection | | Multiple retry attempts | First-time accurate targeting |

This is why our YAML-based approach is so powerful - you write in natural language, and Playwright MCP handles the complex element targeting automatically.

🌟 Key Features

🌍 Multi-Environment Support: Support for dev/test/prod environments with automatic configuration loading
📚 Reusable Step Libraries: Parameterized reusable step libraries to improve testing efficiency
🗣️ Natural Language: Direct use of natural language for test step descriptions, making tests easy to read and write
🔧 Environment Variables: Automatic configuration loading from .env files with secure management of sensitive information
📊 Smart Reporting: Configurable test report generation with embedded data in HTML/JSON formats
📝 Smart Prompts: Windsurf project commands support parameter prompts
🚀 Session Persistence: Revolutionary cross-command session persistence - login once, test forever
⚡ Performance Boost: 80-95% performance improvement after first login with persistent sessions

🗺️ Development Roadmap

We're actively working on exciting new features to make YAML-based testing even more powerful:

✅ Completed Features

✅ Test Suites Support - COMPLETED 🎉

✅ Suite Organization: Group related test cases into logical suites
✅ Batch Execution: Run entire test suites with a single command
✅ Suite-level Configuration: Environment variables and settings per suite
✅ Suite Reporting: Aggregated reports across multiple test cases
✅ Pre/Post Actions: Suite-level setup and cleanup operations
✅ Validation Commands: Comprehensive suite validation functionality

# Example: test-suites/e-commerce.yml
name: "E-commerce Test Suite"
description: "Complete e-commerce workflow testing"
tags:
  - e-commerce
  - integration
test-cases:
  - "test-cases/order.yml"
  - "test-cases/product-details.yml"
  - "test-cases/sort-optimized.yml"

Available Suite Commands:

/run-test-suite suite:e-commerce.yml env:test
/validate-test-suite suite:smoke-tests.yml env:dev

📅 Release Timeline

| Feature | Status | Expected Release | |---------|--------|------------------| | ✅ Test Suites Support | ✅ Completed | ✅ Released |

🚀 Quick Start

1. Install Dependencies

npm install

2. Install Playwright MCP

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": [
        "@playwright/mcp@latest",
        "--browser=chrome",
        "--output-dir=/Users/xxx/playwright_output",
        "--save-trace",
        "--user-data-dir=/Users/xxx/playwright_user_data"
      ]
    }
  }
}

3. Use This Framework

# Clone this project
git clone https://github.com/terryso/windsurf-playwright-mcp-test.git
cd windsurf-playwright-mcp-test

# Open the project with Windsurf
windsurf .

Then in the Windsurf chat window, send:

/run-yaml-test file:test-cases/order.yml env:dev

Simple YAML Test Example

# test-cases/example.yml
tags:
  - smoke
steps:
  - include: "login"
  - "Click Add to Cart button for first product"
  - "Click shopping cart icon"
  - "Verify cart contains 1 item"

📚 Documentation

📖 Installation Guide - Detailed setup instructions
🏗️ Project Structure - Understanding the framework structure
⚡ Commands Reference - Complete command documentation
📝 YAML Format Guide - Writing test cases and step libraries
🔧 Environment Configuration - Multi-environment setup
✨ Best Practices - Tips for effective testing

📊 Latest Test Results

📈 Latest Test Report - Automatically generated after each test run, showing detailed execution results, screenshots, and performance metrics.

💡 Feature Requests

Have ideas for new features? We'd love to hear from you!

Open an Issue with the enhancement label
Join discussions in our community
Contribute to the roadmap planning

🤝 Contributing

Fork the project
Create a feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add some amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🔧 Tools & Integrations

🛠️ Windsurf - AI-powered development environment
🎭 Playwright MCP - Browser automation integration

Happy Testing! 🚀

MCP Servers