# OpenTester

**MCP-First Testing Execution Infrastructure**

[![PyPI version](https://badge.fury.io/py/opentester.svg)](https://badge.fury.io/py/opentester)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![kznr02/OpenTester MCP server](https://glama.ai/mcp/servers/kznr02/open-tester/badges/score.svg)](https://glama.ai/mcp/servers/kznr02/open-tester)

OpenTester is a testing execution engine designed for AI coding tools (Claude Code, Cursor, OpenCode, etc.). It provides a unified DSL format and MCP interface, enabling Agents to generate, execute, and manage test cases, achieving an automated "code-test-fix" workflow.

[![kznr02/OpenTester MCP server](https://glama.ai/mcp/servers/kznr02/open-tester/badges/card.svg)](https://glama.ai/mcp/servers/kznr02/open-tester)

## Core Positioning

```
┌─────────────────────────────────────────┐
│  AI Agent (Claude Code / Cursor / ...)  │
│  ├─ Generate DSL test cases             │
│  ├─ Decide testing strategies           │
│  └─ Analyze failure reasons             │
├─────────────────────────────────────────┤
│  OpenTester (MCP Server)                │
│  ├─ Validate DSL syntax                 │
│  ├─ Execute tests (CLI/Web)             │
│  ├─ Store cases/projects                │
│  └─ Return structured results           │
├─────────────────────────────────────────┤
│  Web UI (Auxiliary Observation Panel)   │
│  ├─ View execution progress             │
│  ├─ Debug cases (create/edit)           │
│  └─ View history reports                │
└─────────────────────────────────────────┘
```

**Design Principles**:
- **Agent Intelligence**: Test generation and failure analysis are handled by the Agent
- **OpenTester Execution**: Focuses on DSL validation and test execution
- **MCP-First**: All core features exposed through MCP
- **Web UI Auxiliary**: Visual monitoring and debugging, not required

## Installation

### Option 1: Install from PyPI (Recommended)

```bash
pip install opentester

# Or with uv
uv pip install opentester
```

### Option 2: Install from Source

```bash
# Clone repository
git clone https://github.com/kznr02/OpenTester.git
cd OpenTester

# Install backend
uv pip install ./backend

# Install frontend dependencies (optional)
cd frontend
npm install
```

### Option 3: Direct Run (Development)

```bash
cd backend
uv run opentester start
```

## Quick Start

### Start Services

```bash
# Start both FastAPI + MCP (foreground mode with prefixed logs)
opentester start

# Daemon mode (detached background with log files)
opentester start --daemon

# Start only API
opentester start --api

# Start only MCP
opentester start --mcp

# Custom ports
opentester start --api-port 8080 --mcp-port 8081

# Check status
opentester status

# Stop services
opentester stop

# Environment check
opentester doctor
```

**Foreground Mode Log Output:**
```
[API] INFO:     Started server process [12345]
[API] INFO:     Waiting for application startup.
[MCP] INFO:     Started MCP server on port 8001
[API] INFO:     Application startup complete.
```

### Configure Claude Code (Streamable HTTP)

Add MCP configuration in `.claude/settings.json`:
```json
{
  "mcpServers": {
    "opentester": {
      "url": "http://localhost:8001/mcp"
    }
  }
}
```

> **Note**: Current runtime transport is Streamable HTTP.

### Usage Example

Conversation with Claude Code:

```
You: Help me test the login functionality

Claude: I'll create tests for the login feature...
        [Generate DSL test case]
        [Call MCP: validateDSL] ✓ Validation passed
        [Call MCP: createProject] ✓ Project created
        [Call MCP: saveCase] ✓ Case saved
        [Call MCP: runCase]
        Executing...
        ✓ All passed
```

## Project Structure

```
OpenTester/
├── backend/              # FastAPI + Python
│   ├── opentester/
│   │   ├── api/         # REST API (for Web UI)
│   │   ├── core/        # Execution engine, storage
│   │   ├── models/      # Pydantic models (DSL, Project)
│   │   └── mcp/         # MCP Server (core)
│   └── pyproject.toml
├── frontend/            # React + TypeScript + Vite
│   └── src/             # Web UI (observation panel)
├── docs/
│   ├── SKILL_PROMPT.md  # Agent Skill Prompt template
│   ├── DSL_SPEC.md      # DSL syntax specification
│   └── MCP.md           # MCP interface documentation
├── README.md            # This document
└── LICENSE              # MIT License
```

## Core Features

### 1. DSL Validation

After Agent generates DSL, OpenTester validates syntax correctness:

```yaml
version: "1.0"
meta:
  name: "Login Test"
steps:
  - action: exec
    command: "curl http://localhost:3000/login"
  - action: assert
    assertion:
      type: stdout_contains
      expected: "token"
```

### 2. Test Execution

Supports execution targets:
- **CLI**: subprocess command execution (implemented)
- **WEB**: Playwright-based browser automation (implemented)
- **GUI**: experimental executor routing (disabled by default)
- **TUI**: experimental executor routing (disabled by default)

### 3. Web Testing MVP + AI DOM Analysis

- Web actions are executed by `WebExecutor` (Playwright)
- Browser console, page errors, and failed requests are normalized into `diagnostic_events` for each execution step
- Supports AI-assisted locator flow via `ai_locator` in DSL steps
- Execution can pause in `paused_waiting_for_ai` status and wait for AI selector submission
- Supports DOM snapshot + optional screenshot capture for AI analysis
- Web UI and REST clients can retrieve persisted browser diagnostics from the execution diagnostics endpoints for post-run investigation

### 3. Project Management

- Projects stored in XDG data directory (`~/.local/share/opentester/` on Linux, `~/Library/Application Support/opentester/` on macOS, `%LOCALAPPDATA%\opentester\` on Windows)
- PRD content persisted with projects (provided by Agent)
- Test case version management
- Template library for reusable DSL patterns

### 4. Real-time Monitoring

- WebSocket real-time execution progress push
- Web UI visual display
- Execution history traceability

## MCP Tools

| Tool | Description |
|------|-------------|
| `list_projects` | List all test projects |
| `get_project` | Get project details (including cases) |
| `create_project` | Create project |
| `delete_project` | Delete project |
| `validate_dsl` | Validate DSL syntax |
| `save_case` | Save test case |
| `delete_case` | Delete test case |
| `run_case` | Execute single case |
| `run_project` | Execute project cases |
| `stop_execution` | Stop execution |
| `get_execution_status` | Get execution status |
| `get_execution_log` | Get detailed logs |
| `request_dom_analysis` | Get DOM snapshot for paused AI step |
| `submit_ai_selector` | Submit selector to resume paused execution |
| `list_paused_executions` | List executions waiting for AI analysis |
| `list_templates` | List templates |
| `create_template` | Create DSL template |
| `instantiate_template` | Create case from template |

See [MCP Interface Documentation](docs/MCP.md) for details.

## Documentation

See [documentation](docs/) for detailed guides:
- [Architecture](docs/ARCHITECTURE.md) - System architecture and design decisions
- [MCP Interface](docs/MCP.md) - MCP tool specifications
- [DSL Specification](docs/DSL_SPEC.md) - YAML-based test definition language
- [Development Guide](docs/DEVELOPMENT.md) - Development setup and contribution

Agents using OpenTester need to include DSL generation specifications. Refer to [SKILL_PROMPT.md](docs/SKILL_PROMPT.md)

## DSL Specification

YAML-based test definition language. See [DSL_SPEC.md](docs/DSL_SPEC.md) for details.

## Architecture

See [ARCHITECTURE.md](docs/ARCHITECTURE.md) for system architecture, design principles, and architectural decisions.

## Development Guide

Developers refer to [DEVELOPMENT.md](docs/DEVELOPMENT.md)

## Web UI

Auxiliary features:
- View project list and details
- Edit DSL cases (Monaco editor)
- Monitor execution progress
- View history reports

**Note**: Web UI is not the main entry point. All core features are provided through MCP.

## Ports

- FastAPI (Web UI / REST API): http://localhost:8000
- MCP Server: http://localhost:8001/mcp
- API Docs: http://localhost:8000/docs
- Web UI Dev Server: http://localhost:5173

## Data Storage

OpenTester follows the XDG Base Directory Specification:

- **Projects**: `<XDG_DATA_HOME>/opentester/projects/{project_id}.json`
  - Linux: `~/.local/share/opentester/projects/`
  - macOS: `~/Library/Application Support/opentester/projects/`
  - Windows: `%LOCALAPPDATA%\opentester\projects\`
- **Executions**: `<XDG_DATA_HOME>/opentester/executions/`
- **Templates**: `<XDG_DATA_HOME>/opentester/templates/`
- **Logs**: `<XDG_DATA_HOME>/opentester/logs/daemon/` (daemon mode service logs)
- **Config**: `~/.config/opentester/` (or `$XDG_CONFIG_HOME/opentester/`)


## Distribution

Refer to [DISTRIBUTION.md](docs/DISTRIBUTION.md) for:
- PyPI publishing
- PyInstaller packaging
- Docker images
- System package managers

Quick build executable:
```bash
cd backend
pip install pyinstaller
pyinstaller opentester.spec
# Output: dist/opentester.exe (Windows) or dist/opentester (Linux/Mac)
```

## Contributing

We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

## Documentation

- [English Documentation](docs/)
- [中文文档](docs/zh/)

## License

[MIT License](LICENSE)

---

**OpenTester** - MCP-First Testing Execution Platform