# CrowdSentinel MCP Server ### AI-Powered Threat Hunting & Incident Response Framework [![PyPI](https://img.shields.io/pypi/v/crowdsentinel-mcp-server.svg)](https://pypi.org/project/crowdsentinel-mcp-server/) [![PyPI Downloads](https://img.shields.io/pypi/dm/crowdsentinel-mcp-server.svg)](https://pypi.org/project/crowdsentinel-mcp-server/) [![Python](https://img.shields.io/pypi/pyversions/crowdsentinel-mcp-server?cacheSeconds=3600)](https://pypi.org/project/crowdsentinel-mcp-server/) [![License](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](LICENSE) [![MCP](https://img.shields.io/badge/MCP-Compatible-purple.svg)](https://modelcontextprotocol.io/) [![Tools](https://img.shields.io/badge/MCP%20Tools-139-brightgreen.svg)](https://github.com/thomasxm/CrowdSentinels-AI-MCP) [![Rules](https://img.shields.io/badge/Detection%20Rules-6%2C060-blue.svg)](https://github.com/thomasxm/CrowdSentinels-AI-MCP) [![Install from PyPI](https://img.shields.io/badge/Install-pip%20install%20crowdsentinel--mcp--server-3775A9?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/crowdsentinel-mcp-server/) [MCP Official Registry](https://registry.modelcontextprotocol.io/v0/servers?search=io.github.thomasxm/crowdsentinel-mcp-server) · [PyPI Package](https://pypi.org/project/crowdsentinel-mcp-server/) **Open-source threat hunting orchestrator connecting LLMs to enterprise security data via Model Context Protocol (MCP)** [Quick Start](#quick-start) · [Installation](#installation) · [CLI Usage](#cli-usage) · [Features](#key-features) · [Architecture](#architecture) · [Documentation](#documentation) · [Examples](#usage-examples)

> **Warning** > This project is in active development and intended for **security testing, research, and educational purposes only**. It is not production-ready. Do not deploy in production environments. APIs, tool interfaces, and data formats may change without notice. Use at your own risk. --- ## Demo https://github.com/user-attachments/assets/0d0381f0-5b68-43b2-8630-19ec130885b2 --- ## What is CrowdSentinel? **CrowdSentinel** transforms traditional SIEM querying into intelligent, framework-driven investigations using natural language. It serves as a unified security intelligence layer that connects large language models to enterprise security data sources, enabling: - **Natural Language Threat Hunting** — Query Elasticsearch using plain English - **AI-Guided Investigation Workflows** — Built-in prompts guide agents through proper IR methodology - **Persistent Investigation State** — Memory-managed IoC tracking, forensic timelines, and cross-query correlation that survives across sessions (8GB FIFO storage) - **Cross-Tool IoC Correlation** — IoCs discovered in one tool are automatically available to all others - **Multi-Source Analysis** — Elasticsearch, EVTX logs (Chainsaw), PCAP files (Wireshark), live endpoint forensics (Velociraptor), local host forensics (osquery) - **Velociraptor Endpoint Forensics** — 25 MCP tools for live artefact collection (processes, network, persistence, execution evidence, NTFS MFT, SRUM, remote EVTX) with automatic IoC extraction - **Encoded-Command Deobfuscation** — Automatically peels base64/UTF-16LE/gzip/hex layers off encoded commands (e.g. PowerShell `-EncodedCommand`) so C2 addresses hidden inside them surface as IoCs - **Adaptive Schema Intelligence** — Data-driven schema registry (ECS, Sysmon, Windows Security, auditd, Zeek, Packetbeat, AWS CloudTrail) adapts hunts to each index's field conventions - **DFIR Knowledge Resources** — 9 MCP resources exposing investigation playbooks, Pyramid of Pain reference, and cross-correlation guidance directly to connected AI agents - **Standalone CLI** — Full threat hunting from the terminal without an MCP client --- ## Installation ### Install from PyPI (recommended) ```bash # Install with pip pip install crowdsentinel-mcp-server # Or install with uv uv pip install crowdsentinel-mcp-server # Download detection rules, Chainsaw, and Sigma rules (one-time) crowdsentinel setup # Validate connector configuration and readiness (offline, no external calls) crowdsentinel doctor ``` **Optional extras:** ```bash # Velociraptor live endpoint forensics (adds 25 MCP tools) pip install 'crowdsentinel-mcp-server[velociraptor]' # Long-tail deobfuscation coverage via chepy (XOR, exotic encodings). # The stdlib decoder core works without this — chepy only extends it. pip install 'crowdsentinel-mcp-server[deobf]' ``` Detection rules (6,060 Lucene + EQL + ES|QL) are **bundled with the package** — no download needed. The `setup` command downloads additional tools: - Chainsaw binary for EVTX analysis - 3,000+ Sigma rules for Chainsaw Downloaded tools are stored in `~/.crowdsentinel/` and persist across package upgrades. **System dependency for PCAP analysis:** ```bash # Required for network traffic analysis and cross-tool IoC correlation sudo apt install tshark # Debian/Ubuntu/Kali sudo dnf install wireshark-cli # Fedora/RHEL brew install wireshark # macOS ``` ### Run directly with uvx (no install needed) ```bash # Elasticsearch 8.x (default) uvx crowdsentinel-mcp-server # Other backends uvx crowdsentinel-mcp-server-es7 # Elasticsearch 7.x uvx crowdsentinel-mcp-server-es9 # Elasticsearch 9.x uvx opensearch-mcp-server # OpenSearch 1.x/2.x/3.x ``` ### Install from source ```bash git clone https://github.com/thomasxm/CrowdSentinels-AI-MCP.git cd CrowdSentinels-AI-MCP chmod +x setup.sh && ./setup.sh # Or install everything, including all optional extras (deobf/chepy, velociraptor) ./setup.sh --full ``` The setup script will: - Install dependencies (pipx, uv, Claude Code CLI if needed) - Bundle 6,060 detection rules and download Chainsaw binary - Prompt for Elasticsearch credentials (never hardcoded) - Configure the MCP server with Claude Code - Validate your connection With `--full`, every optional extra is installed too, so all 139 MCP tools have their dependencies available — the recommended mode for a production-ready server. ### Installed Size CrowdSentinel bundles 6,060 detection rules and integrates with external analysis tools. Below is the full disk space breakdown so you can plan accordingly. **Core package (via `pip` or `uvx`):** | Component | Size | Notes | |---|---|---| | **CrowdSentinel package** | **49 MB** | The server itself | | — Bundled Sigma rules (`src/rules/`) | 30 MB | 6,060 pre-converted detection rules | | — Elastic TOML rules (`src/detection-rules/`) | 17 MB | Original TOML format rules + hunting queries | | — Python code (clients, tools, etc.) | 2 MB | Actual application code | | **Dependencies** | **64 MB** | All transitive deps | | — `cryptography` | 14 MB | Largest dependency (TLS) | | — `elasticsearch` | 8.3 MB | ES Python client | | — `pygments` | 5.2 MB | Syntax highlighting | | — `pydantic_core` | 5 MB | Validation engine | | — `opensearchpy` | 3.6 MB | OpenSearch client | | — Others (27 packages) | ~28 MB | mcp, fastmcp, httpx, anthropic, etc. | | **Core total** | **113 MB** | `pip install crowdsentinel-mcp-server` | **Additional tools (via `crowdsentinel setup`):** | Component | Download | Installed | Notes | |---|---|---|---| | Chainsaw binary (v2.13.1) | ~3 MB | ~15 MB | EVTX log analysis engine | | Sigma rules (SigmaHQ) | ~3 MB | ~30 MB | 3,000+ Sigma rules for Chainsaw | | Chainsaw mappings | — | <1 MB | Event log source mappings | | **Setup total** | **~6 MB** | **~46 MB** | Stored in `~/.crowdsentinel/` | **System dependency (via package manager):** | Component | Installed | Install Command | Notes | |---|---|---|---| | tshark + Wireshark libs | ~132 MB | `sudo apt install tshark` | PCAP network analysis — required for cross-tool IoC correlation | **Full installation summary:** | Scenario | Total Disk Space | |---|---| | Core only (`pip install`) | ~113 MB | | Core + setup (`crowdsentinel setup`) | ~159 MB | | Full platform (+ tshark) | **~291 MB** | > **Note:** PyPI download size is only **8.9 MB** (wheel) thanks to compression of the bundled detection rules. --- ## Quick Start ### 1. Set environment variables ```bash export ELASTICSEARCH_HOSTS="https://localhost:9200" export ELASTICSEARCH_API_KEY="your_api_key" # Or use username/password: # export ELASTICSEARCH_USERNAME="elastic" # export ELASTICSEARCH_PASSWORD="your_password" export VERIFY_CERTS="false" ``` ### 2. Connect to an MCP Client CrowdSentinel works with any MCP-compatible AI agent. Choose your client below:

Claude Code (CLI)

```bash claude mcp add crowdsentinel \ -e ELASTICSEARCH_HOSTS="https://localhost:9200" \ -e ELASTICSEARCH_API_KEY="your_api_key" \ -e VERIFY_CERTS="false" \ -- uvx crowdsentinel-mcp-server ```

Claude Desktop

Edit `~/.config/Claude/claude_desktop_config.json` (Linux) or `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS): ```json { "mcpServers": { "crowdsentinel": { "command": "uvx", "args": ["crowdsentinel-mcp-server"], "env": { "ELASTICSEARCH_HOSTS": "https://localhost:9200", "ELASTICSEARCH_API_KEY": "your_api_key", "VERIFY_CERTS": "false" } } } } ```

VS Code Copilot

Create `.vscode/mcp.json` in your workspace: ```json { "servers": { "crowdsentinel": { "command": "uvx", "args": ["crowdsentinel-mcp-server"], "env": { "ELASTICSEARCH_HOSTS": "https://localhost:9200", "ELASTICSEARCH_API_KEY": "your_api_key", "VERIFY_CERTS": "false" } } } } ``` Then enable MCP in VS Code settings: `"chat.mcp.enabled": true`

Cursor

Create or edit `~/.cursor/mcp.json`: ```json { "mcpServers": { "crowdsentinel": { "command": "uvx", "args": ["crowdsentinel-mcp-server"], "env": { "ELASTICSEARCH_HOSTS": "https://localhost:9200", "ELASTICSEARCH_API_KEY": "your_api_key", "VERIFY_CERTS": "false" } } } } ```

Roo Code (VS Code Extension)

Create `.roo/mcp.json` in your workspace: ```json { "mcpServers": { "crowdsentinel": { "command": "uvx", "args": ["crowdsentinel-mcp-server"], "env": { "ELASTICSEARCH_HOSTS": "https://localhost:9200", "ELASTICSEARCH_API_KEY": "your_api_key", "VERIFY_CERTS": "false" } } } } ``` Or configure via Roo Code settings panel: Settings > MCP Servers > Add Server.

5ire

In 5ire settings (v0.15.0+), add an MCP server with: - **Command**: `uvx` - **Arguments**: `crowdsentinel-mcp-server` - **Environment Variables**: - `ELASTICSEARCH_HOSTS` = `https://localhost:9200` - `ELASTICSEARCH_API_KEY` = `your_api_key` - `VERIFY_CERTS` = `false` > **Note:** 5ire v0.14.0 has known MCP compatibility issues. Use v0.15.0+ for reliable operation.

Any MCP Client (Generic)

**stdio transport** (default — works with most clients): ```json { "mcpServers": { "crowdsentinel": { "command": "uvx", "args": ["crowdsentinel-mcp-server"], "env": { "ELASTICSEARCH_HOSTS": "https://localhost:9200", "ELASTICSEARCH_API_KEY": "your_api_key", "VERIFY_CERTS": "false" } } } } ``` **SSE transport** (for web-based clients): ```bash crowdsentinel-mcp-server --transport sse --port 8001 # Connect to: http://localhost:8001/sse/ ``` **HTTP transport** (for REST API clients): ```bash crowdsentinel-mcp-server --transport streamable-http --port 8001 # Connect to: http://localhost:8001/mcp/ ```

### 3. Or use the CLI directly ```bash # Download rules and tools (one-time) crowdsentinel setup # Check cluster health crowdsentinel health # Hunt for threats crowdsentinel hunt "powershell encoded" -i winlogbeat-* # Run detection rules crowdsentinel rules -p windows --tactic credential_access crowdsentinel detect windows_builtin_win_alert_mimikatz_keywords_lucene -i winlogbeat-* # Analyse PCAP files crowdsentinel pcap overview capture.pcap crowdsentinel pcap beaconing capture.pcap # Hunt EVTX logs with Chainsaw crowdsentinel chainsaw hunt /path/to/evtx/ --sigma-rules /path/to/sigma/ ``` --- ## CLI Usage CrowdSentinel provides a full CLI for threat hunting from the terminal: ```bash pip install crowdsentinel-mcp-server crowdsentinel setup # Download rules, Chainsaw, Sigma (one-time) crowdsentinel --help ``` ### Available Commands | Command | Description | Example | |:--------|:------------|:--------| | `setup` | Download detection rules, Chainsaw, and Sigma rules | `crowdsentinel setup` | | `health` | Show cluster health | `crowdsentinel health` | | `indices` | List all indices | `crowdsentinel indices` | | `hunt` | IR-focused threat hunt with IoC extraction | `crowdsentinel hunt "powershell" -i winlogbeat-*` | | `eql` | Execute an EQL query | `crowdsentinel eql "process where process.name == 'cmd.exe'" -i winlogbeat-*` | | `esql` | Execute an ES\|QL query | `crowdsentinel esql "FROM logs-* \| LIMIT 10"` | | `detect` | Execute a detection rule by ID | `crowdsentinel detect win_susp_logon -i winlogbeat-*` | | `rules` | List available detection rules | `crowdsentinel rules -p windows --tactic credential_access --type eql` | | `schema` | Detect schema for an index pattern | `crowdsentinel schema -i winlogbeat-*` | | `ioc` | Hunt for a specific Indicator of Compromise | `crowdsentinel ioc 203.0.113.42 --type ip -i winlogbeat-*` | | `analyse` | Analyse search results from stdin (JSON) | `cat results.json \| crowdsentinel analyse -c "context"` | | `analyse --mcp` | AI agent analysis using all 139 MCP tools | `crowdsentinel hunt "query" \| crowdsentinel analyse --mcp -c "context"` | | `auth` | Manage LLM authentication for agent mode | `crowdsentinel auth login` | | `pcap` | Analyse PCAP files (overview, beaconing, lateral movement) | `crowdsentinel pcap beaconing capture.pcap` | | `chainsaw` | Hunt EVTX logs with Chainsaw and Sigma rules | `crowdsentinel chainsaw hunt /path/to/evtx/` | | `doctor` | Validate connector configuration and readiness (offline — no external calls) | `crowdsentinel doctor` | ### Output Formats All commands support `--output/-o` with three formats: ```bash crowdsentinel hunt "failed login" -i winlogbeat-* -o json # Structured JSON (default) crowdsentinel hunt "failed login" -i winlogbeat-* -o table # Human-readable table crowdsentinel hunt "failed login" -i winlogbeat-* -o summary # Condensed summary ``` ### Agent Mode (`--mcp`) The `analyse --mcp` flag replaces deterministic analysis with an AI agent that autonomously uses all 139 MCP tools to investigate. The agent follows the 4-phase IR methodology: hunt, analyse, correlate, report. **Authentication:** ```bash # Option 1: Browser sign-in (ChatGPT subscription — no API billing) crowdsentinel auth login # Option 2: Anthropic (setup-token or API key) crowdsentinel auth login --provider anthropic # Option 3: Environment variable export ANTHROPIC_API_KEY="sk-ant-..." # or OPENAI_API_KEY # Option 4: Local models (Ollama, vLLM — free) crowdsentinel analyse --mcp --model-url http://localhost:11434/v1 --model llama3.1 # Check auth status crowdsentinel auth status ``` **Agent flags:** | Flag | Default | Description | |:-----|:--------|:------------| | `--mcp` | off | Enable AI agent with MCP tools | | `--mcp-server NAME:CMD` | none | Add external MCP server (e.g., VirusTotal) | | `--model` | auto-detect | LLM model to use | | `--model-url` | none | OpenAI-compatible API endpoint | | `--max-steps` | 30 | Maximum tool calls | | `--timeout` | 300 | Maximum seconds | ### Pipeline Examples **Deterministic analysis (no API key needed):** ```bash # Hunt then analyse crowdsentinel hunt "powershell encoded" -i winlogbeat-* -o json | \ crowdsentinel analyse -c "Encoded PowerShell commands" -o summary # Investigate failed authentication attempts crowdsentinel hunt "event.code:4625" -i winlogbeat-* -o json | \ crowdsentinel analyse -c "Failed login brute force investigation" -o summary # Triage process execution and privilege escalation crowdsentinel hunt "event.code:4688 OR event.code:4672 OR event.code:1" -i winlogbeat-* -o json | \ crowdsentinel analyse -c "Process execution and privilege escalation" -o summary ``` **AI agent investigation (requires auth):** ```bash # Credential dumping investigation — agent hunts, analyses kill chain, checks adjacent stages crowdsentinel hunt "mimikatz OR lsass OR procdump" -i winlogbeat-* -o json | \ crowdsentinel analyse --mcp -c "Credential dumping tools investigation" --max-steps 15 -o summary # Encoded PowerShell — full IR workflow with kill chain and adjacent stage hunting crowdsentinel hunt "powershell -enc OR FromBase64String" -i winlogbeat-* -o json | \ crowdsentinel analyse --mcp -c "Full IR workflow: encoded PowerShell" --max-steps 30 -o table # Process execution with detection rules crowdsentinel hunt "event.code:4688" -i winlogbeat-* -o json | \ crowdsentinel analyse --mcp -c "Execute detection rules against process creation" --max-steps 20 -o summary # PCAP beaconing — agent generates IoCs and maps to kill chain crowdsentinel pcap beaconing capture.pcap -o json | \ crowdsentinel analyse --mcp -c "Investigate beaconing for C2 infrastructure" --max-steps 10 -o summary # Anti-forensics investigation crowdsentinel hunt "event.code:1102" -i winlogbeat-* -o json | \ crowdsentinel analyse --mcp -c "Security log cleared - anti-forensics" --max-steps 10 -o summary # With external MCP server (e.g., VirusTotal) crowdsentinel hunt "powershell" -i winlogbeat-* -o json | \ crowdsentinel analyse --mcp --mcp-server "vt:uvx virustotal-mcp-server" \ -c "Check IoCs against VirusTotal" -o summary ``` --- ## Key Features

### 139 MCP Tools Threat hunting, detection rules, forensics, endpoint collection, network analysis, cross-correlation, command deobfuscation, and IoC enrichment — all accessible via natural language ### 6,060 Detection Rules Pre-built Lucene, EQL & ES|QL rules with automatic MITRE ATT&CK mapping ### Investigation State Persistent IoC tracking across tools and sessions with cross-source correlation and FIFO storage

### 4 Security Frameworks - Cyber Kill Chain (7 stages) - Pyramid of Pain (6 levels) - Diamond Model (4 vertices) - MITRE ATT&CK (automatic mapping) ### 5 Data Sources + Threat Intel - Elasticsearch / OpenSearch (SIEM) - Velociraptor (live endpoint forensics) - osquery (local host forensics) - EVTX logs (Chainsaw + Sigma) - PCAP files (Wireshark/TShark) - IoC enrichment (Shodan, VirusTotal, AbuseIPDB, ThreatFox) - MISP (export events, search attributes)

--- ## Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ LLM Client / Claude Code CLI │ └─────────────────────────────┬───────────────────────────────────┘ │ MCP Protocol (stdio/SSE/HTTP) │ HTTP: Origin validation (anti DNS-rebinding) │ + transport-level audit logging ▼ ┌─────────────────────────────────────────────────────────────────┐ │ CrowdSentinel MCP Server │ │ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────┐ │ │ │ 139 Tools │ │ 6,060 Rules │ │ Security Frameworks │ │ │ │ - Hunting │ │ - Lucene │ │ - Cyber Kill Chain │ │ │ │ - Detection │ │ - EQL │ │ - Pyramid of Pain │ │ │ │ - Forensics │ │ - ES|QL │ │ - Diamond Model │ │ │ │ - Endpoint │ │ │ │ - MITRE ATT&CK │ │ │ │ - Network │ ├───────────────┤ │ │ │ │ │ - Enrichment │ │ Schema │ │ 9 MCP Resources │ │ │ │ - Correlation │ │ Registry │ │ (DFIR knowledge base) │ │ │ │ - Deobfuscate │ │ (data-driven) │ │ │ │ │ └───────────────┘ └───────────────┘ └───────────────────────┘ │ │ ┌─────────────────────────────────────────────────────────────┐│ │ │ Investigation State (Persistent) ││ │ │ Cross-source IoC sharing, auto-capture, STIX 2.1 export ││ │ └─────────────────────────────────────────────────────────────┘│ └─────┬──────────┬──────────┬──────────────┬─────────────┬────────┘ │ │ │ │ │ ▼ ▼ ▼ ▼ ▼ ┌───────────┐ ┌──────────┐ ┌────────┐ ┌───────────┐ ┌───────────┐ │Elastic- │ │Velocirap-│ │osquery │ │ Chainsaw │ │ Wireshark │ │search/ │ │tor │ │(local │ │(EVTX/ │ │ (PCAP) │ │OpenSearch │ │(EDR/DFIR)│ │ host) │ │ Sigma) │ │ (Network) │ │ (SIEM) │ │(Endpoint)│ │ │ │(Offline) │ │ │ └───────────┘ └──────────┘ └────────┘ └───────────┘ └───────────┘ │ ┌───────────────────────┼───────────────────────┐ ▼ ▼ ▼ ┌───────────────┐ ┌───────────────┐ ┌───────────────────────────┐ │ Shodan │ │ VirusTotal │ │ AbuseIPDB + ThreatFox │ │ InternetDB │ │ (v3) │ │ + MISP (export/search) │ │ (free/no key)│ │ (free tier) │ │ (free tier) │ └───────────────┘ └───────────────┘ └───────────────────────────┘ │ ▼ (Roadmap) ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ Splunk │ │ Wazuh │ │ Zeek │ │ │ │ (EDR/XDR) │ │ (NSM/IDS) │ └───────────────┘ └───────────────┘ └───────────────┘ ``` --- ## What's Included ### Tool Categories (139 Tools) | Category | Tools | Description | |:---------|:-----:|:------------| | **Elasticsearch Core** | 19 | Index, document, cluster, alias, data stream, raw API operations | | **Threat Hunting** | 8 | Attack pattern detection, kill-chain stage hunting, timeframe hunts, anomaly detection | | **EQL & ES\|QL Hunting** | 11 | Event-sequence queries, ES\|QL pipes, saved hunt history, capability detection | | **Asset Discovery & Schema Intelligence** | 12 | Index discovery, data-driven schema detection (ECS, Sysmon, auditd, Zeek, CloudTrail…), smart search | | **IoC Analysis & Deobfuscation** | 6 | IoC extraction, encoded-command decoding, kill-chain mapping, investigation reports | | **Detection Rules** | 9 | 6,060 rule library — list, execute, validate, suggest, batch execution | | **Investigation Prompts** | 5 | Fast triage spine — 10 critical IR questions | | **Chainsaw (EVTX)** | 5 | Sigma rule hunting, iterative IoC discovery | | **Wireshark (PCAP)** | 11 | Network forensics, beaconing, lateral movement detection, traffic decoding | | **Threat Intelligence** | 5 | IoC enrichment (Shodan, VirusTotal, AbuseIPDB, ThreatFox) + MISP export/search | | **Investigation State** | 10 | Persistent IoCs, cross-tool sharing, STIX 2.1 export, reporting | | **Velociraptor (Endpoint)** | 25 | Live forensic artefact collection — processes, network, persistence, execution evidence, MFT, SRUM | | **osquery (Local Host)** | 4 | SQL-based local host forensics — processes, sockets, users, ad-hoc queries | | **Cross-Source Correlation** | 5 | SIEM ↔ endpoint pivots, IoC-driven endpoint hunts, unified timelines, time correlation | | **Workflow Guidance & Status** | 4 | Next-step guidance, progressive disclosure, system status, storage cleanup | > Velociraptor tools register when `VELOCIRAPTOR_API_CONFIG` is set; osquery tools register when a local `osqueryi` binary is present; the 10 high-risk write tools are hidden when `DISABLE_HIGH_RISK_OPERATIONS=true`. ### Security Frameworks | Framework | Purpose | |:----------|:--------| | **Cyber Kill Chain** | Hunt by attack stage (7 stages), predict adversary's next move | | **Pyramid of Pain** | Prioritise IoCs by difficulty for attackers to change (6 levels) | | **Diamond Model** | Map adversary, capability, infrastructure, victim relationships | | **MITRE ATT&CK** | Automatic technique mapping for all detections | ### Detection Rules (6,060 Rules) | Type | Count | Source | Description | |:-----|------:|:-------|:------------| | **Lucene** | 1,966 | Sigma-converted | Fast pattern matching queries | | **EQL** | 3,963 | Sigma-converted + Elastic | Event sequences and correlations | | **ES\|QL** | 131 | Elastic TOML rules | Pipe-based query language (ES 8.11+) | **Platforms:** Windows, Linux, macOS, Cloud (AWS/Azure/GCP), Network, Identity **Log Sources:** PowerShell, Sysmon, Security Events, Process Creation, Audit logs ### Encoded-Command Deobfuscation Real intrusions hide C2 addresses and URLs inside encodings the SIEM stores verbatim — most commonly PowerShell `-EncodedCommand` (base64 of UTF-16LE), but also `FromBase64String`, base64→gzip, bare base64, and hex blobs. CrowdSentinel decodes these layers automatically during IoC extraction and via the dedicated `decode_command` tool, so the inner indicators surface instead of staying buried: - **Zero extra dependencies** — the decoder core uses only the Python standard library (`base64`/`zlib`/`codecs`/`binascii`) - **Optional long tail** — install the `deobf` extra (`pip install 'crowdsentinel-mcp-server[deobf]'`) to add [chepy](https://github.com/securisec/chepy) for XOR and exotic encodings; its absence degrades gracefully - **Provenance** — every decode records its recipe (e.g. `base64->utf-16le`) for the IR report - **Safe by design** — decode-only (nothing is executed), with depth and output-size bounds so a decompression bomb cannot hang the server --- ## Configuration ### Environment Variables ```bash # Connection (required — choose one) ELASTICSEARCH_HOSTS="https://localhost:9200" # Self-hosted # OR ELASTICSEARCH_CLOUD_ID="deployment:base64..." # Elastic Cloud # Authentication — choose one (in priority order): ELASTICSEARCH_BEARER_TOKEN="service_token_here" # Service/bearer token ELASTICSEARCH_API_KEY="your_api_key" # API key (recommended) ELASTICSEARCH_USERNAME="elastic" # Basic auth ELASTICSEARCH_PASSWORD="your_password" # TLS / Certificate verification VERIFY_CERTS="true" # Verify against system CA bundle # VERIFY_CERTS="/path/to/ca.crt" # Verify against custom CA certificate # ELASTICSEARCH_CA_CERT="/path/to/ca.crt" # Explicit CA certificate path # ELASTICSEARCH_CLIENT_CERT="/path/to/client.crt" # Client certificate (mTLS) # ELASTICSEARCH_CLIENT_KEY="/path/to/client.key" # Client private key (mTLS) # Options REQUEST_TIMEOUT="30" # Request timeout in seconds DISABLE_HIGH_RISK_OPERATIONS="true" # Block all write operations # HTTP/SSE transport security (stdio unaffected) # MCP_ALLOWED_ORIGINS="https://app.example.com" # Extra allowed Origins (comma-separated); loopback always allowed # Velociraptor endpoint forensics (optional — enables 25 extra tools) # VELOCIRAPTOR_API_CONFIG="/path/to/api_client.yaml" # Threat Intelligence (optional — Shodan InternetDB works without any key) VIRUSTOTAL_API_KEY="your_vt_key" # Free: 500 lookups/day ABUSEIPDB_API_KEY="your_abuse_key" # Free: 1,000 lookups/day THREATFOX_API_KEY="your_tf_key" # Free: unlimited # MISP Integration (optional — offline JSON export works without a server) MISP_URL="https://misp.example.org" # MISP instance URL MISP_API_KEY="your_misp_key" # MISP API key (40-char hex) # MISP_SSL_VERIFY="true" # Set "false" for self-signed (dev only) ``` > **Security Warning:** Never use `VERIFY_CERTS="false"` or plain-text passwords in production. Use API keys or service tokens with TLS certificate verification enabled. For self-signed certificates, set `ELASTICSEARCH_CA_CERT` to your CA certificate path. ### Production Configuration Examples

Elastic Cloud

```bash ELASTICSEARCH_CLOUD_ID="my-deployment:dXMtY2VudHJhbC0x..." ELASTICSEARCH_API_KEY="your_cloud_api_key" VERIFY_CERTS="true" ```

Self-Hosted with Custom CA

```bash ELASTICSEARCH_HOSTS="https://es-cluster.internal:9200" ELASTICSEARCH_API_KEY="your_api_key" ELASTICSEARCH_CA_CERT="/etc/elasticsearch/certs/ca.crt" VERIFY_CERTS="true" ```

Mutual TLS (mTLS)

```bash ELASTICSEARCH_HOSTS="https://es-cluster.internal:9200" ELASTICSEARCH_CA_CERT="/etc/elasticsearch/certs/ca.crt" ELASTICSEARCH_CLIENT_CERT="/etc/elasticsearch/certs/client.crt" ELASTICSEARCH_CLIENT_KEY="/etc/elasticsearch/certs/client.key" VERIFY_CERTS="true" ```

Development / Testing (insecure)

```bash ELASTICSEARCH_HOSTS="http://localhost:9200" ELASTICSEARCH_USERNAME="elastic" ELASTICSEARCH_PASSWORD="test123" VERIFY_CERTS="false" ``` > This configuration is **not suitable for production**. Use API keys with TLS in production environments.

### Transport Modes | Mode | Use Case | Command | |:-----|:---------|:--------| | **stdio** (default) | MCP clients (Claude Code, etc.) | Auto-configured by setup.sh | | **SSE** | Web/streaming clients | `--transport sse --port 8000` | | **HTTP** | REST API access | `--transport streamable-http` | The HTTP transports are hardened at the transport layer: - **Origin validation (anti DNS-rebinding)** — browser requests are rejected unless their `Origin` is loopback or listed in `MCP_ALLOWED_ORIGINS` (comma-separated; `*` alone disables the check). Requests without an `Origin` header — every non-browser MCP client — are unaffected. - **Audit logging** — every HTTP exchange is logged at the transport boundary (JSON-RPC method, tool name, body size, path), including requests the tool layer never sees, such as unknown tools or protocol errors. Logs are metadata-only, memory-bounded, and sanitised against log forgery. ### Disabling Write Operations For read-only access (recommended for production): ```bash DISABLE_HIGH_RISK_OPERATIONS="true" ``` This hides destructive tools: `create_index`, `delete_index`, `index_document`, `delete_document`, `delete_by_query`, `create_data_stream`, `delete_data_stream`, `put_alias`, `delete_alias`, `general_api_request` --- ## Usage Examples ### Example 1: Hunt for Brute Force Attacks ``` You: Hunt for brute force attacks in the last hour CrowdSentinel: Searching for failed authentication attempts... [Calls hunt_by_timeframe with attack_types=["brute_force"]] Found 47 failed login attempts from 3 source IPs targeting the "admin" account. Top sources: - 192.168.1.105: 28 attempts - 10.0.0.42: 15 attempts - 172.16.0.8: 4 attempts ``` ### Example 2: Investigate a Suspicious Host ``` You: Show me all activity on host DESKTOP-ABC123 in the last 24 hours CrowdSentinel: Generating a forensic timeline for that host... [Calls get_host_activity_timeline] Timeline shows: - 09:15: User "jsmith" logged in (Event 4624) - 09:17: PowerShell executed encoded command - 09:18: New scheduled task created "WindowsUpdate" - 09:22: Outbound connection to 203.0.113.42:443 - 09:25: LSASS memory access detected ``` ### Example 3: Kill Chain Analysis ``` You: We found C2 beaconing. What should we hunt for next? CrowdSentinel: Hunting adjacent kill chain stages... [Calls hunt_adjacent_stages with current_stage="COMMAND_AND_CONTROL"] Looking backwards (Installation stage): - Found scheduled task persistence on 3 hosts - Registry Run key modifications detected Looking forwards (Actions on Objectives): - Lateral movement to 2 additional hosts via SMB - Credential dumping attempts detected ``` ### Example 4: Network Traffic Analysis ``` You: Analyse this PCAP for C2 beaconing CrowdSentinel: Checking for regular connection intervals... [Calls detect_beaconing on the PCAP file] Detected beaconing pattern: - Host: 192.168.1.50 → 203.0.113.100:443 - Interval: 60 seconds (±2s jitter) - Duration: 4 hours - Confidence: HIGH Recommend blocking 203.0.113.100 and investigating 192.168.1.50. ``` ### Example 5: CLI Threat Hunt ```bash # Hunt for encoded PowerShell crowdsentinel hunt "powershell -enc" -i winlogbeat-* --timeframe 1440 -o json # Pipe results to analysis crowdsentinel hunt "event.code:4625" -i winlogbeat-* -o json | \ crowdsentinel analyse -c "Failed authentication investigation" # Search detection rules for lateral movement crowdsentinel rules --tactic lateral_movement -p windows ``` --- ## Compatibility | Package | Backend | Install | |:--------|:--------|:--------| | `crowdsentinel-mcp-server` | Elasticsearch 8.x (default) | `pip install crowdsentinel-mcp-server` | | `crowdsentinel-mcp-server-es7` | Elasticsearch 7.x | `pip install crowdsentinel-mcp-server-es7` | | `crowdsentinel-mcp-server-es9` | Elasticsearch 9.x | `pip install crowdsentinel-mcp-server-es9` | | `opensearch-mcp-server` | OpenSearch 1.x, 2.x, 3.x | `pip install opensearch-mcp-server` | --- ## For Developers

Project Structure

``` crowdsentinel-mcp-server/ ├── src/ │ ├── server.py # MCP server entry point │ ├── version.py # Version constant │ ├── risk_config.py # Write operation controls │ ├── transport_security.py # Origin validation for HTTP transports (anti DNS-rebinding) │ ├── transport_audit.py # Transport-level audit logging (every HTTP exchange) │ │ │ ├── cli/ # Standalone CLI │ │ └── main.py # CLI entry point (argparse) — 15 commands incl. doctor │ │ │ ├── agent/ # `analyse --mcp` AI agent mode (multi-provider LLM auth) │ ├── config/ # Connector configuration │ │ │ ├── clients/ # Backend logic layer │ │ ├── base.py # Base client, authentication │ │ ├── exceptions.py # Exception handling decorators │ │ └── common/ │ │ ├── client.py # Unified SearchClient (multiple inheritance) │ │ ├── threat_hunting.py # Threat hunting queries │ │ ├── ioc_analysis.py # IoC extraction & analysis │ │ ├── deobfuscate.py # Encoded-command decoding (stdlib core + optional chepy) │ │ ├── cyber_kill_chain.py # Kill chain logic │ │ ├── rule_loader.py # Detection rule loading │ │ ├── chainsaw_client.py # EVTX/Sigma integration │ │ ├── osquery_client.py # Local osquery execution (sqlglot-validated SQL) │ │ ├── misp_client.py # MISP export/search │ │ ├── threat_intel.py # Shodan/VirusTotal/AbuseIPDB/ThreatFox enrichment │ │ └── schemas/ # Data-driven schema registry │ │ ├── ecs.py, sysmon.py, windows_security.py, auditd.py │ │ └── defs/ # JSON schema defs (zeek, packetbeat, aws_cloudtrail, …) │ │ │ ├── tools/ # MCP tool interfaces (25 modules, 139 tools) │ │ ├── register.py # Dynamic tool registration │ │ ├── threat_hunting.py # Hunting tool definitions │ │ ├── rule_management.py # Rule management tools │ │ ├── velociraptor_tools.py # Live endpoint forensics (25 tools) │ │ ├── osquery_tools.py # Local host forensics (4 tools) │ │ ├── cross_correlation.py # SIEM ↔ endpoint correlation (5 tools) │ │ ├── chainsaw_hunting.py # Chainsaw tools │ │ ├── wireshark_tools.py # Network analysis tools │ │ ├── threat_intel.py # Enrichment + MISP tools │ │ └── investigation_state_tools.py # State management tools │ │ │ ├── storage/ # Persistent investigation state │ │ ├── investigation_state.py # Core state management │ │ ├── storage_manager.py # File system storage (8GB FIFO) │ │ └── models.py # Pydantic models (IoC, Investigation) │ │ │ ├── utils/ # Shared utilities │ │ │ └── wireshark/ # Network traffic analysis │ ├── core/ # TShark execution, PCAP parsing │ ├── hunting/ # Beaconing, lateral movement, IoC hunting │ ├── baseline/ # Traffic baseline creation │ ├── extraction/ # File carving from traffic │ └── reporting/ # NCSC-style reports, timelines │ ├── rules/ # 6,060 detection rules (EQL + Lucene) ├── chainsaw/ # Chainsaw binary + 3,000+ Sigma rules ├── skills/ # Claude Code agent skills ├── Makefile # Test pyramid + release automation └── tests/ # Tiered test suite (unit / integration / api / e2e / system) ```

Design Patterns

| Pattern | Usage | |:--------|:------| | **Multiple Inheritance** | `SearchClient` composes all specialised clients | | **Decorator** | Exception handling via `@handle_exceptions` | | **Factory** | `create_search_client()` creates appropriate client | | **Plugin Architecture** | Tools registered dynamically via `ToolsRegister` | | **Auto-Capture** | Tool results automatically analysed for IoCs |

Adding a New Tool

1. **Create client method** in `src/clients/common/your_module.py`: ```python class YourClient(SearchClientBase): def your_method(self, param: str) -> dict: # Implementation return results ``` 2. **Add to SearchClient** in `src/clients/common/client.py`: ```python class SearchClient(YourClient, OtherClients, ...): pass ``` 3. **Create tool wrapper** in `src/tools/your_tools.py`: ```python class YourTools: def __init__(self, client, mcp): self.client = client self.mcp = mcp def register_tools(self): @self.mcp.tool() def your_tool(param: str) -> str: """Tool description for LLM.""" result = self.client.your_method(param) return json.dumps(result) ``` 4. **Register in server** in `src/server.py`: ```python from src.tools.your_tools import YourTools def _register_tools(self): # ... existing tools ... YourTools(self.client, self.mcp).register_tools() ```

Running Tests

The test suite is a bottom-up pyramid (see `tests/README.md`): hermetic unit tests, then live tiers that spin up a real Elasticsearch (via [testcontainers](https://testcontainers-python.readthedocs.io/)) and the real MCP server. ```bash make test-unit # fast hermetic loop (default developer command) make coverage # unit tier + coverage policy gate make check-tests # policy: every new function vs origin/main has a test make test-integration # real ES via testcontainers (Docker required) make test-api # + real MCP server subprocess over streamable-http make test-e2e # + real MCP client journey over the wire make test-suite # the whole pyramid, bottom-up, with a summary table ``` | Tier | What it proves | Backing services | |:-----|:---------------|:-----------------| | Unit | Function-level behaviour, fail-fast validation | None (hermetic) | | Integration | App ↔ Elasticsearch trust boundary (auth, TLS, seeded data) | Real ES via testcontainers | | API | MCP-over-HTTP contract (schemas, errors, Origin security, audit logs) | Real ES + real server subprocess | | E2E | Full analyst journey via a real MCP client | Real ES + real server + `fastmcp.Client` | | System | Full pipeline against your own range cluster | BYO Elasticsearch | Live tiers are opt-in (`--run-integration` / `--run-api` / `--run-e2e` / `--run-system`), so plain `pytest tests/` stays fast and hermetic.

Local Testing Environment

```bash # Start Elasticsearch docker-compose -f docker-compose-elasticsearch.yml up -d # Start OpenSearch docker-compose -f docker-compose-opensearch.yml up -d ``` **Default credentials (testing only):** - Elasticsearch: `elastic` / `test123` - OpenSearch: `admin` / `admin`

--- ## Roadmap | Feature | Status | Description | |:--------|:------:|:------------| | **Velociraptor Integration** | **Done** | 25 live endpoint forensics tools + 5 cross-correlation tools via the Velociraptor API | | **osquery Integration** | **Done** | SQL-based local host forensics (processes, sockets, users, ad-hoc queries) | | **Encoded-Command Deobfuscation** | **Done** | Recover IoCs hidden in base64/UTF-16LE/gzip/hex-encoded commands (optional chepy for the long tail) | | **Threat Intel Enrichment** | **Done** | IoC enrichment via Shodan InternetDB, VirusTotal, AbuseIPDB, ThreatFox + STIX 2.1 export | | **MISP Integration** | **Done** | Export investigation IoCs as MISP events, search MISP attributes (offline JSON export without a server) | | **Zeek Integration** | Partial | Zeek logs indexed into ES are hunted via the data-driven schema registry; native NSM integration planned | | **Wazuh Integration** | Planned | Open-source EDR/XDR — its indexer is OpenSearch-based, so CrowdSentinel's existing OpenSearch client can hunt Wazuh alerts and agent telemetry | | **Splunk Integration** | Planned | Add Splunk as a data source alongside Elasticsearch | | **Sigma Rule Converter** | Planned | Convert Sigma rules to native ES/Splunk queries | | **Case Management** | Planned | Export investigations to TheHive, JIRA | | **Custom Rule Builder** | Planned | Create detection rules via natural language | See [CHANGELOG.md](CHANGELOG.md) for detailed version history. --- ## Velociraptor Integration (Live Endpoint Forensics) CrowdSentinel integrates with [Velociraptor](https://docs.velociraptor.app/) for live endpoint forensic artefact collection. When configured, 25 additional MCP tools and 5 cross-correlation tools become available. ### Setup ```bash # Install optional dependencies pip install crowdsentinel-mcp-server[velociraptor] # Set the Velociraptor API config path export VELOCIRAPTOR_API_CONFIG="/path/to/api_client.yaml" ``` The API client config is generated from the Velociraptor server: ```bash velociraptor config api_client --name crowdsentinel --role administrator,api \ --config server.config.yaml /path/to/api_client.yaml ``` ### Available Tools | Category | Tools | Use Case | |:---------|:------|:---------| | **Discovery** | `velociraptor_client_info`, `velociraptor_list_clients`, `velociraptor_list_artifacts` | Resolve hostnames, enumerate enrolled endpoints, discover available artefacts | | **Generic Collection** | `velociraptor_collect_artifact`, `velociraptor_get_collection_results` | Collect any Velociraptor artefact and fetch its results | | **Live State** | `velociraptor_pslist`, `velociraptor_netstat`, `velociraptor_users`, `velociraptor_groups`, `velociraptor_mounts` | Running processes, active network connections, accounts, groups, mounted volumes | | **Execution Evidence** | `velociraptor_prefetch`, `velociraptor_amcache`, `velociraptor_shimcache`, `velociraptor_userassist`, `velociraptor_bam`, `velociraptor_srum` | Forensic proof of programme execution and resource usage | | **Persistence** | `velociraptor_services`, `velociraptor_scheduled_tasks`, `velociraptor_autoruns` | Service, scheduled task, and autorun persistence mechanisms | | **User Activity** | `velociraptor_shellbags`, `velociraptor_recentdocs`, `velociraptor_lnk`, `velociraptor_evidence_of_download` | User browsing, document access, link files, file downloads | | **Filesystem & Logs** | `velociraptor_ntfs_mft`, `velociraptor_evtx` | MFT search by name/path/timestamp, remote EVTX queries | | **Cross-Correlation** | `correlate_siem_with_endpoint`, `endpoint_to_siem_pivot`, `endpoint_hunt_from_ioc`, `correlate_by_time`, `build_unified_timeline` | Validate SIEM findings on endpoints, pivot from endpoint IoCs to fleet-wide SIEM searches, unified timelines | ### Cross-Source IoC Sharing IoCs extracted from Velociraptor artefacts are automatically captured into the active investigation alongside SIEM, Chainsaw, and Wireshark findings. Use `get_shared_iocs` to retrieve the combined indicator set for cross-source correlation. ``` SIEM (Elasticsearch) Endpoint (Velociraptor) │ │ hunt_by_timeframe() velociraptor_pslist() │ │ auto_capture ──────► Investigation State ◄────── auto_capture │ get_shared_iocs() ``` ### DFIR Knowledge Resources 9 MCP resources provide structured investigation reference data directly to connected AI agents: | Resource | Content | |:---------|:--------| | `crowdsentinel://investigation-workflow` | Mandatory 4-phase IR workflow | | `crowdsentinel://ioc-reference` | IoC types ranked by Pyramid of Pain | | `crowdsentinel://ioc-reference/data` | Machine-readable IoC reference data | | `crowdsentinel://cross-correlation-playbooks` | 5 investigation playbooks (suspicious process, brute force, lateral movement, persistence, exfiltration) | | `crowdsentinel://velociraptor-guide` | Artefact reference tables and "found X in SIEM → check Y on endpoint" decision tree | | `crowdsentinel://data-sources` | All data source capabilities and investigation decision matrix | | `crowdsentinel://tool-recommendations` | Which tool to reach for at each investigation step | | `crowdsentinel://schemas` | Supported log-source schemas (ECS, Sysmon, Windows Security, auditd, Zeek, …) | | `crowdsentinel://schemas/list` | Machine-readable schema listing | --- ## Documentation ### User Guides | Document | Description | |:---------|:------------| | [FIRST_TIME_SETUP.md](FIRST_TIME_SETUP.md) | Detailed first-time setup instructions | | [HOW_TO_USE.md](HOW_TO_USE.md) | Comprehensive usage guide | | [QUICK_START.md](QUICK_START.md) | 5-minute quick start | | [TRANSPORT_MODES.md](TRANSPORT_MODES.md) | stdio, SSE, HTTP configuration | ### Feature Guides | Document | Description | |:---------|:------------| | [THREAT_HUNTING_GUIDE.md](THREAT_HUNTING_GUIDE.md) | Threat hunting workflows | | [DETECTION_RULES_GUIDE.md](DETECTION_RULES_GUIDE.md) | Using 6,060 detection rules | | [CYBER_KILL_CHAIN_GUIDE.md](CYBER_KILL_CHAIN_GUIDE.md) | Kill chain analysis | | [CHAINSAW_GUIDE.md](CHAINSAW_GUIDE.md) | EVTX log analysis with Sigma | | [INVESTIGATION_PROMPTS_GUIDE.md](INVESTIGATION_PROMPTS_GUIDE.md) | Fast triage spine | | [AI_AGENT_INTEGRATION.md](AI_AGENT_INTEGRATION.md) | Workflow guidance for AI agents | ### Developer Guides | Document | Description | |:---------|:------------| | [ARCHITECTURE.md](ARCHITECTURE.md) | Detailed architecture documentation | | [CONTRIBUTING.md](CONTRIBUTING.md) | Contribution guidelines | --- ## Contributing Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. --- ## Licence GNU General Public Licence v3.0 — See [LICENSE](LICENSE) for details. --- ## Cyber Range CrowdSentinel ships with a companion **[crowdsentinel-range](https://github.com/thomasxm/crowdsentinel-range)** — a fully ephemeral AWS lab that generates ground-truth-labeled attack telemetry so you can investigate real detections rather than synthetic data. ### Current range (v1) — 2-host AWS lab ``` ┌──────────────────────────── AWS VPC (private subnet) ────────────────────────────────┐ │ │ │ ┌──────────────────────────────┐ ┌──────────────────────────────────────────┐ │ │ │ Linux SIEM (Ubuntu 22.04) │ │ Windows Range Box (Server 2022) │ │ │ │ Elasticsearch 8.x (TLS) │◄─────│ Sysmon (olafhartong) + Winlogbeat 9 │ │ │ │ Auditbeat + Filebeat │ │ Atomic Red Team + PurpleSharp │ │ │ └──────────────────────────────┘ └──────────────────────────────────────────┘ │ │ SSM VPC endpoints — no inbound ports open │ └───────────────────────────────────────┬────────────────────────────────────────────--┘ │ SSM port-forward :19200 ┌──────────▼──────────────────────┐ │ ./range-mcp.sh → CrowdSentinel │ │ MCP Server → Claude Code │ └─────────────────────────────────┘ ``` **3 commands to go from zero to hunting:** ```bash cd crowdsentinel-range && make up && make attack-windows cd .. && ./range-mcp.sh # then in Claude Code: /mcp → "investigate potential intrusions on the windows machine" ``` > Demo: [73,328 events triaged in ~40 seconds](https://github.com/thomasxm/crowdsentinel-range/blob/main/docs/demo-windows-triage-investigation.md) — 2 shellcode payloads decoded, C2 IP extracted from raw bytes, full IR report with 19 MITRE techniques. --- ### Future range (v2) — enterprise AD environment The next range generation targets **full Active Directory kill-chain coverage**: initial access → lateral movement → domain compromise, with a network security monitor layer, runtime security instrumentation, centralised vulnerability management, and deception infrastructure. Inspired by [DetectionLab](https://github.com/clong/DetectionLab), [Ludus](https://docs.ludus.cloud/), and [SimuLand](https://github.com/Azure/SimuLand). ``` ┌───────────────────────────────────── AWS / Multi-VPC ───────────────────────────────────────────────────┐ │ │ │ ┌──── Attacker Subnet ────┐ ┌───────────── Corporate Subnet ──────────────────────────────────────┐ │ │ │ │ │ │ │ │ │ Kali Linux (C2 server) │ │ ┌─────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ │ │ │ Caldera + Metasploit │ │ │ Domain │ │ Windows 11 │ │ Windows 11 │ │ │ │ │ Covenant C2 │──▶│ │ Controller │ │ Workstation 1 │ │ Workstation 2 │ │ │ │ │ │ │ │ (AD DS, DNS) │ │ (domain-joined) │ │ (domain-joined) │ │ │ │ └──────────────────────────┘ │ └─────────────────┘ └──────────────────┘ └──────────────────┘ │ │ │ │ ┌──────────────────────────────────────────────────────────────┐ │ │ │ ┌──── DMZ Subnet ─────────┐ │ │ Linux File/Web Server (IIS / Apache, SMB share) │ │ │ │ │ │ │ └──────────────────────────────────────────────────────────────┘ │ │ │ │ Nginx reverse proxy │ └───────────────────────────────────────────────────────────────────┘ │ │ │ Simulated internet SVC │ │ │ └──────────────────────────┘ ┌───────────── Management / SIEM Subnet ──────────────────────────┐ │ │ │ │ │ │ │ ┌─────────────────────┐ ┌───────────────────────────────┐ │ │ │ │ │ Elasticsearch + │ │ Zeek / Suricata │ │ │ │ │ │ Kibana (SIEM) │ │ (NSM — full packet + alerts) │ │ │ │ │ └─────────────────────┘ └───────────────────────────────┘ │ │ │ │ ┌─────────────────────┐ ┌───────────────────────────────┐ │ │ │ │ │ Velociraptor │ │ Falco (runtime security) │ │ │ │ │ │ (live EDR/forensics)│ │ eBPF probe (CO-RE, default) │ │ │ │ │ └─────────────────────┘ │ + kernel module fallback │ │ │ │ │ ┌─────────────────────┐ └───────────────────────────────┘ │ │ │ │ │ DefectDojo │ ┌───────────────────────────────┐ │ │ │ │ │ (vuln management) │ │ Honeypots + Canary tokens │ │ │ │ │ │ Falco·Zeek·scanners │ │ (deception layer) │ │ │ │ │ └─────────────────────┘ └───────────────────────────────┘ │ │ │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ │ │ Grafana — centralized dashboards + alerting │ │ │ │ │ │ ES · Falco · Suricata · Velociraptor · DefectDojo │ │ │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ └───────────────────────────────────────────────────────────────┘ │ │ VyOS router — inter-subnet ACLs + egress filter │ └───────────────────────────────────────────────────────────────────────────────────────────────────────┘ │ SSM / VPN CrowdSentinel MCP + Claude Code ``` **v2 target capabilities:** | Capability | v1 (current) | v2 (planned) | |------------|-------------|--------------| | Hosts | 2 (Linux SIEM + Windows) | 7–10 (DC + 2× WS + Linux + NSM + attacker + SIEM) | | Network segments | 1 (private subnet) | 4 (attacker / corp / DMZ / management) | | Active Directory | — | Full AD DS domain + GPO + Kerberos | | Lateral movement coverage | Limited | Pass-the-Hash, Kerberoasting, DCSync | | Network detection | — | Zeek (NSM) + Suricata IDS | | Runtime security | — | Falco; modern eBPF probe (CO-RE, bundled, default) + kernel module fallback; syscall-level host + container detection | | Endpoint forensics | — | Velociraptor live triage | | Vuln management | — | DefectDojo; aggregates Falco alerts, Zeek/Suricata findings, scanner results (OpenVAS, Trivy) into unified tracking + remediation workflow | | Dashboards | — | Grafana; centralised metrics + alert visualisation across ES, Falco, Suricata, Velociraptor, and DefectDojo | | Deception layer | — | Honeypot + canary tokens | | C2 infrastructure | ART atomics only | Caldera + Metasploit + Covenant | | Attack scenarios | Windows atomics + EVTX replay | Full kill-chain: initial access → domain compromise | | CrowdSentinel coverage | ES hunt + IR | ES + Zeek/Suricata + Falco + Velociraptor cross-source correlation; DefectDojo/Grafana as upstream context | **Design references:** [DetectionLab](https://github.com/clong/DetectionLab) (Sysmon+WEF+Splunk convergence), [Ludus](https://docs.ludus.cloud/) (modular Proxmox/Ansible, VLAN segmentation), [SimuLand](https://github.com/Azure/SimuLand) (multi-domain AD forest, cloud-native), [SANS FOR508](https://www.sans.org/cyber-security-courses/advanced-incident-response-threat-hunting-training/) (artifact-first DFIR workflow), [Falco](https://falco.org/docs/concepts/falco-kernel-testing/) (CO-RE eBPF probe architecture, kernel module fallback). ## Acknowledgements CrowdSentinel stands on the shoulders of outstanding open-source projects. Thank you to all of their maintainers and contributors. **Protocol & server framework** - [Model Context Protocol](https://modelcontextprotocol.io/) — the protocol connecting LLMs to tools, by Anthropic - [FastMCP](https://github.com/jlowin/fastmcp) — the Python MCP server framework this project is built on **Analysis engines & forensic tools** - [Chainsaw](https://github.com/WithSecureLabs/chainsaw) — EVTX log analyser by WithSecure Labs - [Velociraptor](https://docs.velociraptor.app/) — live endpoint forensics and DFIR platform by Velocidex/Rapid7 - [osquery](https://osquery.io/) — SQL-powered host instrumentation by the osquery Foundation - [Wireshark / TShark](https://www.wireshark.org/) — network protocol analysis by the Wireshark Foundation - [Chepy](https://github.com/securisec/chepy) — CyberChef-like decoding library powering optional long-tail deobfuscation - [sqlglot](https://github.com/tobymao/sqlglot) — SQL parser used to safely validate osquery queries **Detection content** - [Sigma / SigmaHQ](https://github.com/SigmaHQ/sigma) — the community detection rule standard and rule corpus - [Elastic detection-rules](https://github.com/elastic/detection-rules) — Elastic's prebuilt EQL/ES|QL detection rules - Community-contributed Sigma and custom rules **Data platforms & threat intelligence** - [Elasticsearch](https://github.com/elastic/elasticsearch-py) and [OpenSearch](https://github.com/opensearch-project/opensearch-py) Python clients - [MISP / PyMISP](https://github.com/MISP/PyMISP) — open-source threat intelligence platform and client - [stix2](https://github.com/oasis-open/cti-python-stix2) — OASIS STIX 2.1 library for structured IoC export - [Shodan InternetDB](https://internetdb.shodan.io/), [VirusTotal](https://www.virustotal.com/), [AbuseIPDB](https://www.abuseipdb.com/), and [abuse.ch ThreatFox](https://threatfox.abuse.ch/) — IoC enrichment services with free tiers **Development & testing** - [testcontainers-python](https://github.com/testcontainers/testcontainers-python) — real Elasticsearch containers for the live test tiers - [pytest](https://pytest.org/), [Ruff](https://github.com/astral-sh/ruff), [Bandit](https://github.com/PyCQA/bandit) — test runner, linting, and SAST **Methodology frameworks** - Cyber Kill Chain (Lockheed Martin), Pyramid of Pain (David J. Bianco), Diamond Model (Caltagirone, Pendergast & Betz), [MITRE ATT&CK](https://attack.mitre.org/) ---

**Made for the security community by [medjedtxm](https://github.com/thomasxm)** [![GitHub](https://img.shields.io/badge/GitHub-CrowdSentinel-181717?style=for-the-badge&logo=github)](https://github.com/thomasxm/CrowdSentinels-AI-MCP) [![PyPI](https://img.shields.io/badge/PyPI-crowdsentinel--mcp--server-3775A9?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/crowdsentinel-mcp-server/)