# Awesome Web Agents

A curated list of tools, frameworks, and resources for building AI agents that can browse and interact with the web.

About Steel

Steel is an [open-source](https://github.com/steel-dev/steel-browser) browser API built specifically for AI agents. We make it easy to build AI applications that can effectively interact with the web. ✨ Get started for free [here](https://app.steel.dev).

Contents

- [Awesome Web Agents](#awesome-web-agents) - [Autonomous Web Agents](#autonomous-web-agents) - [Computer-use Agents](#computer-use-agents) - [AI Web Automation Tools](#ai-web-automation-tools) - [Dev Tools](#dev-tools) - [AI Web Scrapers/Crawlers](#ai-web-scraperscrawlers) - [Web Search \& Query Tools](#web-search--query-tools) - [Benchmarks \& Research](#benchmarks--research) - [Tutorials \& Guides](#tutorials--guides) - [Archive](#archive) - [Interested in implementing Steel?](#interested-in-implementing-steel) - [Join the Community](#join-the-community) - [Contributing](#contributing) - [Contributors](#contributors) ## Autonomous Web Agents AI agents that autonomously navigate and interact with the web through a user-friendly interface. (a.k.a Browser Agents) - [OpenAI Operator](https://openai.com/index/introducing-operator/) - OpenAI's AI agents that can browse the web for you. - [Browser-Use](https://www.browser-use.com) - SOTA agent and framework that makes the web LLM-friendly. ![GitHub Repo stars](https://img.shields.io/github/stars/Browser-Use/browser-use?style=social) - [Skyvern-AI](https://www.skyvern.com/) - Framework to automate browser-based workflows. ![GitHub Repo stars](https://img.shields.io/github/stars/Skyvern-AI/skyvern?style=social) - [Google Project Mariner](https://deepmind.google/technologies/project-mariner/) - A research prototype exploring the future of human-agent interaction, starting with your browser. - [Sentience API](https://www.sentienceapi.com) - A tool for building more deterministic and explainable web agents using semantic geometry on web content. - [Runner H](https://www.hcompany.ai/) - State-of-the-art AI agent that helps automate complex, cumbersome, multi-step tasks without repetitive manual input. - [WebVoyager (Agent)](https://github.com/MinorJerry/WebVoyager) - Vision-enabled web agent. ![GitHub Repo stars](https://img.shields.io/github/stars/MinorJerry/WebVoyager?style=social) - [AgentGPT](https://github.com/reworkd/AgentGPT) - Deploy autonomous AI agents in your browser. ![GitHub Repo stars](https://img.shields.io/github/stars/reworkd/AgentGPT?style=social) - [Agent-E](https://github.com/EmergenceAI/Agent-E) - Agent & framework with HTML DOM distillation. ![GitHub Repo stars](https://img.shields.io/github/stars/EmergenceAI/Agent-E?style=social) - [Manus](https://manus.im/) - A general AI agent that can execute long running tasks across tools like browsers, terminals, and text editors. - [doBrowser](https://www.dobrowser.io) - An AI-powered Chrome extension that understands natural language and takes actions in your browser on your behalf. - [WebSurfer (Autogen)](https://microsoft.github.io/autogen/stable/reference/python/autogen_ext.agents.web_surfer.html#autogen_ext.agents.web_surfer.MultimodalWebSurfer) - MultimodalWebSurfer is a multimodal agent that can search the web and visit web pages. ![GitHub Repo stars](https://img.shields.io/github/stars/microsoft/autogen?style=social) - [Magentic-One](https://www.microsoft.com/en-us/research/articles/magentic-one-a-generalist-multi-agent-system-for-solving-complex-tasks/) - A generalist multi-agent system for solving complex tasks including surfing the web via Autogen's MultimodalWebSurfer. - [Harpa.ai](https://harpa.ai/) - An AI-powered Chrome extension & browser agent that understands natural language and takes actions on your behalf. - [Yutori](https://yutori.com/) - A multi-agent system that executes browser-based tasks in parallel given a natural language prompt. - [rtrvr.ai](https://www.rtrvr.ai/) - AI web agent Chrome extension that autonomously does tasks, scrapes to Sheets, and calls APIs with prompts in your own browser. - [Nanobrowser](https://nanobrowser.ai) - An open-source & local-first AI web agent Chrome extension with flexible LLM options and multi-agent system. ![GitHub Repo stars](https://img.shields.io/github/stars/nanobrowser/nanobrowser?style=social) - [Browserable](https://browserable.ai) - An open-source & self-hostable browser automation library for AI agents. ![GitHub Repo stars](https://img.shields.io/github/stars/browserable/browserable?style=social) - [Tongyi WebAgent](https://github.com/Alibaba-NLP/WebAgent) - WebAgent for information seeking built by Tongyi Lab, Alibaba Group. ![GitHub Repo stars](https://img.shields.io/github/stars/Alibaba-NLP/WebAgent?style=social) - [Openwork](https://github.com/accomplish-ai/openwork) - An MIT-licensed, open alternative to Anthropic's Cowork built with Opencode and dev-browser. Supports multiple LLM providers for launching computer-use agents to automate browser workflows. ![GitHub Repo stars](https://img.shields.io/github/stars/accomplish-ai/openwork?style=social) - [Dassi](https://www.dassi.ai/) - An AI coworking agent in your browser that automates tasks, navigates pages, and works with files and 2000+ apps from a side panel. - [Caesar](https://github.com/jasonzliang/caesar-agent) - Autonomous research agent that traverses the web to build a knowledge graph, then synthesizes an answer through adversarial redrafting. ![GitHub Repo stars](https://img.shields.io/github/stars/jasonzliang/caesar-agent?style=social) ### Computer-use Agents - [Anthropic Computer Use](https://www.anthropic.com/news/3-5-models-and-computer-use) - Computer use agent that can control your browser. - [Self-Operating Computer Framework](https://github.com/OthersideAI/self-operating-computer) - A framework to enable multimodal models to operate a computer. ![GitHub Repo stars](https://img.shields.io/github/stars/OthersideAI/self-operating-computer?style=social) - [Highlight](https://highlightai.com/) - Desktop activity layer that helps models understand your workflow and complete tasks faster. - [OpenInterpreter](https://github.com/openinterpreter/open-interpreter) - An open-source CLI based agent that can write & execute code as well as control your browser. ![GitHub Repo stars](https://img.shields.io/github/stars/openinterpreter/open-interpreter?style=social) - [UI-TARS](https://github.com/bytedance/UI-TARS?tab=readme-ov-file) - A GUI agent model designed to interact seamlessly with GUIs using human-like perception, reasoning, and action capabilities. ![GitHub Repo stars](https://img.shields.io/github/stars/bytedance/UI-TARS?style=social) ## AI Web Automation Tools Tools, frameworks and libraries that translate natural language instructions into web interactions. - [Asteroid.ai](https://asteroid.ai/) - Hosted browser agents for SMEs to automate complex workflows. ![GitHub Repo stars](https://img.shields.io/github/stars/asteroidai/asteroid?style=social) - [PulsarRPA](https://github.com/platonai/pulsarRPA) - AI-powered browser automation for data extraction. ![GitHub Repo stars](https://img.shields.io/github/stars/platonai/pulsarRPA?style=social) - [VimGPT](https://github.com/ishan0102/vimGPT) - Experimental project using GPT-4 Vision to browse the web via the Vimium extension. ![GitHub Repo stars](https://img.shields.io/github/stars/ishan0102/vimGPT?style=social) - [Cekura.io](https://www.cekura.io/) - An AI browser agent that helps companies maintain up-to-date documentation. - [Dex by Dexterity](https://getdexterity.com/) - An AI coworker embedding into and controlling your browser. - [Autobrowser](https://chromewebstore.google.com/detail/autobrowser/lebgieachadpombahcacicmebckokcjl) - A free, experimental Chrome extension that leverages Claude Computer Use to automate tasks in your browser. - [Bytebot](https://bytebot.ai) - AI-powered scraping automations that evolve with your target sites. - [Runcopycat](https://www.runcopycat.com/) - A no-code browser automation platform that turns screen recordings into reusable automated workflows. - [Bardeen.ai](https://bardeen.ai) - A Chrome extension that enables AI-powered browser automations, allowing users to automate tasks and workflows directly within the browser. - [Starizon.ai](https://starizon.ai/) - Browser assistant for web task automation. - [BrowserGPT](https://browsegpt.ai/) - Browser extension for page summaries and Q&A. - [Browse.ai](https://www.browse.ai/) - Chrome extension webscraping that can leverage AI for structured data extraction. - [Deta.surf](https://deta.surf/) - An integrated platform that combines a browser, file manager, and AI assistant with browser-level context. - [Comet by Perplexity](https://www.perplexity.ai/comet) - An AI-powered browser by Perplexity. Not much more details out yet. - [Dia Browser](https://www.diabrowser.com/) - AI-first web browser envisioned by The Browser Company (Arc). - [Reworkd](https://reworkd.ai) - No-code web data extraction solution using agentic AI. - [Onpiste](https://onpiste.work) - Chrome extension that uses AI to control and read webpages, including auto summaries, web automation, scraping, and MCP support. - [Komos](https://www.komos.ai/) - AI browser workflow automation platform that turns recorded web tasks into reusable runs with API triggers, schedules, credentials, logs, and human review. - [agent-qa](https://github.com/vostride/agent-qa) - Self-improving QA harness for natural-language web test runs with execution memory and self-healing actions. ![GitHub Repo stars](https://img.shields.io/github/stars/vostride/agent-qa?style=social) ### Dev Tools - [Steel.dev](https://steel.dev) - Open-source headless browser API built specifically for AI agents and apps. ![GitHub Repo stars](https://img.shields.io/github/stars/steel-dev/steel-browser?style=social) - [Atlas](https://github.com/steel-dev/atlas) - Open-source deep research harness for building cited web research agents with ledger-based coverage audits, pluggable search providers, and Steel-backed browser fetches. ![GitHub Repo stars](https://img.shields.io/github/stars/steel-dev/atlas?style=social) - [Omniparser](https://microsoft.github.io/OmniParser/) - Tool for parsing GUIs for vision based agents. ![GitHub Repo stars](https://img.shields.io/github/stars/microsoft/OmniParser?style=social) - [LaVague](https://www.lavague.ai/) - Framework for natural language web automation. ![GitHub Repo stars](https://img.shields.io/github/stars/lavague-ai/LaVague?style=social) - [LangChain Playwright Toolkit](https://python.langchain.com/docs/integrations/tools/playwright/#use-within-an-agent) - Toolkit integration with AI agents. - [Browserbase](https://browserbase.com) - A headless browser API for AI workflows. - [Stagehand](https://www.stagehand.dev/) - AI web browsing framework. ![GitHub Repo stars](https://img.shields.io/github/stars/browserbase/stagehand?style=social) - [Tarsier](https://github.com/reworkd/tarsier) - Vision utilities library for web interaction agents. ![GitHub Repo stars](https://img.shields.io/github/stars/reworkd/tarsier?style=social) - [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT) - Experimental agent for task completion and web browsing. ![GitHub Repo stars](https://img.shields.io/github/stars/Significant-Gravitas/AutoGPT?style=social) - [TinyFish](https://www.tinyfish.ai) - Remote web agents that execute tasks on any website and return structured JSON via a single API call. ![GitHub Repo stars](https://img.shields.io/github/stars/tinyfish-io/tinyfish-cookbook?style=social) - [Bytebot](https://github.com/bytebot-ai/bytebot) - Containerized computer use agent framework with a virtual desktop environment. ![GitHub Repo stars](https://img.shields.io/github/stars/bytebot-ai/bytebot?style=social) - [Lumen](https://github.com/omxyz/lumen) - Vision-first browser agent with self-healing deterministic replay. Screenshot → model → action loop over CDP, multi-provider (Anthropic, Google, OpenAI), action caching for zero-token reruns. ![GitHub Repo stars](https://img.shields.io/github/stars/omxyz/lumen?style=social) - [Webagent-cloud](https://webagent.cloud) - Open-source API for browser agents to automate repetitive workflows. Works with multiple browsers/LLM providers, and minimizes costs with a self-healing action cache. ![GitHub Repo stars](https://img.shields.io/github/stars/webagent-cloud/webagent?style=social) - [BrowserTrace](https://github.com/aaronlab/browsertrace) - Local-first trace viewer for debugging Playwright, Browser Use, Stagehand, and other web-agent runs with redacted shareable exports. ![GitHub Repo stars](https://img.shields.io/github/stars/aaronlab/browsertrace?style=social) - [Notte](https://notte.cc) - Browser infrastructure for AI agents with managed sessions, an agent runtime, and credential vault and persona authentication primitives. ![GitHub Repo stars](https://img.shields.io/github/stars/nottelabs/notte?style=social) - [invisible-playwright](https://github.com/feder-cr/invisible_playwright) - Playwright wrapper for a stealth-patched Firefox 150 build. Drop-in replacement returning native Playwright Browser objects; spoofing happens in C++ source with no JS-level overrides. ![GitHub Repo stars](https://img.shields.io/github/stars/feder-cr/invisible_playwright?style=social) - [Webwright](https://github.com/microsoft/Webwright) - Browser agent framework from Microsoft Research where the agent writes and runs Playwright scripts in a terminal workspace; supports OpenAI, Anthropic, and OpenRouter backends. ![GitHub Repo stars](https://img.shields.io/github/stars/microsoft/Webwright?style=social) - [BrowserAct](https://github.com/browser-act/skills) - Browser automation CLI and skills for AI agents to operate real browsers, manage sessions, support human handoff, and capture screenshots and evidence. ![GitHub Repo stars](https://img.shields.io/github/stars/browser-act/skills?style=social) - [Agent Browser Shield](https://github.com/pixiebrix/agent-browser-shield) - Browser extension that sits between an AI agent and the page, stripping prompt injection, masking PII/credentials, and removing dark patterns before content reaches the model. ![GitHub Repo stars](https://img.shields.io/github/stars/pixiebrix/agent-browser-shield?style=social) - [HUD](https://github.com/hud-evals/hud-python) - Open-source SDK for building browser and computer-use RL environments to evaluate and train web agents, with task-based verifiable rewards runnable as evals or RL training across any model. ![GitHub Repo stars](https://img.shields.io/github/stars/hud-evals/hud-python?style=social) - [Webfuse](https://www.webfuse.com) - Configurable web proxy and browser-as-a-service for deploying and operating AI agents in a sandbox layer on top of any third-party website, using client-side extensions and without source-code access. - [Webcmd](https://github.com/agentrhq/webcmd) - Self-learning browser infrastructure for AI agents that records how a site is navigated, then compiles that context into deterministic CLI adapters and reusable sitemap memory for later runs. ![GitHub Repo stars](https://img.shields.io/github/stars/agentrhq/webcmd?style=social) - [Hermes Connector](https://github.com/CorsenAI/hermes-connector) - Unofficial open-source Chrome extension and local companion that let Hermes Agent control only explicitly attached tabs in a user's Chrome session. ![GitHub Repo stars](https://img.shields.io/github/stars/CorsenAI/hermes-connector?style=social) ## AI Web Scrapers/Crawlers Web crawlers & scrapers that leverage AI to navigate websites and extract content. - [FireCrawl](https://www.firecrawl.dev/) - APIs for turning websites into LLM-friendly markdown. ![GitHub Repo stars](https://img.shields.io/github/stars/mendableai/firecrawl?style=social) - [Crawl4AI](https://crawl4ai.com) - Open-source LLM Friendly Web Crawler & Scraper. ![GitHub Repo stars](https://img.shields.io/github/stars/unclecode/crawl4ai?style=social) - [ScrapeGraphAI](https://scrapegraphai.com/) - Python scraper based on AI. ![GitHub Repo stars](https://img.shields.io/github/stars/ScrapeGraphAI/Scrapegraph-ai?style=social) - [WebAgent (OpenAgents)](https://github.com/xlang-ai/OpenAgents) - The web-browsing agent module of the OpenAgents platform (HKU). Enables autonomous navigation of websites via natural language, as part of a larger multi-modal agent framework. ![GitHub Repo stars](https://img.shields.io/github/stars/xlang-ai/OpenAgents?style=social) - [Expand.ai](https://www.expand.ai/) - Turns any website into a type-safe API you can rely on. - [LLM Scraper](https://github.com/mishushakov/llm-scraper) - Uses LLMs for intelligent scraping and content understanding. ![GitHub Repo stars](https://img.shields.io/github/stars/mishushakov/llm-scraper?style=social) - [Plasmate](https://github.com/plasmate-labs/plasmate) - Open-source headless browser engine for AI agents. Compiles HTML to Semantic Object Model (SOM) with 17.5x token compression. 13 MCP tools. First browser tool on the MCP Registry. Rust, Apache-2.0. ![GitHub Repo stars](https://img.shields.io/github/stars/plasmate-labs/plasmate?style=social) - [SpiderCreator](https://github.com/carlosplanchon/spidercreator) - Create complex Playwright spiders with natural language prompts. ![GitHub Repo stars](https://img.shields.io/github/stars/carlosplanchon/spidercreator?style=social) - [DataLens](https://datalens.uk) - AI web data agent that plans browser scraping tasks, extracts structured web data, and exposes MCP tools for dataset workflows. - [Context.dev](https://www.context.dev/) - Web APIs and MCP tools for search, scraping, crawling, schema-based extraction, document parsing, monitoring, and batch jobs. ## Web Search & Query Tools Utilities that help agents search the web or query web data via natural language. - [AgentQL](https://www.agentql.com/) - A query language and toolkit that makes the web AI-ready. ![GitHub Repo stars](https://img.shields.io/github/stars/tinyfish-io/agentql?style=social) - [SerpAPI](https://serpapi.com/) - Search API that provides Google Search results for your agents. - [Serper.dev](https://serper.dev/) - Performant and cost effective search API that provides Google Search results for your agents. - [Jina.ai](https://jina.ai/) - Neural search platform for web data. - [Exa.ai](https://exa.ai) - The fastest and most accurate web search API for AI agents. - [Not Human Search](https://nothumansearch.ai) - Search engine that indexes 1,750+ agent-first tools ranked by agentic readiness. Available as an MCP server with tools for searching, scoring, and monitoring agent infrastructure. - [Superhighway](https://superhighway.walls.sh) - Web search API for AI agents with five tools (search, news, images, scrape, research); agents pay per call in USDC via the x402 protocol, or use a free API key. - [Zoom Search](https://github.com/goofrey/zoom-search) - Open-source web search and evidence tool for AI agents with query rewriting, source-domain zoom-in, structured sourced outputs, and MCP and LangGraph integrations. ![GitHub Repo stars](https://img.shields.io/github/stars/goofrey/zoom-search?style=social) - [SocialCrawl](https://www.socialcrawl.dev) - Unified social and commerce data API covering 50+ platforms as one GET and one JSON schema. ## Benchmarks & Research Datasets, benchmarks, and notable research efforts for evaluating and advancing web-capable AI agents. - [Web Agent Leaderboard](https://leaderboard.steel.dev) - Leaderboard compiling AI agent products and their performance on widely used WebVoyager benchmarks. ![GitHub Repo stars](https://img.shields.io/github/stars/steel-dev/leaderboard?style=social) - [Web Games by Convergence](https://webgames.convergence.ai/) - A collection of challenges designed for testing general-purpose web-browsing AI agents. ![GitHub Repo stars](https://img.shields.io/github/stars/convergence-ai/webgames?style=social) - [Bananalyzer](https://github.com/reworkd/bananalyzer) - An open-source evaluation framework for web-based AI agents. ![GitHub Repo stars](https://img.shields.io/github/stars/reworkd/bananalyzer?style=social) - [Mind2Web](https://osu-nlp-group.github.io/Mind2Web) - A large-scale dataset for generalist web agents. ![GitHub Repo stars](https://img.shields.io/github/stars/OSU-NLP-Group/Mind2Web?style=social) - [World of Bits: An Open-Domain Platform for Web-Based Agents](https://proceedings.mlr.press/v70/shi17a/shi17a.pdf) - OpenAI's research paper that introduces World of Bits: a platform where agents complete tasks on the internet by performing low-level keyboard and mouse actions. - [MiniWoB++](https://miniwob.farama.org) - A classic suite of 104 mini web browser tasks in a synthetic environment. It is an extension of the OpenAI MiniWoB benchmark. ![GitHub Repo stars](https://img.shields.io/github/stars/Farama-Foundation/miniwob-plusplus?style=social) - [WebTaskBench](https://github.com/plasmate-labs/plasmate-benchmarks) - 51-URL benchmark comparing HTML vs Markdown vs SOM representations for AI agents. Measures token efficiency, latency, and accuracy across GPT-4o and Claude Sonnet 4. ![GitHub Repo stars](https://img.shields.io/github/stars/plasmate-labs/plasmate-benchmarks?style=social) - [WebArena](https://webarena.dev) - A realistic, self-hostable web environment for autonomous agents. Includes official leaderboard tracking agent performance. ![GitHub Repo stars](https://img.shields.io/github/stars/web-arena-x/webarena?style=social) - [WebCanvas](https://github.com/iMeanAI/WebCanvas) - An online evaluation framework for dynamic web environments. Tests agents on live websites. ![GitHub Repo stars](https://img.shields.io/github/stars/iMeanAI/WebCanvas?style=social) - [WebGPT](https://openai.com/research/webgpt) - OpenAI's browser-assisted question-answering research project. - [WebShop](https://webshop-pnlp.github.io) - A simulated e-commerce shopping environment with 1.18M real Amazon products. ![GitHub Repo stars](https://img.shields.io/github/stars/princeton-nlp/WebShop?style=social) - [WebVoyager (Benchmark)](https://arxiv.org/abs/2401.13919) - Vision-enabled benchmark for real-world website interaction with large multimodal models. ![GitHub Repo stars](https://img.shields.io/github/stars/MinorJerry/WebVoyager?style=social) - [WorkArena](https://github.com/ServiceNow/WorkArena) - A suite of 33 browser-based tasks for enterprise "knowledge worker" scenarios. ![GitHub Repo stars](https://img.shields.io/github/stars/ServiceNow/WorkArena?style=social) - [BrowserGym by ServiceNow](https://github.com/ServiceNow/BrowserGym) - A gym environment for web task automation. ![GitHub Repo stars](https://img.shields.io/github/stars/ServiceNow/BrowserGym?style=social) - [TimeWarp](https://timewarp-web.github.io) - A benchmark on historical versions of web UI. - [ClawBench](https://github.com/reacher-z/ClawBench) - 283 everyday tasks (V1 153 + V2 130) on 163 live production websites across 15 categories. Two-stage scoring (final HTTP-request interception + LLM judge) blocks only the write request so real sites stay clean. Public leaderboard with 5-layer execution traces (recording, action log, request log, agent messages, interception). ![GitHub Repo stars](https://img.shields.io/github/stars/reacher-z/ClawBench?style=social) ## Tutorials & Guides Resources for learning how to build, deploy, or utilize AI web agents. - [LangGraph WebVoyager Tutorial](https://langchain-ai.github.io/langgraph/tutorials/web-navigation/web_voyager/) - Tutorial demonstrating how to build a web navigation agent using LangGraph Agents, Vision Models, and Web Voyager. - [Build an AI Browser Agent](https://dzone.com/articles/build-ai-browser-agent-llms-playwright-browser-use) - Step-by-step guide to create an AI that browses the web using Playwright and the Browser-Use library. - [Install & Run Browser-Use Locally](https://aleksandarhaber.com/install-and-run-browser-use-ai-agents-locally-using-ollama/) - Instructions on installing the open-source Browser-Use agent with a local LLM. - [Build a Browser Agent with DeepSeek](https://dev.to/nodeshiftcloud/build-a-browser-use-agent-with-deepseek-a-step-by-step-guide-2n59) - Walks through deploying a Browser-Use web UI agent powered by the DeepSeek model on a cloud VM. ## Archive Historical or inactive projects are tracked in [ARCHIVE.md](ARCHIVE.md). ## Interested in implementing Steel? Feel free to reach out at [team@steel.dev](mailto:team@steel.dev?subject=Hello%20from%20github!) or on [Discord](https://discord.gg/steel-dev). Steel is an [open-source](https://github.com/steel-dev/steel-browser) browser API built specifically for AI agents. Get started for free [here](https://app.steel.dev). ## Join the Community - Follow [@steeldotdev](https://x.com/steeldotdev) on X. - Join the [Discord community](https://discord.gg/steel-dev). - Feel free to reach out to us at [team@steel.dev](mailto:team@steel.dev?subject=Hello%20from%20github!) ## Contributing [Contributions of any kind welcome, just follow the guidelines](CONTRIBUTING.md)! ### Contributors [Thanks goes to these contributors](https://github.com/steel-dev/awesome-web-agents/graphs/contributors)!