# ScrapFly Scrapers 🕷️ **English** · [简体中文](docs/README.zh-CN.md) · [日本語](docs/README.ja.md) · [Español](docs/README.es.md) · [Русский](docs/README.ru.md) This repository contains educational example scrapers for popular web scraping targets using the [ScrapFly](https://scrapfly.io) web scraping API and Python. Most scrapers use a simple web scraping stack: - Python version 3.10+ - [Scrapfly's Python SDK](https://github.com/scrapfly/python-scrapfly) for sending HTTP requests, getting past blocking and parsing the HTML with the built-in [parsel](https://pypi.org/project/parsel/) selector. - [asyncio](https://docs.python.org/3/library/asyncio.html) for writing concurrent code using the async/await syntax. - [JMESPath](https://pypi.org/project/jmespath/) and [nested-lookup](https://pypi.org/project/nested-lookup/) for JSON parsing when needed. - [loguru](https://pypi.org/project/loguru/) for logging. To learn more about web scraping, see our full tutorials on how to scrape these targets (and many others) in the [scrapeguide directory](https://scrapfly.io/blog/tag/scrapeguide/). For AI agents and LLM tools, [llms.txt](llms.txt) lists every scraper with its functions and sample output, and explains how to run a scraper and how to write a new one. ## Quick start Every scraper lives in its own folder and runs on its own (Python 3.10+ and [Poetry](https://python-poetry.org/docs/#installation)): ```shell git clone https://github.com/scrapfly/scrapfly-scrapers.git cd scrapfly-scrapers/airbnb-scraper # any scraper folder from the list below export SCRAPFLY_KEY="YOUR SCRAPFLY KEY" # from https://scrapfly.io/dashboard poetry install --no-root poetry run python run.py # example scrape, saves its output in ./results poetry run pytest test.py # the scraper's tests ``` Each scraper folder's README lists what it scrapes. ## List of Scrapers Below is the list of available web scrapers for the supported domains along with their scrape guide, sample datasets, and status. 👇 The status comes from the nightly test run, which also orders the table: scrapers whose tests pass come first, failing and untested ones last. A badge reads **passing** (every test passed), **N/M passing** (some tests passed), **failing** (none passed) or **not run**; click it to see that scraper's last results, with a link to each test's log.
Domain Guide Sample Datasets Status
Air France How to Scrape Air France Flights with Python (2026) airfrance-scraper-status
Airbnb.com How to Scrape Airbnb Listings and Prices (2026) airbnb-scraper-status
Aliexpress.com How to Scrape Aliexpress.com (2025 Update) aliexpress-scraper-status
Allegro.pl How to Scrape Allegro.pl in 2026 (Without Getting Blocked) allegro-scraper-status
Amazon.com How to Scrape Amazon.com Product Data and Reviews amazon-scraper-status
App Store - app-store-scraper-status
AutoScout24.com How to Scrape AutoScout24 in 2026 autoscout24-scraper-status
BestBuy.com How to Scrape BestBuy Product, Offer and Review Data bestbuy-scraper-status
Bing.com How to Scrape Bing Search with Python bing-scraper-status
Booking.com How to Scrape Booking.com (2025 Update) bookingcom-scraper-status
Capterra.com How to Scrape Capterra Software Reviews and Listings (2026) capterra-scraper-status
ChatGPT.com How to Scrape ChatGPT in 2026 chatgpt-scraper-status
Crunchbase.com How to Scrape Crunchbase in 2025 crunchbase-scraper-status
DHL How to Scrape DHL Parcel Tracking Data in 2026 dhl-scraper-status
DigiKey.com How to Scrape DigiKey Pricing, Stock, and Specs in 2026 digikey-scraper-status
Domain.com.au How to Scrape Domain.com.au Real Estate Property Data domaincom-scraper-status
Emirates How to Scrape Emirates Flights with Python (2026) emirates-scraper-status
Etsy.com How to Scrape Etsy.com Product, Shop and Search Data etsy-scraper-status
Expedia.com - expedia-scraper-status
Facebook.com How to Scrape Facebook in 2026 facebook-scraper-status
Fashionphile.com How to Scrape Fashionphile for Second Hand Fashion Data fashionphile-scraper-status
G2.com How to Scrape G2 Company Data and Reviews g2-scraper-status
Glassdoor.com How to Scrape Glassdoor (2025 update) glassdoor-scraper-status
Goat.com How to Scrape Goat.com for Fashion Apparel Data in Python goat-scraper-status
Goibibo How to Scrape Goibibo Prices in 2026 goibibo-scraper-status
Goodreads.com How to Scrape Goodreads for Book Data, Ratings, and Reviews goodreads-scraper-status
Google Flights How to Scrape Google Flights Data in 2026 google-flights-scraper-status
Google Jobs How to Scrape Google Jobs in 2026 google-jobs-scraper-status
Google Play How to Scrape Google Play App Reviews and Data google-play-scraper-status
Google.com How to Scrape Google Search Results - How to Scrape Google Maps google-scraper-status
Homegate.ch How to Scrape Homegate.ch Real Estate Property Data homegate-scraper-status
Idealista.com How to Scrape Idealista.com in Python - Real Estate Property Data idealista-scraper-status
Idealo.de - idealo-scraper-status
IMDb.com How to Scrape IMDb Data, Ratings, and Featured Reviews imdb-scraper-status
Immobilienscout24.de How to Scrape Immobilienscout24.de Real Estate Data immobilienscout24-scraper-status
Immoscout24.ch How to Scrape Immoscout24.ch Real Estate Property Data immoscout24-scraper-status
Immowelt.de How to Scrape Immowelt.de Real Estate Data immowelt-scraper-status
ImovelWeb.com.br How to Scrape Imovelweb Without Getting Blocked in 2026 imovelweb-scraper-status
Indeed.com How to Scrape Indeed.com (2025 Update) indeed-scraper-status
Instagram.com How to Scrape Instagram in 2025 instagram-scraper-status
Kayak How to Scrape Kayak Flight Prices in 2026 kayak-scraper-status
Leboncoin.fr How to Web Scrape Leboncoin.fr using Python leboncoin-scraper-status
Linkedin.com How to Scrape LinkedIn in 2025 linkedin-scraper-status
LoopNet.com - loopnet-scraper-status
Lowes.com How to Scrape Lowe's Product and Pricing Data in 2026 lowes-scraper-status
Maersk How to Scrape Maersk Shipping Data in 2026 maersk-scraper-status
Marriott.com Scraping Marriott Hotel Prices and Availability (2026) marriott-scraper-status
Mouser.com How to Scrape Mouser.com in 2026 mouser-scraper-status
Naver.com How to Scrape Naver.com: 2026 Python Guide naver-scraper-status
Nordstorm.com How to Scrape Nordstrom Fashion Product Data nordstorm-scraper-status
Perplexity.ai How to Scrape Perplexity AI in 2026 perplexity-scraper-status
Pinterest.com How to Scrape Pinterest in 2026 pinterest-scraper-status
Realestate.com.au How to Scrape Realestate.com.au Property Listing Data realestatecom-scraper-status
Realtor.com How to Scrape Realtor.com - Real Estate Property Data realtorcom-scraper-status
Reddit.com How to Scrape Reddit Posts, Subreddits and Profiles reddit-scraper-status
Redfin.com How to Scrape Redfin Real Estate Property Data in Python redfin-scraper-status
Rightmove.com How to Scrape RightMove Real Estate Property Data with Python rightmove-scraper-status
Rs-online.com How to Scrape RS-Online (rs-online.com) in 2026 rs-online-scraper-status
Seloger.com How to Scrape Seloger.com - Real Estate Listing Data seloger-scraper-status
Shopify stores How to Scrape Shopify Stores for Products, Prices, and Variants shopify-scraper-status
Similarweb.com How to Scrape SimilarWeb Website Traffic Analytics similarweb-scraper-status
Skyscanner How to Scrape Skyscanner Flight Prices in Python (2026) skyscanner-scraper-status
Stockx.com How to Scrape StockX e-commerce Data with Python stockx-scraper-status
Target.com How to Scrape Target.com Product and Pricing Data target-scraper-status
Threads.net How to scrape Threads by Meta using Python (2025 Update) threads-scraper-status
Ticketmaster.com How to Scrape Ticketmaster in 2026 ticketmaster-scraper-status
TikTok.com How To Scrape TikTok in 2025 tiktok-scraper-status
Tripadvisor.com How to Scrape TripAdvisor.com (2025 Updated) tripadvisor-scraper-status
Trustpilot.com How to Scrape Trustpilot.com Reviews and Company Data trustpilot-scraper-status
Twitter(X).com How to Scrape X.com (Twitter) using Python (2025 Update) twitter-scraper-status
VestiaireCollective.com How to Scrape Vestiaire Collective for Fashion Product Data vestiairecollective-scraper-status
Walmart.com How to Scrape Walmart.com Product Data (2025 Update) walmart-scraper-status
Wellfound.com How to Scrape Wellfound Company Data and Job Listings wellfound-scraper-status
Yellowpages.com How to Scrape YellowPages.com in 2025 yellowpages-scraper-status
Yelp.com How to Web Scrape Yelp.com (2025 update) yelp-scraper-status
YouTube.com How to Scrape YouTube in 2025 youtube-scraper-status
Zillow.com How to Scrape Zillow Real Estate Property Data in Python zillow-scraper-status
Zoominfo.com How to Scrape Zoominfo Company Data (2025 Update) zoominfo-scraper-status
Zoopla.co.uk How to Scrape Zoopla Real Estate Property Data in Python zoopla-scraper-status
Zoro.com How to Scrape Zoro.com in 2026 (without getting blocked) zoro-scraper-status
Ebay.com How to Scrape Ebay Using Python (2025 Update) ebay-scraper-status
## Fair Use and Legal Disclaimer This repository contains _educational_ reference material to illustrate how accessible web scraping can be and the provided programs are not intended to be used in web scraping production. That being said, Scrapfly team is constantly updating and improving all of this code for optimal experience. Scrapfly does not offer legal advice and as always, consult a lawyer when creating programs that interact with other people's websites though here's a good general intro of what NOT to do: - Do not store PII (personally identifiable information) of EU citizens who are protected by GDPR. - Do not scrape and repurpose entire public datasets which can be protected by database protection laws in some countries. - Do not scrape at rates that could damage the website and scrape only publicly available data. ## License This repository is licensed under the [Non-Profit Open Software License 3.0](LICENSE).