# ScrapFly Scrapers 🕷️
**English** · [简体中文](docs/README.zh-CN.md) · [日本語](docs/README.ja.md) · [Español](docs/README.es.md) · [Русский](docs/README.ru.md)
This repository contains educational example scrapers for popular web scraping targets using the [ScrapFly](https://scrapfly.io) web scraping API and Python.
Most scrapers use a simple web scraping stack:
- Python version 3.10+
- [Scrapfly's Python SDK](https://github.com/scrapfly/python-scrapfly) for sending HTTP requests, getting past blocking and parsing the HTML with the built-in [parsel](https://pypi.org/project/parsel/) selector.
- [asyncio](https://docs.python.org/3/library/asyncio.html) for writing concurrent code using the async/await syntax.
- [JMESPath](https://pypi.org/project/jmespath/) and [nested-lookup](https://pypi.org/project/nested-lookup/) for JSON parsing when needed.
- [loguru](https://pypi.org/project/loguru/) for logging.
To learn more about web scraping, see our full tutorials on how to scrape these targets (and many others) in the [scrapeguide directory](https://scrapfly.io/blog/tag/scrapeguide/).
For AI agents and LLM tools, [llms.txt](llms.txt) lists every scraper with its functions and sample output, and explains how to run a scraper and how to write a new one.
## Quick start
Every scraper lives in its own folder and runs on its own (Python 3.10+ and [Poetry](https://python-poetry.org/docs/#installation)):
```shell
git clone https://github.com/scrapfly/scrapfly-scrapers.git
cd scrapfly-scrapers/airbnb-scraper # any scraper folder from the list below
export SCRAPFLY_KEY="YOUR SCRAPFLY KEY" # from https://scrapfly.io/dashboard
poetry install --no-root
poetry run python run.py # example scrape, saves its output in ./results
poetry run pytest test.py # the scraper's tests
```
Each scraper folder's README lists what it scrapes.
## List of Scrapers
Below is the list of available web scrapers for the supported domains along with their scrape guide, sample datasets, and status. 👇
The status comes from the nightly test run, which also orders the table: scrapers whose tests pass come first, failing and untested ones last.
A badge reads **passing** (every test passed), **N/M passing** (some tests passed), **failing** (none passed) or **not run**; click it to see that scraper's last results, with a link to each test's log.
| Domain |
Guide |
Sample Datasets |
Status |
| Air France |
How to Scrape Air France Flights with Python (2026) |
|
 |
| Airbnb.com |
How to Scrape Airbnb Listings and Prices (2026) |
|
 |
| Aliexpress.com |
How to Scrape Aliexpress.com (2025 Update) |
|
 |
| Allegro.pl |
How to Scrape Allegro.pl in 2026 (Without Getting Blocked) |
|
 |
| Amazon.com |
How to Scrape Amazon.com Product Data and Reviews |
|
 |
| App Store |
- |
|
 |
| AutoScout24.com |
How to Scrape AutoScout24 in 2026 |
|
 |
| BestBuy.com |
How to Scrape BestBuy Product, Offer and Review Data |
|
 |
| Bing.com |
How to Scrape Bing Search with Python |
|
 |
| Booking.com |
How to Scrape Booking.com (2025 Update) |
|
 |
| Capterra.com |
How to Scrape Capterra Software Reviews and Listings (2026) |
|
 |
| ChatGPT.com |
How to Scrape ChatGPT in 2026 |
|
 |
| Crunchbase.com |
How to Scrape Crunchbase in 2025 |
|
 |
| DHL |
How to Scrape DHL Parcel Tracking Data in 2026 |
|
 |
| DigiKey.com |
How to Scrape DigiKey Pricing, Stock, and Specs in 2026 |
|
 |
| Domain.com.au |
How to Scrape Domain.com.au Real Estate Property Data |
|
 |
| Emirates |
How to Scrape Emirates Flights with Python (2026) |
|
 |
| Etsy.com |
How to Scrape Etsy.com Product, Shop and Search Data |
|
 |
| Expedia.com |
- |
|
 |
| Facebook.com |
How to Scrape Facebook in 2026 |
|
 |
| Fashionphile.com |
How to Scrape Fashionphile for Second Hand Fashion Data |
|
 |
| G2.com |
How to Scrape G2 Company Data and Reviews |
|
 |
| Glassdoor.com |
How to Scrape Glassdoor (2025 update) |
|
 |
| Goat.com |
How to Scrape Goat.com for Fashion Apparel Data in Python |
|
 |
| Goibibo |
How to Scrape Goibibo Prices in 2026 |
|
 |
| Goodreads.com |
How to Scrape Goodreads for Book Data, Ratings, and Reviews |
|
 |
| Google Flights |
How to Scrape Google Flights Data in 2026 |
|
 |
| Google Jobs |
How to Scrape Google Jobs in 2026 |
|
 |
| Google Play |
How to Scrape Google Play App Reviews and Data |
|
 |
| Google.com |
How to Scrape Google Search Results - How to Scrape Google Maps |
|
 |
| Homegate.ch |
How to Scrape Homegate.ch Real Estate Property Data |
|
 |
| Idealista.com |
How to Scrape Idealista.com in Python - Real Estate Property Data |
|
 |
| Idealo.de |
- |
|
 |
| IMDb.com |
How to Scrape IMDb Data, Ratings, and Featured Reviews |
|
 |
| Immobilienscout24.de |
How to Scrape Immobilienscout24.de Real Estate Data |
|
 |
| Immoscout24.ch |
How to Scrape Immoscout24.ch Real Estate Property Data |
|
 |
| Immowelt.de |
How to Scrape Immowelt.de Real Estate Data |
|
 |
| ImovelWeb.com.br |
How to Scrape Imovelweb Without Getting Blocked in 2026 |
|
 |
| Indeed.com |
How to Scrape Indeed.com (2025 Update) |
|
 |
| Instagram.com |
How to Scrape Instagram in 2025 |
|
 |
| Kayak |
How to Scrape Kayak Flight Prices in 2026 |
|
 |
| Leboncoin.fr |
How to Web Scrape Leboncoin.fr using Python |
|
 |
| Linkedin.com |
How to Scrape LinkedIn in 2025 |
|
 |
| LoopNet.com |
- |
|
 |
| Lowes.com |
How to Scrape Lowe's Product and Pricing Data in 2026 |
|
 |
| Maersk |
How to Scrape Maersk Shipping Data in 2026 |
|
 |
| Marriott.com |
Scraping Marriott Hotel Prices and Availability (2026) |
|
 |
| Mouser.com |
How to Scrape Mouser.com in 2026 |
|
 |
| Naver.com |
How to Scrape Naver.com: 2026 Python Guide |
|
 |
| Nordstorm.com |
How to Scrape Nordstrom Fashion Product Data |
|
 |
| Perplexity.ai |
How to Scrape Perplexity AI in 2026 |
|
 |
| Pinterest.com |
How to Scrape Pinterest in 2026 |
|
 |
| Realestate.com.au |
How to Scrape Realestate.com.au Property Listing Data |
|
 |
| Realtor.com |
How to Scrape Realtor.com - Real Estate Property Data |
|
 |
| Reddit.com |
How to Scrape Reddit Posts, Subreddits and Profiles |
|
 |
| Redfin.com |
How to Scrape Redfin Real Estate Property Data in Python |
|
 |
| Rightmove.com |
How to Scrape RightMove Real Estate Property Data with Python |
|
 |
| Rs-online.com |
How to Scrape RS-Online (rs-online.com) in 2026 |
|
 |
| Seloger.com |
How to Scrape Seloger.com - Real Estate Listing Data |
|
 |
| Shopify stores |
How to Scrape Shopify Stores for Products, Prices, and Variants |
|
 |
| Similarweb.com |
How to Scrape SimilarWeb Website Traffic Analytics |
|
 |
| Skyscanner |
How to Scrape Skyscanner Flight Prices in Python (2026) |
|
 |
| Stockx.com |
How to Scrape StockX e-commerce Data with Python |
|
 |
| Target.com |
How to Scrape Target.com Product and Pricing Data |
|
 |
| Threads.net |
How to scrape Threads by Meta using Python (2025 Update) |
|
 |
| Ticketmaster.com |
How to Scrape Ticketmaster in 2026 |
|
 |
| TikTok.com |
How To Scrape TikTok in 2025 |
|
 |
| Tripadvisor.com |
How to Scrape TripAdvisor.com (2025 Updated) |
|
 |
| Trustpilot.com |
How to Scrape Trustpilot.com Reviews and Company Data |
|
 |
| Twitter(X).com |
How to Scrape X.com (Twitter) using Python (2025 Update) |
|
 |
| VestiaireCollective.com |
How to Scrape Vestiaire Collective for Fashion Product Data |
|
 |
| Walmart.com |
How to Scrape Walmart.com Product Data (2025 Update) |
|
 |
| Wellfound.com |
How to Scrape Wellfound Company Data and Job Listings |
|
 |
| Yellowpages.com |
How to Scrape YellowPages.com in 2025 |
|
 |
| Yelp.com |
How to Web Scrape Yelp.com (2025 update) |
|
 |
| YouTube.com |
How to Scrape YouTube in 2025 |
|
 |
| Zillow.com |
How to Scrape Zillow Real Estate Property Data in Python |
|
 |
| Zoominfo.com |
How to Scrape Zoominfo Company Data (2025 Update) |
|
 |
| Zoopla.co.uk |
How to Scrape Zoopla Real Estate Property Data in Python |
|
 |
| Zoro.com |
How to Scrape Zoro.com in 2026 (without getting blocked) |
|
 |
| Ebay.com |
How to Scrape Ebay Using Python (2025 Update) |
|
 |
## Fair Use and Legal Disclaimer
This repository contains _educational_ reference material to illustrate how accessible web scraping can be and the provided programs are not intended to be used in web scraping production.
That being said, Scrapfly team is constantly updating and improving all of this code for optimal experience.
Scrapfly does not offer legal advice and as always, consult a lawyer when creating programs that interact with other people's websites though here's a good general intro of what NOT to do:
- Do not store PII (personally identifiable information) of EU citizens who are protected by GDPR.
- Do not scrape and repurpose entire public datasets which can be protected by database protection laws in some countries.
- Do not scrape at rates that could damage the website and scrape only publicly available data.
## License
This repository is licensed under the [Non-Profit Open Software License 3.0](LICENSE).