webscraping repositories

Trending repositories tagged webscraping, ranked by trending score. Updated daily.

  1. 1
    firecrawl/firecrawl

    Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

    GitHub · TypeScript · 205,406 stars · 10,742 forks · +2,082 stars in 24 hours · Trending score 5.92

  2. 2
    D4Vinci/Scrapling

    🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

    GitHub · Python · 104,687 stars · 10,932 forks · +448 stars in 24 hours · Trending score 5.23

  3. 3
    ProxyShard/ShardBrowser

    Free, open-source anti-detect browser launcher for web scraping and multi-accounting. By the ShardX team (formerly ProxyShard). Engine-level fingerprint spoofing in Chromium 158 (WebGL / WebGPU / Client Hints / fonts / TLS), 220+ device profiles bundled, stable QUIC + WebRTC over SOCKS5.

    GitHub · TypeScript · 1,704 stars · 240 forks · +270 stars in 24 hours · Trending score 4.36

  4. 4
    tiliondev/fortress

    Stealth Chromium engine that stops scrapers and browser agents from getting blocked, with one line of code change.

    GitHub · Python · 810 stars · 55 forks · +86 stars in 24 hours · Trending score 3.22

  5. 5
    CloakHQ/CloakBrowser

    Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.

    GitHub · Python · 33,003 stars · 2,740 forks · +28 stars in 24 hours · Trending score 3.01

  6. 6
    getmaxun/maxun

    Turn any website into a structured API. Extract, automate, search and monitor the web.

    GitHub · TypeScript · 18,533 stars · 1,597 forks · +17 stars in 24 hours · Trending score 3.00

  7. 7
    huginn/huginn

    Create agents that monitor and act on your behalf. Your agents are standing by!

    GitHub · Ruby · 50,601 stars · 4,363 forks · +22 stars in 24 hours · Trending score 2.67

  8. 8
    ovobrowsersupport/ovobrowser

    源码级 Chromium 指纹内核,检测站按普通 Chrome 评分。Source-level Chromium fingerprint kernel that passes bot detection tests. Not JS injection — C++ patches compiled into the binary.

    GitHub · 1,274 stars · 71 forks · +19 stars in 24 hours · Trending score 2.58

  9. 9
    oss-invisible/playwright

    invisible playwright: Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.

    GitHub · Python · 2,612 stars · 279 forks · +20 stars in 24 hours · Trending score 2.36

  10. 10
    pystardust/ani-cli

    A cli tool to browse and play anime

    GitHub · Shell · 14,142 stars · 899 forks · +31 stars in 24 hours · Trending score 2.35

  11. 11
    itsOwen/CyberScraper-2077

    A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

    GitHub · Python · 3,302 stars · 360 forks · +14 stars in 24 hours · Trending score 1.76

  12. 12
    anaskhan96/soup

    Web Scraper in Go, similar to BeautifulSoup

    GitHub · Go · 2,302 stars · 180 forks · +5 stars in 24 hours · Trending score 1.36

  13. 13
    cdpdriver/zendriver

    A blazing fast, async-first, undetectable webscraping/web automation framework based on ultrafunkamsterdam/nodriver. Now with Docker support!

    GitHub · Python · 1,483 stars · 110 forks · +1 stars in 24 hours · Trending score 0.89

  14. 14
    wkoverfield/nba2kapi

    NBA 2k API

    GitHub · TypeScript · 22 stars · 5 forks · 0 stars in 24 hours · Trending score 0.79

  15. 15
    felipeall/transfermarkt-api

    API service to get data from Transfermarkt

    GitHub · Python · 473 stars · 162 forks · 0 stars in 24 hours · Trending score 0.77

  16. 16
    Vinyzu/cursory

    Generate 100% human-realistic Mouse Trajectories with Timings.

    GitHub · Python · 192 stars · 21 forks · 0 stars in 24 hours · Trending score 0.75

  17. 17
    clearcotelabs/clearcote-browser

    Open-source stealth Chromium with engine-level fingerprint spoofing - de-Googled, drop-in Playwright, fully buildable and verifiable from source.

    GitHub · Python · 255 stars · 35 forks · 0 stars in 24 hours · Trending score 0.68

  18. 18
    woob/woob

    Web outside of Browsers (core, site modules and applications)

    GitLab · 150 stars · 269 forks · 0 stars in 24 hours · Trending score 0.67

  19. 19
    scrapfly/scrapfly-scrapers

    Scalable Python web scraping scripts for +80 popular domains

    GitHub · Python · 1,131 stars · 207 forks · 0 stars in 24 hours · Trending score 0.46

  20. 20
    scrapfly/Antibot-Detector

    Real-time detection of anti-bot systems, captchas & fingerprinting techniques. Identifies Cloudflare, Akamai, DataDome, reCAPTCHA, hCaptcha, Shape Security & more with confidence scoring and advanced capture tools.

    GitHub · JavaScript · 632 stars · 74 forks · 0 stars in 24 hours · Trending score 0.45

  21. 21
    alex-on-ai/WebReaper

    AI-native web scraper. Single binary with a bundled Claude Code skill. MIT-licensed alternative to Firecrawl.

    GitHub · C# · 153 stars · 33 forks · 0 stars in 24 hours · Trending score 0.32

  22. 22
    openzim/python-scraperlib

    Collection of Python code to re-use across Python-based scrapers

    GitHub · Python · 33 stars · 34 forks · 0 stars in 24 hours · Trending score 0.30

  23. 23
    junguler/m3u-audiobooks

    collection of 100k+ audio books, radio porgrams, music etc from archive.org in a easy to listen m3u playlist format

    GitHub · HTML · 33 stars · 5 forks · 0 stars in 24 hours · Trending score 0.24

  24. 24
    DevEnterpriseSoftware/patchright-dotnet

    Undetected .NET version of the Playwright testing and automation library.

    GitHub · C# · 54 stars · 7 forks · 0 stars in 24 hours · Trending score 0.21