webscraping repositories
Trending repositories tagged webscraping, ranked by trending score. Updated daily.
-
1
firecrawl/firecrawl
Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.
GitHub · TypeScript · 205,406 stars · 10,742 forks · +2,082 stars in 24 hours · Trending score 5.92
-
2
D4Vinci/Scrapling
🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
GitHub · Python · 104,687 stars · 10,932 forks · +448 stars in 24 hours · Trending score 5.23
-
3
ProxyShard/ShardBrowser
Free, open-source anti-detect browser launcher for web scraping and multi-accounting. By the ShardX team (formerly ProxyShard). Engine-level fingerprint spoofing in Chromium 158 (WebGL / WebGPU / Client Hints / fonts / TLS), 220+ device profiles bundled, stable QUIC + WebRTC over SOCKS5.
GitHub · TypeScript · 1,704 stars · 240 forks · +270 stars in 24 hours · Trending score 4.36
-
4
tiliondev/fortress
Stealth Chromium engine that stops scrapers and browser agents from getting blocked, with one line of code change.
GitHub · Python · 810 stars · 55 forks · +86 stars in 24 hours · Trending score 3.22
-
5
CloakHQ/CloakBrowser
Stealth Chromium that passes every bot detection test. Drop-in Playwright replacement with source-level fingerprint patches. 50+ tests passed.
GitHub · Python · 33,003 stars · 2,740 forks · +28 stars in 24 hours · Trending score 3.01
-
6
getmaxun/maxun
Turn any website into a structured API. Extract, automate, search and monitor the web.
GitHub · TypeScript · 18,533 stars · 1,597 forks · +17 stars in 24 hours · Trending score 3.00
-
7
huginn/huginn
Create agents that monitor and act on your behalf. Your agents are standing by!
GitHub · Ruby · 50,601 stars · 4,363 forks · +22 stars in 24 hours · Trending score 2.67
-
8
ovobrowsersupport/ovobrowser
源码级 Chromium 指纹内核,检测站按普通 Chrome 评分。Source-level Chromium fingerprint kernel that passes bot detection tests. Not JS injection — C++ patches compiled into the binary.
GitHub · 1,274 stars · 71 forks · +19 stars in 24 hours · Trending score 2.58
-
9
oss-invisible/playwright
invisible playwright: Playwright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
GitHub · Python · 2,612 stars · 279 forks · +20 stars in 24 hours · Trending score 2.36
-
10
pystardust/ani-cli
A cli tool to browse and play anime
GitHub · Shell · 14,142 stars · 899 forks · +31 stars in 24 hours · Trending score 2.35
-
11
itsOwen/CyberScraper-2077
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
GitHub · Python · 3,302 stars · 360 forks · +14 stars in 24 hours · Trending score 1.76
-
12
anaskhan96/soup
Web Scraper in Go, similar to BeautifulSoup
GitHub · Go · 2,302 stars · 180 forks · +5 stars in 24 hours · Trending score 1.36
-
13
cdpdriver/zendriver
A blazing fast, async-first, undetectable webscraping/web automation framework based on ultrafunkamsterdam/nodriver. Now with Docker support!
GitHub · Python · 1,483 stars · 110 forks · +1 stars in 24 hours · Trending score 0.89
-
14
wkoverfield/nba2kapi
NBA 2k API
GitHub · TypeScript · 22 stars · 5 forks · 0 stars in 24 hours · Trending score 0.79
-
15
felipeall/transfermarkt-api
API service to get data from Transfermarkt
GitHub · Python · 473 stars · 162 forks · 0 stars in 24 hours · Trending score 0.77
-
16
Vinyzu/cursory
Generate 100% human-realistic Mouse Trajectories with Timings.
GitHub · Python · 192 stars · 21 forks · 0 stars in 24 hours · Trending score 0.75
-
17
clearcotelabs/clearcote-browser
Open-source stealth Chromium with engine-level fingerprint spoofing - de-Googled, drop-in Playwright, fully buildable and verifiable from source.
GitHub · Python · 255 stars · 35 forks · 0 stars in 24 hours · Trending score 0.68
-
18
woob/woob
Web outside of Browsers (core, site modules and applications)
GitLab · 150 stars · 269 forks · 0 stars in 24 hours · Trending score 0.67
-
19
scrapfly/scrapfly-scrapers
Scalable Python web scraping scripts for +80 popular domains
GitHub · Python · 1,131 stars · 207 forks · 0 stars in 24 hours · Trending score 0.46
-
20
scrapfly/Antibot-Detector
Real-time detection of anti-bot systems, captchas & fingerprinting techniques. Identifies Cloudflare, Akamai, DataDome, reCAPTCHA, hCaptcha, Shape Security & more with confidence scoring and advanced capture tools.
GitHub · JavaScript · 632 stars · 74 forks · 0 stars in 24 hours · Trending score 0.45
-
21
alex-on-ai/WebReaper
AI-native web scraper. Single binary with a bundled Claude Code skill. MIT-licensed alternative to Firecrawl.
GitHub · C# · 153 stars · 33 forks · 0 stars in 24 hours · Trending score 0.32
-
22
openzim/python-scraperlib
Collection of Python code to re-use across Python-based scrapers
GitHub · Python · 33 stars · 34 forks · 0 stars in 24 hours · Trending score 0.30
-
23
junguler/m3u-audiobooks
collection of 100k+ audio books, radio porgrams, music etc from archive.org in a easy to listen m3u playlist format
GitHub · HTML · 33 stars · 5 forks · 0 stars in 24 hours · Trending score 0.24
-
24
DevEnterpriseSoftware/patchright-dotnet
Undetected .NET version of the Playwright testing and automation library.
GitHub · C# · 54 stars · 7 forks · 0 stars in 24 hours · Trending score 0.21