scraper repositories
Trending repositories tagged scraper, ranked by trending score. Updated daily.
-
1
firecrawl/firecrawl
Knowledge infrastructure to make your agents smarter. Search, scrape and 550+ data providers through one API.
GitHub · TypeScript · 204,946 stars · 10,743 forks · +1,470 stars in 24 hours · Trending score 5.69
-
2
unclecode/crawl4ai
Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
GitHub · Python · 87,596 stars · 9,143 forks · +87 stars in 24 hours · Trending score 3.84
-
3
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
GitHub · TypeScript · 26,505 stars · 1,774 forks · +40 stars in 24 hours · Trending score 3.23
-
4
getmaxun/maxun
Turn any website into a structured API. Extract, automate, search and monitor the web.
GitHub · TypeScript · 18,532 stars · 1,596 forks · +28 stars in 24 hours · Trending score 3.20
-
5
huginn/huginn
Create agents that monitor and act on your behalf. Your agents are standing by!
GitHub · Ruby · 50,598 stars · 4,361 forks · +23 stars in 24 hours · Trending score 2.66
-
6
Thordata/awesome-free-proxy-list
Daily-verified free proxy list with GeoIP country, latency tiers, anonymity ratings, survival streaks, Clash/V2Ray subscriptions and a live dashboard. Runs entirely on GitHub Actions.
GitHub · Python · 66 stars · 16 forks · +11 stars in 24 hours · Trending score 1.88
-
7
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included.
GitHub · Python · 9,680 stars · 837 forks · +3 stars in 24 hours · Trending score 1.59
-
8
maximilianfeix/proxy-scraper
Free HTTP/SOCKS proxy list, rebuilt every hour, with the result of every check on record. CLI, Python API, rotating proxy server, MCP server.
GitHub · Python · 98 stars · 24 forks · +2 stars in 24 hours · Trending score 1.45
-
9
0x676e67/wreq
An ergonomic, privacy-aware Rust HTTP Client
GitHub · Rust · 1,116 stars · 148 forks · +3 stars in 24 hours · Trending score 1.41
-
10
oxylabs/how-to-scrape-google-jobs
Learn how to build your own Google Jobs scraper that simultaneously scrapes Google Jobs for multiple search queries and geo-locations with Python and Oxylabs’ Google Jobs Scraper API. https://oxylabs.io/blog/how-to-scrape-google-jobs
GitHub · Python · 1,770 stars · 9 forks · 0 stars in 24 hours · Trending score 1.36
-
11
coflyn/Mori
Client-side media downloader App and CLI built with Capacitor and Node.js.
GitHub · JavaScript · 472 stars · 67 forks · 0 stars in 24 hours · Trending score 1.33
-
12
NaiboWang/EasySpider
A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/网页爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。
GitHub · JavaScript · 44,799 stars · 5,393 forks · +2 stars in 24 hours · Trending score 1.11
-
13
yaroslav2901/OE_OUTAGE_DATA
Репозиторій містить JSON-дані про відключення електроенергії та згенеровані зображення розкладів для кожної групи та загальне зображення для всіх груп.
GitHub · 40 stars · 2 forks · +3 stars in 24 hours · Trending score 1.05
-
14
alex-on-ai/WebReaper
AI-native web scraper. Single binary with a bundled Claude Code skill. MIT-licensed alternative to Firecrawl.
GitHub · C# · 153 stars · 33 forks · +4 stars in 24 hours · Trending score 1.05
-
15
henrique-coder/perplexity-webui-scraper
Unofficial Python client, CLI, MCP server, and OpenAI-compatible REST API for Perplexity's WebUI.
GitHub · Python · 143 stars · 30 forks · +3 stars in 24 hours · Trending score 1.00
-
16
plhery/universal-parcel-scraper
Self-hosted parcel tracking: 105 carriers, a TypeScript library, CLI, and HTTP API.
GitHub · TypeScript · 14 stars · 7 forks · +1 stars in 24 hours · Trending score 1.00
-
17
guohuiyuan/go-novel-dl
go-novel-dl 是 Go 语言开发的多源小说下载器,支持 CLI 与 Web 界面,可跨站点并发搜索、一键下载并导出,已适配 ESJZone、SFACG、刺猬猫、若初文学等多个书源。novel-dl交流群:1094455212
GitHub · Go · 335 stars · 32 forks · 0 stars in 24 hours · Trending score 0.98
-
18
Solr159/JavBoss
开箱即用的本地 JAV/视频 刮削、管理、播放软件,支持命令行一键安装和 Docker 部署。只需简单添加目录,即可打造你的私人 JAV/视频 媒体库,带给你顶级的浏览体验,懒人必备。| Your local JAV/video manager.
GitHub · Go · 932 stars · 94 forks · 0 stars in 24 hours · Trending score 0.98
-
19
opensanctions/opensanctions
An open database of international sanctions data, persons of interest and politically exposed persons
GitHub · Python · 880 stars · 202 forks · +2 stars in 24 hours · Trending score 0.94
-
20
monbooru/monloader
Online media downloader for monbooru
GitHub · Go · 20 stars · 8 forks · 0 stars in 24 hours · Trending score 0.90
-
21
Hedon-Haven/Hedon-Haven
An adult content aggregator app.
GitHub · Dart · 97 stars · 11 forks · 0 stars in 24 hours · Trending score 0.90
-
22
TeamNewPipe/NewPipeExtractor
NewPipe's core library for extracting data from streaming sites
GitHub · Java · 2,066 stars · 633 forks · 0 stars in 24 hours · Trending score 0.89
-
23
ndenissov/HDRezka
An asynchronous Python client for parsing and interacting with the HDRezka streaming platform. Built for scalable backends, it features TLS impersonation for Cloudflare, catalog browsing, account management, and seamless stream or subtitle extraction. Perfect for developing custom video APIs and automated scrapers.
GitHub · Python · 14 stars · 3 forks · +2 stars in 24 hours · Trending score 0.88
-
24
lbesnard/bdnex
BDneX - BD metadata scrapper (bedetheque.com, bdfuge...) in ComicRack format for french comics
GitHub · Python · 18 stars · 2 forks · +2 stars in 24 hours · Trending score 0.88