🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
GitHub repository with 60,933 stars and 5,873 forks.
Language: Python
Topics: crawler, crawling, crawling-python, playwright, python, scraping, selectors, stealth, web-scraper, web-scraping
2026-06-05: 60,933 stars and 5,873 forks.
Videodl: A lightweight video downloader written in pure python. (轻量级视频下载器，优先高清无水印，支持抖音，快手，小红书，B站，TikTok，YouTube，FIFA+，优酷，腾讯，爱奇艺，1905电影网，乐视，芒果，咪咕，PPTV，搜狐，Facebook，Twitter，新浪微博，今日头条，网易公开课，全民K歌，CCTV央视频，酷狗音乐MV，新片场，知乎，百度贴吧，TED等海量流媒体平台)
GitHub repository with 2,207 stars and 397 forks.
Trending score: 2.34; stars gained: +24; forks gained: +2.
Language: Python
Topics: video-downloader, cntv, douyin, crawler, migu, bilibili
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
GitHub repository with 9,144 stars and 747 forks.
Trending score: 1.08; stars gained: +7; forks gained: -1.
Language: Python
Topics: apify, automation, beautifulsoup, crawler, crawling, headless
FreeProxy: Collecting free proxies from internet. (全球海量高质量免费代理，支持爬取数十个免费代理分享源，支持自定义规则代理筛选，爬虫与数据分析必备，每日更新海量免费代理。)
GitHub repository with 452 stars and 79 forks.
Trending score: 0.60; stars gained: +3; forks gained: +0.
Language: Python
Topics: proxy-sources, 996icu, crawler, icu, proxy, proxy-server
🕷 Automatically detect changes made to the official Telegram sites, clients and servers.
GitHub repository with 351 stars and 46 forks.
Trending score: 0.41; stars gained: +1; forks gained: +0.
Language: Python
Topics: crawler, parser, telegram, telegram-org, telegram-updates, crawling
Flarecrawl CLI — Cloudflare Browser Rendering alternative to Firecrawl, cost-efficient at scale
GitHub repository with 6 stars and 0 forks.
Trending score: 0.33; stars gained: +1; forks gained: +0.
Language: Python
Topics: browser-rendering, cli, cloudflare, crawler, favicon, firecrawl
The definitive list of the latest libraries, tools, APIs and providers for web scraping. The only daily-updated collection of web scraping resources.
GitHub repository with 9 stars and 2 forks.
Trending score: 0.18; stars gained: +0; forks gained: +0.
Language: Python
Topics: awesome, scraper, scraping, webscraping, crawler, web-scraping
The agent that grows with you
GitHub repository with 181,594 stars and 31,156 forks.
Trending score: 5.95; stars gained: +1,867; forks gained: +361.
Language: Python
Topics: ai, ai-agent, ai-agents, anthropic, chatgpt, claude
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 60-95% fewer tokens, same answers. Library, proxy, MCP server.
GitHub repository with 13,361 stars and 853 forks.
Trending score: 5.69; stars gained: +2,829; forks gained: +175.
Language: Python
Topics: agent, ai, anthropic, compression, context-engineering, context-window
Academic Research Skills for Claude Code: research → write → review → revise → finalize
GitHub repository with 27,422 stars and 2,253 forks.
Trending score: 5.52; stars gained: +1,079; forks gained: +89.
Language: Python
Topics: academic-pipeline, academic-writing, ai-research, claude, claude-code, literature-review
GitHub repository with 30,002 stars and 4,224 forks.
Trending score: 4.88; stars gained: +688; forks gained: +114.
Language: Python
Turn any technical book PDF into a Claude Code skill — ready to study, reference, and use while you work.
GitHub repository with 4,250 stars and 534 forks.
Trending score: 4.88; stars gained: +476; forks gained: +68.
Language: Python
An opinionated list of Python frameworks, libraries, tools, and resources
GitHub repository with 301,371 stars and 28,044 forks.
Trending score: 4.60; stars gained: +518; forks gained: +24.
Language: Python
Topics: awesome, python, collections, python-frameworks, python-libraries, python-tools
The API to search, scrape, and interact with the web at scale. 🔥
GitHub repository with 128,853 stars and 7,670 forks.
Trending score: 4.80; stars gained: +954; forks gained: +49.
Language: TypeScript
Topics: ai, ai-agents, ai-crawler, ai-scraping, ai-search, crawler
Videodl: A lightweight video downloader written in pure python. (轻量级视频下载器，优先高清无水印，支持抖音，快手，小红书，B站，TikTok，YouTube，FIFA+，优酷，腾讯，爱奇艺，1905电影网，乐视，芒果，咪咕，PPTV，搜狐，Facebook，Twitter，新浪微博，今日头条，网易公开课，全民K歌，CCTV央视频，酷狗音乐MV，新片场，知乎，百度贴吧，TED等海量流媒体平台)
GitHub repository with 2,207 stars and 397 forks.
Trending score: 2.34; stars gained: +24; forks gained: +2.
Language: Python
Topics: video-downloader, cntv, douyin, crawler, migu, bilibili
An ergonomic Rust HTTP Client with TLS fingerprint
GitHub repository with 842 stars and 109 forks.
Trending score: 1.21; stars gained: +3; forks gained: +0.
Language: Rust
Topics: akamai, client, crawler, fingerprint, http, http-client
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
GitHub repository with 1,287 stars and 148 forks.
Trending score: 1.20; stars gained: +17; forks gained: +0.
Language: Rust
Topics: ai-agents, cli, llm, markdown, mcp, rust
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
GitHub repository with 9,144 stars and 747 forks.
Trending score: 1.08; stars gained: +7; forks gained: -1.
Language: Python
Topics: apify, automation, beautifulsoup, crawler, crawling, headless
Low latency web data collector
GitHub repository with 2,523 stars and 207 forks.
Trending score: 0.79; stars gained: +3; forks gained: +0.
Language: Rust
Topics: crawler, rust, spider, headless-chrome, scraping, automation