spider repositories
Trending repositories tagged spider, ranked by trending score. Updated daily.
-
1
spider-rs/spider
Foundational low latency web data collecting in Rust
GitHub · Rust · 2,855 stars · 330 forks · +24 stars in 24 hours · Trending score 2.48
-
2
CharlesPikachu/freeproxy
FreeProxy: Collecting free proxies from internet. (全球海量高质量免费代理,支持爬取数十个免费代理分享源,支持自定义规则代理筛选,爬虫与数据分析必备,每日更新海量免费代理。)
GitHub · Python · 592 stars · 103 forks · +12 stars in 24 hours · Trending score 1.86
-
3
cv-cat/catbus
一个命令操作小红书、抖音、TikTok、B 站、快手、微博、闲鱼、淘宝、京东、X:统一的命令与 JSON 输出,给人和 AI Agent 用
GitHub · TypeScript · 631 stars · 195 forks · +3 stars in 24 hours · Trending score 1.62
-
4
PhiFever/AfdianToMarkdown
爱发电爬虫(afdian.com)
GitHub · Go · 81 stars · 16 forks · +4 stars in 24 hours · Trending score 1.30
-
5
NaiboWang/EasySpider
A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/网页爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。
GitHub · JavaScript · 44,798 stars · 5,394 forks · +2 stars in 24 hours · Trending score 1.11
-
6
DedSecInside/TorBot
Dark Web OSINT Tool
GitHub · Python · 5,197 stars · 780 forks · 0 stars in 24 hours · Trending score 1.03
-
7
niubihu1/tvbox
🐯 牛虎 ┃ TVBox 旗舰聚合接口 · 4K原画秒播 · 纯净零广告 · 2000+高清电视直播(含EPG台标) · 影音壳单仓与多仓精选 · 双核爬虫驱动 · 大数据共识优选 · 每日自动巡检保活与去死链
GitHub · JavaScript · 237 stars · 53 forks · 0 stars in 24 hours · Trending score 0.74
-
8
signal18/replication-manager
Signal 18 repman - Replication Manager for MySQL / MariaDB / Percona Server
GitHub · Go · 747 stars · 177 forks · +1 stars in 24 hours · Trending score 0.63
-
9
scrapfly/scrapfly-scrapers
Scalable Python web scraping scripts for +80 popular domains
GitHub · Python · 1,131 stars · 208 forks · 0 stars in 24 hours · Trending score 0.55
-
10
scrapfly/Antibot-Detector
Real-time detection of anti-bot systems, captchas & fingerprinting techniques. Identifies Cloudflare, Akamai, DataDome, reCAPTCHA, hCaptcha, Shape Security & more with confidence scoring and advanced capture tools.
GitHub · JavaScript · 632 stars · 74 forks · 0 stars in 24 hours · Trending score 0.50
-
11
ndgigliotti/shopify-spy
Python library and CLI for scraping products, images and collections from Shopify and WooCommerce stores.
GitHub · Python · 124 stars · 54 forks · 0 stars in 24 hours · Trending score 0.48
-
12
alltheplaces/alltheplaces
A set of spiders and scrapers to extract location information from places that post their location on the internet.
GitHub · Python · 852 stars · 311 forks · 0 stars in 24 hours · Trending score 0.38
-
13
BlessedRebuS/Krawl
Krawl is a customizable, lightweight, cloud-native web deception server and anti-crawler that creates fake web applications with low-hanging vulnerabilities using realistic, randomly generated decoy data and AI-generated HTML templates.
GitHub · Python · 841 stars · 77 forks · 0 stars in 24 hours · Trending score 0.34
-
14
polyrabbit/hacker-news-digest
:newspaper: Let LLM Summarize Hacker News for You
GitHub · Python · 754 stars · 94 forks · 0 stars in 24 hours · Trending score 0.21
-
15
qnxg/hnu_query
湖南大学校内系统查询库,将湖南大学的校内系统(如个人门户,教务系统等)的接口进行包装,提供一套获取结构化数据的能力。
GitHub · Rust · 21 stars · 6 forks · 0 stars in 24 hours · Trending score 0.15
-
16
postmodern/spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
GitHub · Ruby · 837 stars · 105 forks · 0 stars in 24 hours · Trending score 0.14
-
17
yutao8/starred
GitHub Star 收藏资源库,自动同步个人收藏项目。收录 2000+ 个精选开源项目,覆盖开发框架、组件、SDK、模板、API、脚本、工具及各类实用资源,帮助开发者快速发现优质项目。
GitHub · PHP · 176 stars · 22 forks · 0 stars in 24 hours · Trending score 0.10
-
18
elixir-crawly/crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
GitHub · Elixir · 1,129 stars · 127 forks · −2 stars in 24 hours · Trending score 0.10
-
19
Norconex/crawler
Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or filesystem to various data repositories such as search engines.
GitHub · Java · 205 stars · 71 forks · 0 stars in 24 hours · Trending score 0.09
-
20
hupo376787/HelloCrab
A smart tool to download social media files. For learning only.
GitHub · C# · 34 stars · 6 forks · 0 stars in 24 hours · Trending score 0.09
-
21
RavelloH/PSGameSpider
自动爬取所有PlayStationStore中的所有游戏信息,包括封面、描述、价格、评分等,生成网页并索引 # # # Automatically crawl all game infos in all playstationstore, automatically generate web pages and index them
GitHub · JavaScript · 73 stars · 6 forks · 0 stars in 24 hours · Trending score 0.08
-
22
yagami-hue/MisaTV
轻量、本地优先的 TVBox 客户端(Flutter 重写自 Win-box):支持 jar / js / py 三类蜘蛛、网盘取流、磁力 BT、弹幕、外挂字幕、直播 EPG 与 DLNA 投屏
GitHub · Dart · 111 stars · 24 forks · 0 stars in 24 hours · Trending score 0.07
-
23
lulu38m/spider-crawler
🕷️ A literal web crawler: drop a document or a link, watch a spider read it, get the gist. Procedural animation, 100% in your browser.
GitHub · TypeScript · 5 stars · 1 forks · 0 stars in 24 hours · Trending score 0.07
-
24
WuKongSecurity/SpiderBOX
SpiderBox - 虫盒 - 爬虫逆向资源导航站
GitHub · CSS · 159 stars · 29 forks · 0 stars in 24 hours · Trending score 0.07