spider-rs/spider

Low latency web data collector

GitHub repository with 2,523 stars and 207 forks.

Language: Rust

Topics: crawler, rust, spider, headless-chrome, scraping, automation, ai-agent, web-crawler, web-scraping

Open provider repository

24h trend summary

Trending score 0.79, activity score 1.34, stars gained +3, forks gained +0.

Latest metric snapshot

2026-06-05: 2,523 stars and 207 forks.

Similar repositories

  1. 1. 0x676e67/wreq

    An ergonomic Rust HTTP Client with TLS fingerprint

    GitHub repository with 842 stars and 109 forks.

    Trending score: 1.21; stars gained: +3; forks gained: +0.

    Language: Rust

    Topics: akamai, client, crawler, fingerprint, http, http-client

  2. 2. 0xMassi/webclaw

    Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

    GitHub repository with 1,288 stars and 148 forks.

    Trending score: 1.20; stars gained: +17; forks gained: +0.

    Language: Rust

    Topics: ai-agents, cli, llm, markdown, mcp, rust

  3. 3. spider-rs/spider

    Low latency web data collector

    GitHub repository with 2,523 stars and 207 forks.

    Trending score: 0.79; stars gained: +3; forks gained: +0.

    Language: Rust

    Topics: crawler, rust, spider, headless-chrome, scraping, automation

  4. 4. us/crw

    Fast, lightweight Firecrawl alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/v1/scrape, /v1/crawl, /v1/search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

    GitHub repository with 138 stars and 12 forks.

    Trending score: 0.78; stars gained: +3; forks gained: +2.

    Language: Rust

    Topics: ai, ai-agents, crawler, data-extraction, docker, firecrawl

  5. 5. 0x676e67/wreq-python

    An ergonomic Python HTTP Client with TLS fingerprint

    GitHub repository with 1,366 stars and 106 forks.

    Trending score: 0.05.

    Language: Rust

    Topics: akamai, fingerprint, http, http-client, http2, https

Trending in Rust

  1. 1. BigPizzaV3/CodexPlusPlus

    An enhanced tool for CodexApp, striving to make Codex better to use and more comfortable 一个CodexApp的增强工具，努力让Codex变得更好用更舒服

    GitHub repository with 13,799 stars and 857 forks.

    Trending score: 5.16; stars gained: +916; forks gained: +44.

    Language: Rust

  2. 2. Hmbown/CodeWhale

    DeepSeek + MiMo coding agent in terminal

    GitHub repository with 37,132 stars and 3,195 forks.

    Trending score: 4.80; stars gained: +393; forks gained: +32.

    Language: Rust

    Topics: cli, deepseek, llm, rust, terminal, tui

  3. 3. openai/codex

    Lightweight coding agent that runs in your terminal

    GitHub repository with 88,842 stars and 13,055 forks.

    Trending score: 4.58; stars gained: +326; forks gained: +48.

    Language: Rust

  4. 4. tinyhumansai/openhuman

    Your Personal AI super intelligence. Private, Simple and extremely powerful.

    GitHub repository with 30,826 stars and 2,977 forks.

    Trending score: 4.37; stars gained: +332; forks gained: +50.

    Language: Rust

  5. 5. fallow-rs/fallow

    Codebase intelligence for TypeScript and JavaScript. Free static layer: unused code, duplication, circular deps, complexity hotspots, architecture boundaries. Optional paid runtime layer: hot-path review and cold-path deletion evidence from real production traffic. Rust-native, sub-second, zero-config framework support.

    GitHub repository with 3,058 stars and 94 forks.

    Trending score: 4.05; stars gained: +346; forks gained: +16.

    Language: Rust

    Topics: cli, code-duplication, code-quality, codebase-intelligence, copy-paste-detection, dead-code

  6. 6. aaif-goose/goose

    an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM

    GitHub repository with 46,568 stars and 4,863 forks.

    Trending score: 3.80; stars gained: +152; forks gained: +28.

    Language: Rust

    Topics: acp, ai, ai-agents, mcp

Trending topic: crawler

  1. 1. firecrawl/firecrawl

    The API to search, scrape, and interact with the web at scale. 🔥

    GitHub repository with 128,853 stars and 7,670 forks.

    Trending score: 4.80; stars gained: +954; forks gained: +49.

    Language: TypeScript

    Topics: ai, ai-agents, ai-crawler, ai-scraping, ai-search, crawler

  2. 2. CharlesPikachu/videodl

    Videodl: A lightweight video downloader written in pure python. (轻量级视频下载器，优先高清无水印，支持抖音，快手，小红书，B站，TikTok，YouTube，FIFA+，优酷，腾讯，爱奇艺，1905电影网，乐视，芒果，咪咕，PPTV，搜狐，Facebook，Twitter，新浪微博，今日头条，网易公开课，全民K歌，CCTV央视频，酷狗音乐MV，新片场，知乎，百度贴吧，TED等海量流媒体平台)

    GitHub repository with 2,207 stars and 397 forks.

    Trending score: 2.34; stars gained: +24; forks gained: +2.

    Language: Python

    Topics: video-downloader, cntv, douyin, crawler, migu, bilibili

  3. 3. ScrapeGraphAI/Scrapegraph-ai

    Python scraper based on AI

    GitHub repository with 26,738 stars and 2,486 forks.

    Trending score: 1.91; stars gained: +105; forks gained: +7.

    Language: Python

    Topics: ai-crawler, ai-scraping, ai-search, crawler, data-extraction, firecrawl-alternative

  4. 4. apify/crawlee

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

    GitHub repository with 23,677 stars and 1,417 forks.

    Trending score: 1.27; stars gained: +20; forks gained: +0.

    Language: TypeScript

    Topics: apify, automation, crawler, crawling, headless, headless-chrome

  5. 5. 0x676e67/wreq

    An ergonomic Rust HTTP Client with TLS fingerprint

    GitHub repository with 842 stars and 109 forks.

    Trending score: 1.21; stars gained: +3; forks gained: +0.

    Language: Rust

    Topics: akamai, client, crawler, fingerprint, http, http-client

  6. 6. 0xMassi/webclaw

    Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

    GitHub repository with 1,288 stars and 148 forks.

    Trending score: 1.20; stars gained: +17; forks gained: +0.

    Language: Rust

    Topics: ai-agents, cli, llm, markdown, mcp, rust