ai-scraping repositories

Trending repositories tagged ai-scraping, ranked by trending score. Updated daily.

  1. 1
    firecrawl/firecrawl

    Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

    GitHub · TypeScript · 206,507 stars · 10,795 forks · +2,005 stars in 24 hours · Trending score 5.95

  2. 2
    D4Vinci/Scrapling

    🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev

    GitHub · Python · 104,902 stars · 10,976 forks · +411 stars in 24 hours · Trending score 5.15

  3. 3
    itsOwen/CyberScraper-2077

    A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

    GitHub · Python · 3,302 stars · 360 forks · +14 stars in 24 hours · Trending score 1.76

  4. 4
    WeebDataHoarder/go-away

    [Mirror] Self-hosted abuse detection and rule enforcement against low-effort mass AI scraping and bots.

    GitHub · Go · 222 stars · 13 forks · +10 stars in 24 hours · Trending score 1.66

  5. 5
    ArchiveBox/abx-dl

    ⬇️ A simple all-in-one CLI tool to download EVERYTHING from a URL (like youtube-dl/yt-dlp, forum-dl, gallery-dl, simpler ArchiveBox). 🎭 Uses headless Chrome to get HTML, JS, CSS, images/video/audio/subtitles, PDFs, screenshots, article text, git repos, and more...

    GitHub · Python · 154 stars · 10 forks · +1 stars in 24 hours · Trending score 0.69

  6. 6
    0xMassi/webclaw

    Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

    GitHub · Rust · 2,442 stars · 245 forks · −3 stars in 24 hours · Trending score 0.10

  7. 7
    any4ai/AnyCrawl

    AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.

    GitHub · TypeScript · 3,528 stars · 372 forks · 0 stars in 24 hours · Trending score 0.03

  8. 8
    ScrapeGraphAI/Scrapegraph-ai

    Python scraper based on AI

    GitHub · Python · 32,885 stars · 3,360 forks · 0 stars in 24 hours · Trending score 0.00