common-crawl repositories

Trending repositories tagged common-crawl, ranked by trending score. Updated daily.

  1. 1
    commoncrawl/news-crawl

    News crawling with StormCrawler - stores content as WARC

    GitHub · Java · 383 stars · 44 forks · 0 stars in 24 hours · Trending score 0.11

  2. 2
    pucilpet/crawlgraph-mcp

    MCP server for the CrawlGraph backlink-intelligence API (Common Crawl webgraph). Backlink lookups + competitor gap analysis for Claude, Cursor, Cline, and any MCP client.

    GitHub · JavaScript · 8 stars · 1 forks · 0 stars in 24 hours · Trending score 0.05

  3. 3
    commoncrawl/cc-crawl-statistics

    Statistics of Common Crawl monthly archives mined from URL index files

    GitHub · Python · 233 stars · 19 forks · 0 stars in 24 hours · Trending score 0.00

  4. 4
    MigoXLab/dingo

    Dingo: A Comprehensive AI Data, Model and Application Quality Evaluation Tool

    GitHub · Python · 761 stars · 79 forks · 0 stars in 24 hours · Trending score 0.00

  5. 5
    commoncrawl/cc-webgraph

    Tools to construct and process Common Crawl webgraphs

    GitHub · Java · 118 stars · 7 forks · 0 stars in 24 hours · Trending score 0.00

  6. 6
    superdoc/docx-corpus

    The largest open corpus of classified docx documents

    GitHub · TypeScript · 69 stars · 8 forks · 0 stars in 24 hours · Trending score 0.00