semantic-deduplication repositories

Trending repositories tagged semantic-deduplication, ranked by trending score. Updated daily.

  1. 1
    NVIDIA-NeMo/Curator

    Scalable data pre processing and curation toolkit for LLMs

    GitHub · Python · 1,866 stars · 379 forks · +3 stars in 24 hours · Trending score 1.41

  2. 2
    MinishLab/semhash

    Fast Multimodal Semantic Deduplication & Filtering

    GitHub · Python · 978 stars · 56 forks · 0 stars in 24 hours · Trending score 0.00

  3. 3
    alphaparkinc/genpark-agent-semantic-deduplication-stream-filter-skill

    Streaming semantic text deduplicator using Jaccard token overlap and sliding window hashing to eliminate redundant search snippets

    GitHub · Python · 7 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00