data-preparation repositories

Trending repositories tagged data-preparation, ranked by trending score. Updated daily.

  1. 1
    NVIDIA-NeMo/Curator

    Scalable data pre processing and curation toolkit for LLMs

    GitHub · Python · 1,866 stars · 379 forks · 0 stars in 24 hours · Trending score 0.72

  2. 2
    OpenDCAI/DataMind

    All-in-one intelligent assistant powered by LlamaIndex — RAG, GraphRAG, NL2SQL, Skills & Memory with multimodal support.

    GitHub · Python · 456 stars · 60 forks · +1 stars in 24 hours · Trending score 0.60

  3. 3
    Sriram-PR/doc-scraper

    Go web crawler to scrape documentation sites and convert content to clean Markdown for LLM ingestion (RAG, training data)

    GitHub · Go · 110 stars · 25 forks · 0 stars in 24 hours · Trending score 0.36

  4. 4
    skrub-data/skrub

    Machine learning with dataframes

    GitHub · Python · 1,670 stars · 293 forks · 0 stars in 24 hours · Trending score 0.22

  5. 5
    ciaren-labs/Ciaren

    Open-source, local-first visual ETL and data pipeline builder. Drag-and-drop data prep and ML workflows, preview every step, schedule runs, and export clean pandas or Polars Python code. Self-hosted, no lock-in.

    GitHub · Python · 37 stars · 12 forks · 0 stars in 24 hours · Trending score 0.08

  6. 6
    hegongshan/Storage-for-AI-Paper

    Accelerating AI Training and Inference from Storage Perspective (Must-read Papers on Storage for AI)

    GitHub · 67 stars · 6 forks · 0 stars in 24 hours · Trending score 0.07

  7. 7
    hugues31/duckpad

    The data transformation tool the open-source community deserves.

    GitHub · Python · 301 stars · 10 forks · −6 stars in 24 hours · Trending score 0.07

  8. 8
    HKUSTDial/awesome-data-agents

    Continuously updated paper list on advancements in Data Agents. Companion repo to our paper "A Survey of Data Agents: Emerging Paradigm or Overstated Hype?"

    GitHub · Python · 800 stars · 62 forks · 0 stars in 24 hours · Trending score 0.00

  9. 9
    data-prep-kit/data-prep-kit

    Open source project for data preparation for GenAI applications

    GitHub · HTML · 968 stars · 257 forks · 0 stars in 24 hours · Trending score 0.00