data-preprocessing repositories

Trending repositories tagged data-preprocessing, ranked by trending score. Updated daily.

  1. 1
    wangxb96/Awesome-EdgeAI

    Resources of our survey paper "Optimizing Edge AI: A Comprehensive Survey on Data, Model, and System Strategies" and "Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deployment"

    GitHub · Python · 122 stars · 22 forks · +2 stars in 24 hours · Trending score 0.86

  2. 2
    nao1215/filesql

    loads CSV, TSV, LTSV, JSON, JSONL, Parquet, XLSX, ACH, and Fedwire files into SQLite; includes prep and frame for cleanup and in-memory transforms

    GitHub · Go · 388 stars · 12 forks · +1 stars in 24 hours · Trending score 0.82

  3. 3
    skrub-data/skrub

    Machine learning with dataframes

    GitHub · Python · 1,670 stars · 293 forks · +1 stars in 24 hours · Trending score 0.55

  4. 4
    Desbordante/desbordante-core

    Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.

    GitHub · C++ · 521 stars · 109 forks · 0 stars in 24 hours · Trending score 0.46

  5. 5
    MigoXLab/awesome-data-quality

    A comprehensive collection of data quality resources, tools, papers, and projects across various data types including traditional data, LLM pretraining/fine-tuning data, multimodal data, and more. Essential reference for researchers and practitioners in data-centric AI.

    GitHub · 32 stars · 27 forks · 0 stars in 24 hours · Trending score 0.28

  6. 6
    triton-inference-server/dali_backend

    The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.

    GitHub · C++ · 148 stars · 41 forks · 0 stars in 24 hours · Trending score 0.25

  7. 7
    Encephos/markdown-link-resolver

    A zero-dependency Python tool to resolve relative Markdown links and inline images as Base64 strings for RAG and Multimodal LLMs.

    GitHub · Python · 28 stars · 0 forks · 0 stars in 24 hours · Trending score 0.14

  8. 8
    Burakkylmz/ml-labs

    A hands-on Machine Learning laboratory built with Python, Scikit-learn, Pandas, NumPy, JupyterLab, Docker, and uv. Features reproducible, step-by-step labs covering data analysis, statistics, EDA, preprocessing, model training, evaluation, and forecasting.

    GitHub · Jupyter Notebook · 14 stars · 5 forks · 0 stars in 24 hours · Trending score 0.09

  9. 9
    hegongshan/Storage-for-AI-Paper

    Accelerating AI Training and Inference from Storage Perspective (Must-read Papers on Storage for AI)

    GitHub · 67 stars · 6 forks · 0 stars in 24 hours · Trending score 0.00

  10. 10
    data-prep-kit/data-prep-kit

    Open source project for data preparation for GenAI applications

    GitHub · HTML · 968 stars · 257 forks · 0 stars in 24 hours · Trending score 0.00

  11. 11
    FreshCode-Org/freshdata

    Automated, explainable data cleaning for pandas and Polars, repair messy tabular data safely and see exactly what changed.

    GitHub · Python · 6 stars · 20 forks · 0 stars in 24 hours · Trending score 0.00

  12. 12
    paypal/gators

    Lightning-fast data preprocessing and feature engineering for Python, built on Polars. 108 sklearn-style transformers for imputation, encoding, scaling, outlier clipping, discretization and feature selection, with Pipeline support and ONNX export for low-latency inference.

    GitHub · Python · 30 stars · 8 forks · 0 stars in 24 hours · Trending score 0.00

  13. 13
    AdArya125/Primer-to-Machine-Learning

    'Primer to Machine Learning' is a comprehensive guide covering essential topics in machine learning, including statistics, data preprocessing, supervised and unsupervised learning, neural networks, deep learning, NLP, time series analysis, and reinforcement learning. Perfect for beginners and intermediates.

    GitHub · Jupyter Notebook · 42 stars · 7 forks · 0 stars in 24 hours · Trending score 0.00