data-processing-pipelines repositories

Trending repositories tagged data-processing-pipelines, ranked by trending score. Updated daily.

  1. 1
    edrewitz/WxData

    A Python package & CLI tool that assists users create automated ETL pipelines for various types of real-time and archived weather data & supports GRIB2 -> NETCDF. WxData can access 20+ Weather Models, Observations (METAR & RAWS), Radar Data, Observed Soundings, Fuels Data and interfaces for Open-Meteo and AirNow APIs.

    GitHub · Python · 29 stars · 2 forks · +2 stars in 24 hours · Trending score 0.81

  2. 2
    NVIDIA-NeMo/Curator

    Scalable data pre processing and curation toolkit for LLMs

    GitHub · Python · 1,866 stars · 379 forks · 0 stars in 24 hours · Trending score 0.72