etl repositories

Trending repositories tagged etl, ranked by trending score. Updated daily.

  1. 1
    datazip-inc/olake-go

    OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.

    GitHub · Go · 2,062 stars · 314 forks · +196 stars in 24 hours · Trending score 4.21

  2. 2
    apache/airflow

    Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

    GitHub · Python · 47,569 stars · 18,419 forks · +46 stars in 24 hours · Trending score 3.37

  3. 3
    kestra-io/kestra

    Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.

    GitHub · Java · 30,292 stars · 3,544 forks · +34 stars in 24 hours · Trending score 3.01

  4. 4
    vectordotdev/vector

    A high-performance observability data pipeline.

    GitHub · Rust · 22,918 stars · 2,405 forks · +22 stars in 24 hours · Trending score 2.78

  5. 5
    dagster-io/dagster

    An orchestration platform for the development, production, and observation of data assets.

    GitHub · Python · 16,444 stars · 2,427 forks · +9 stars in 24 hours · Trending score 2.36

  6. 6
    airbytehq/airbyte

    Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

    GitHub · Python · 22,411 stars · 5,550 forks · +5 stars in 24 hours · Trending score 2.32

  7. 7
    turbot/steampipe

    Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.

    GitHub · Go · 8,019 stars · 348 forks · +8 stars in 24 hours · Trending score 1.73

  8. 8
    morph-kgc/morph-kgc

    Powerful RDF knowledge graph generation with RML mappings

    GitHub · Python · 303 stars · 62 forks · +8 stars in 24 hours · Trending score 1.62

  9. 9
    apache/hop

    Hop Orchestration Platform

    GitHub · Java · 1,543 stars · 493 forks · +4 stars in 24 hours · Trending score 1.59

  10. 10
    ucbepic/docetl

    A system for agentic LLM-powered data processing and ETL

    GitHub · Python · 4,259 stars · 471 forks · +2 stars in 24 hours · Trending score 1.58

  11. 11
    apache/devlake

    Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

    GitHub · Go · 3,204 stars · 842 forks · +5 stars in 24 hours · Trending score 1.50

  12. 12
    datavane/tis

    Support agile Ontology DataOps Based on Flink, DataX and Flink-CDC with Web-UI

    GitHub · Java · 1,448 stars · 307 forks · +6 stars in 24 hours · Trending score 1.35

  13. 13
    cocoindex-io/cocoindex

    Incremental engine for long horizon agents 🌟 Star if you like it!

    GitHub · Rust · 11,918 stars · 930 forks · +3 stars in 24 hours · Trending score 1.34

  14. 14
    yuyuejia/datay

    DataY 是一套轻量、高性能的数据平台,一个 JAR、自研流批一体处理引擎,自研分布式调度框架,在 2 核 4G 的机器上 就能跑通 数据集成 → 数据建模 → 数据开发 → 任务调度 → 数据应用 的完整链路。

    GitHub · Java · 50 stars · 23 forks · +2 stars in 24 hours · Trending score 1.28

  15. 15
    reductstore/reductstore

    High-performance, time-indexed object storage for robotics and industrial IoT

    GitHub · Rust · 381 stars · 52 forks · +3 stars in 24 hours · Trending score 1.15

  16. 16
    vfaner/jync

    Jync 捷同 - 开箱即用的异构数据库实时同步工具。单 JAR 启动、浏览器配置,无需 Kafka/ZooKeeper;支持 MySQL、Oracle、PostgreSQL、达梦、金仓等 34 种数据库,内置结构同步、六类日志 CDC、ETL、告警与企业统一认证,内网离线可用。

    GitHub · Java · 93 stars · 25 forks · +1 stars in 24 hours · Trending score 1.12

  17. 17
    timeplus-io/proton

    The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.

    GitHub · C++ · 2,286 stars · 112 forks · +3 stars in 24 hours · Trending score 1.10

  18. 18
    galaxy-io/filament

    Pluggable data replication with checkpointing, batching, and integrity events

    GitHub · Go · 335 stars · 47 forks · 0 stars in 24 hours · Trending score 1.06

  19. 19
    redpanda-data/connect

    Fancy stream processing made operationally mundane

    GitHub · Go · 8,803 stars · 985 forks · +1 stars in 24 hours · Trending score 0.83

  20. 20
    nao1215/filesql

    loads CSV, TSV, LTSV, JSON, JSONL, Parquet, XLSX, ACH, and Fedwire files into SQLite; includes prep and frame for cleanup and in-memory transforms

    GitHub · Go · 388 stars · 12 forks · +1 stars in 24 hours · Trending score 0.82

  21. 21
    datazip-inc/olake-ui

    Frontend & BFF (Backend for frontend) for Olake. This includes the UI code and backend code for storing the configuration of sync and orchestrating it.

    GitHub · TypeScript · 41 stars · 44 forks · +2 stars in 24 hours · Trending score 0.81

  22. 22
    vaquarkhan/data-engineering-agent-skills

    Production-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.

    GitHub · Python · 65 stars · 9 forks · 0 stars in 24 hours · Trending score 0.80

  23. 23
    estuary/flow

    🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊

    GitHub · Rust · 992 stars · 101 forks · +1 stars in 24 hours · Trending score 0.73

  24. 24
    goldsky-io/streamling

    Data streaming runtime focused on performance, consistency, and extensibility. Write plugins in Rust or WASM and process data with data guarantees.

    GitHub · Rust · 111 stars · 9 forks · +1 stars in 24 hours · Trending score 0.73