etl repositories
Trending repositories tagged etl, ranked by trending score. Updated daily.
-
1
datazip-inc/olake-go
OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.
GitHub · Go · 2,062 stars · 314 forks · +196 stars in 24 hours · Trending score 4.21
-
2
apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
GitHub · Python · 47,569 stars · 18,419 forks · +46 stars in 24 hours · Trending score 3.37
-
3
kestra-io/kestra
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
GitHub · Java · 30,292 stars · 3,544 forks · +34 stars in 24 hours · Trending score 3.01
-
4
vectordotdev/vector
A high-performance observability data pipeline.
GitHub · Rust · 22,918 stars · 2,405 forks · +22 stars in 24 hours · Trending score 2.78
-
5
dagster-io/dagster
An orchestration platform for the development, production, and observation of data assets.
GitHub · Python · 16,444 stars · 2,427 forks · +9 stars in 24 hours · Trending score 2.36
-
6
airbytehq/airbyte
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
GitHub · Python · 22,411 stars · 5,550 forks · +5 stars in 24 hours · Trending score 2.32
-
7
turbot/steampipe
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
GitHub · Go · 8,019 stars · 348 forks · +8 stars in 24 hours · Trending score 1.73
-
8
morph-kgc/morph-kgc
Powerful RDF knowledge graph generation with RML mappings
GitHub · Python · 303 stars · 62 forks · +8 stars in 24 hours · Trending score 1.62
-
9
apache/hop
Hop Orchestration Platform
GitHub · Java · 1,543 stars · 493 forks · +4 stars in 24 hours · Trending score 1.59
-
10
ucbepic/docetl
A system for agentic LLM-powered data processing and ETL
GitHub · Python · 4,259 stars · 471 forks · +2 stars in 24 hours · Trending score 1.58
-
11
apache/devlake
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
GitHub · Go · 3,204 stars · 842 forks · +5 stars in 24 hours · Trending score 1.50
-
12
datavane/tis
Support agile Ontology DataOps Based on Flink, DataX and Flink-CDC with Web-UI
GitHub · Java · 1,448 stars · 307 forks · +6 stars in 24 hours · Trending score 1.35
-
13
cocoindex-io/cocoindex
Incremental engine for long horizon agents 🌟 Star if you like it!
GitHub · Rust · 11,918 stars · 930 forks · +3 stars in 24 hours · Trending score 1.34
-
14
yuyuejia/datay
DataY 是一套轻量、高性能的数据平台,一个 JAR、自研流批一体处理引擎,自研分布式调度框架,在 2 核 4G 的机器上 就能跑通 数据集成 → 数据建模 → 数据开发 → 任务调度 → 数据应用 的完整链路。
GitHub · Java · 50 stars · 23 forks · +2 stars in 24 hours · Trending score 1.28
-
15
reductstore/reductstore
High-performance, time-indexed object storage for robotics and industrial IoT
GitHub · Rust · 381 stars · 52 forks · +3 stars in 24 hours · Trending score 1.15
-
16
vfaner/jync
Jync 捷同 - 开箱即用的异构数据库实时同步工具。单 JAR 启动、浏览器配置,无需 Kafka/ZooKeeper;支持 MySQL、Oracle、PostgreSQL、达梦、金仓等 34 种数据库,内置结构同步、六类日志 CDC、ETL、告警与企业统一认证,内网离线可用。
GitHub · Java · 93 stars · 25 forks · +1 stars in 24 hours · Trending score 1.12
-
17
timeplus-io/proton
The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
GitHub · C++ · 2,286 stars · 112 forks · +3 stars in 24 hours · Trending score 1.10
-
18
galaxy-io/filament
Pluggable data replication with checkpointing, batching, and integrity events
GitHub · Go · 335 stars · 47 forks · 0 stars in 24 hours · Trending score 1.06
-
19
redpanda-data/connect
Fancy stream processing made operationally mundane
GitHub · Go · 8,803 stars · 985 forks · +1 stars in 24 hours · Trending score 0.83
-
20
nao1215/filesql
loads CSV, TSV, LTSV, JSON, JSONL, Parquet, XLSX, ACH, and Fedwire files into SQLite; includes prep and frame for cleanup and in-memory transforms
GitHub · Go · 388 stars · 12 forks · +1 stars in 24 hours · Trending score 0.82
-
21
datazip-inc/olake-ui
Frontend & BFF (Backend for frontend) for Olake. This includes the UI code and backend code for storing the configuration of sync and orchestrating it.
GitHub · TypeScript · 41 stars · 44 forks · +2 stars in 24 hours · Trending score 0.81
-
22
vaquarkhan/data-engineering-agent-skills
Production-grade Agent Skills for data engineering AI agents: 73 workflows, platform presets, safe backfill/replay, Kafka & Spark reliability, MCP observability, and VS Code/JetBrains installers.
GitHub · Python · 65 stars · 9 forks · 0 stars in 24 hours · Trending score 0.80
-
23
estuary/flow
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
GitHub · Rust · 992 stars · 101 forks · +1 stars in 24 hours · Trending score 0.73
-
24
goldsky-io/streamling
Data streaming runtime focused on performance, consistency, and extensibility. Write plugins in Rust or WASM and process data with data guarantees.
GitHub · Rust · 111 stars · 9 forks · +1 stars in 24 hours · Trending score 0.73