High-performance open-source synthetic data engine. Uses LLMs for schema design and vectorized NumPy for deterministic, scalable generation.
GitHub repository with 57 stars and 3 forks.
Language: Python
Topics: data-engineering, data-generation, database-seeding, developer-tools, generative-ai, llm, mock-data, numpy, pandas, python
Trending score 0.05, activity score 0.05, stars gained +0, forks gained +0.
2026-06-05: 57 stars and 3 forks.
data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
GitHub repository with 5,436 stars and 523 forks.
Trending score: 0.98; stars gained: +8; forks gained: +0.
Language: Python
Topics: data, data-engineering, data-lake, data-loading, data-warehouse, elt
🐚 Python-powered shell. Full-featured, cross-platform and AI-friendly.
GitHub repository with 9,496 stars and 729 forks.
Trending score: 0.98; stars gained: +7; forks gained: -1.
Language: Python
Topics: xonsh, devops, iterm2, data-engineering, security-automation, raspberry-pi
Zero-config entity resolution that scales from a CSV to 100M+ rows on a Ray cluster (verified: 100M deduped in 213s, 0.30 GB driver). Fuzzy + exact + probabilistic dedupe, identity graph, PPRL, LLM boost. Python + full TypeScript port; SQL-native in PostgreSQL & DuckDB; MCP/REST servers, dbt + Airflow recipes.
GitHub repository with 81 stars and 10 forks.
Trending score: 0.94; stars gained: +8; forks gained: +1.
Language: Python
Topics: active-learning, agent, airflow, auto-config, data-engineering, data-quality
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
GitHub repository with 21,391 stars and 5,206 forks.
Trending score: 0.55; stars gained: +2; forks gained: +4.
Language: Python
Topics: bigquery, change-data-capture, data, data-analysis, data-collection, data-engineering
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
GitHub repository with 2,531 stars and 242 forks.
Trending score: 0.32; stars gained: +1; forks gained: +0.
Language: Python
Topics: connectors, data, data-engineering, data-pipelines, dataops, dataops-platform
Production-style real-time e-commerce lakehouse with Kafka, Airflow, Databricks, Medallion architecture, data quality, quarantine, Terraform, and Dash analytics.
GitHub repository with 27 stars and 0 forks.
Trending score: 0.31; stars gained: +1; forks gained: +0.
Language: Python
Topics: airflow, data-engineering, data-quality, databricks, delta-lake, kafka
The agent that grows with you
GitHub repository with 182,513 stars and 31,295 forks.
Trending score: 5.95; stars gained: +1,867; forks gained: +361.
Language: Python
Topics: ai, ai-agent, ai-agents, anthropic, chatgpt, claude
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 60-95% fewer tokens, same answers. Library, proxy, MCP server.
GitHub repository with 14,053 stars and 885 forks.
Trending score: 5.69; stars gained: +2,829; forks gained: +175.
Language: Python
Topics: agent, ai, anthropic, compression, context-engineering, context-window
Academic Research Skills for Claude Code: research → write → review → revise → finalize
GitHub repository with 27,616 stars and 2,272 forks.
Trending score: 5.52; stars gained: +1,079; forks gained: +89.
Language: Python
Topics: academic-pipeline, academic-writing, ai-research, claude, claude-code, literature-review
Learn it. Build it. Ship it for others.
GitHub repository with 28,711 stars and 4,695 forks.
Trending score: 5.32; stars gained: +1,261; forks gained: +238.
Language: Python
Topics: agents, ai, ai-agents, ai-engineering, computer-vision, course
An opinionated list of Python frameworks, libraries, tools, and resources
GitHub repository with 301,435 stars and 28,046 forks.
Trending score: 4.60; stars gained: +518; forks gained: +24.
Language: Python
Topics: awesome, collections, python, python-frameworks, python-libraries, python-tools
Use claude-code for free in the terminal, VSCode extension or discord like OpenClaw (voice supported)
GitHub repository with 32,540 stars and 4,942 forks.
Trending score: 4.56; stars gained: +467; forks gained: +82.
Language: Python
ktx is an executable context layer for data and analytics agents 🐙 Allow Claude Code, Codex, and any AI agent to query data accurately through MCP with skills, memory and a semantic layer
GitHub repository with 896 stars and 46 forks.
Trending score: 2.76; stars gained: +24; forks gained: +1.
Language: TypeScript
Topics: agent, agent-skills, agents, ai-agent, ai-agents, analytics
Local-first ETL/ELT studio: a drag-and-drop visual pipeline designer that compiles to SQL and runs on DuckDB. Tiny desktop app, no servers, git-friendly workspaces.
GitHub repository with 285 stars and 21 forks.
Trending score: 2.64; stars gained: +36; forks gained: +0.
Language: Rust
Topics: data-engineering, data-integration, data-pipeline, data-quality, desktop-app, drag-and-drop
SeaTunnel Web is a visual tool for building and watching over your Apache SeaTunnel data pipelines, with drag-and-drop DAGs and simple connector setup.
GitHub repository with 290 stars and 30 forks.
Trending score: 1.85; stars gained: +57; forks gained: +5.
Language: TypeScript
Topics: batch, dag, data-engineering, data-integration, data-pipeline, etl
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
GitHub repository with 9,065 stars and 776 forks.
Trending score: 1.62; stars gained: +2; forks gained: +0.
Language: Rust
Topics: apache-iceberg, data-engineering, database, etl-pipeline, event-streaming, kafka
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. The next cohort starts in January 2026. Join the course here 👇🏼
GitHub repository with 41,902 stars and 8,303 forks.
Trending score: 1.49; stars gained: +23; forks gained: +3.
Language: Jupyter Notebook
Topics: course, data-engineering, dbt, docker, free, kafka
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
GitHub repository with 5,547 stars and 483 forks.
Trending score: 1.26; stars gained: +2; forks gained: +1.
Language: Rust
Topics: ai-engineering, ai-pipeline, arrow, artificial-intelligence, big-data, data-engineering