A production-ready PySpark project template with medallion architecture, Python packaging, unit tests, integration tests, CI/CD automation, Databricks Asset Bundles, and DQX data quality framework.
GitHub repository with 71 stars and 31 forks.
Language: Python
Topics: data-engineering, databricks, github-actions, pyspark, medallion-architecture, unity-catalog, uv, databricks-asset-bundles, apache-spark, databricks-cli
Trending score 0.32, activity score 0.05, stars gained +1, forks gained +0.
2026-06-05: 71 stars and 31 forks.
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
GitHub repository with 45,695 stars and 17,175 forks.
Trending score: 1.23; stars gained: +18; forks gained: +20.
Language: Python
Topics: airflow, apache, apache-airflow, automation, dag, data-engineering
An orchestration platform for the development, production, and observation of data assets.
GitHub repository with 15,625 stars and 2,155 forks.
Trending score: 1.02; stars gained: +7; forks gained: +1.
Language: Python
Topics: data-pipelines, dagster, workflow, data-science, workflow-automation, python
🐚 Python-powered shell. Full-featured, cross-platform and AI-friendly.
GitHub repository with 9,496 stars and 729 forks.
Trending score: 0.98; stars gained: +7; forks gained: -1.
Language: Python
Topics: xonsh, devops, iterm2, data-engineering, security-automation, raspberry-pi
Zero-config entity resolution that scales from a CSV to 100M+ rows on a Ray cluster (verified: 100M deduped in 213s, 0.30 GB driver). Fuzzy + exact + probabilistic dedupe, identity graph, PPRL, LLM boost. Python + full TypeScript port; SQL-native in PostgreSQL & DuckDB; MCP/REST servers, dbt + Airflow recipes.
GitHub repository with 79 stars and 10 forks.
Trending score: 0.94; stars gained: +8; forks gained: +1.
Language: Python
Topics: active-learning, agent, airflow, auto-config, data-engineering, data-quality
Always know what to expect from your data.
GitHub repository with 11,542 stars and 1,755 forks.
Trending score: 0.73; stars gained: +3; forks gained: +0.
Language: Python
Topics: pipeline-tests, dataquality, datacleaning, datacleaner, data-science, data-profiling
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
GitHub repository with 21,389 stars and 5,204 forks.
Trending score: 0.55; stars gained: +2; forks gained: +4.
Language: Python
Topics: data, pipeline, data-analysis, data-engineering, java, python
The agent that grows with you
GitHub repository with 181,384 stars and 31,127 forks.
Trending score: 5.95; stars gained: +1,867; forks gained: +361.
Language: Python
Topics: ai, ai-agent, ai-agents, anthropic, chatgpt, claude
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 60-95% fewer tokens, same answers. Library, proxy, MCP server.
GitHub repository with 12,942 stars and 833 forks.
Trending score: 5.69; stars gained: +2,829; forks gained: +175.
Language: Python
Topics: agent, ai, anthropic, claude-code, compression, context-engineering
Academic Research Skills for Claude Code: research → write → review → revise → finalize
GitHub repository with 27,327 stars and 2,249 forks.
Trending score: 5.52; stars gained: +1,079; forks gained: +89.
Language: Python
Topics: academic-pipeline, academic-writing, ai-research, claude, claude-code, literature-review
GitHub repository with 30,002 stars and 4,224 forks.
Trending score: 4.88; stars gained: +688; forks gained: +114.
Language: Python
Turn any technical book PDF into a Claude Code skill — ready to study, reference, and use while you work.
GitHub repository with 4,221 stars and 528 forks.
Trending score: 4.88; stars gained: +476; forks gained: +68.
Language: Python
An opinionated list of Python frameworks, libraries, tools, and resources
GitHub repository with 301,341 stars and 28,044 forks.
Trending score: 4.60; stars gained: +518; forks gained: +24.
Language: Python
Topics: awesome, python, collections, python-frameworks, python-libraries, python-tools
ktx is an executable context layer for data and analytics agents 🐙 Allow Claude Code, Codex, and any AI agent to query data accurately through MCP with skills, memory and a semantic layer
GitHub repository with 880 stars and 45 forks.
Trending score: 2.76; stars gained: +24; forks gained: +1.
Language: TypeScript
Topics: agent, agent-skills, agents, ai-agent, ai-agents, analytics
Apache Superset is a Data Visualization and Data Exploration Platform
GitHub repository with 73,178 stars and 17,519 forks.
Trending score: 2.73; stars gained: +24; forks gained: +19.
Language: TypeScript
Topics: analytics, apache, apache-superset, asf, bi, business-analytics
Local-first ETL/ELT studio: a drag-and-drop visual pipeline designer that compiles to SQL and runs on DuckDB. Tiny desktop app, no servers, git-friendly workspaces.
GitHub repository with 269 stars and 20 forks.
Trending score: 2.64; stars gained: +36; forks gained: +0.
Language: Rust
Topics: data-engineering, data-integration, data-pipeline, data-quality, desktop-app, drag-and-drop
Incremental engine for long horizon agents 🌟 Star if you like it!
GitHub repository with 10,181 stars and 799 forks.
Trending score: 2.57; stars gained: +20; forks gained: +3.
Language: Rust
Topics: agentic-data-framework, ai, ai-agents, change-data-capture, codebase-intelligence, context-engineering
SeaTunnel Web is a visual tool for building and watching over your Apache SeaTunnel data pipelines, with drag-and-drop DAGs and simple connector setup.
GitHub repository with 278 stars and 30 forks.
Trending score: 1.85; stars gained: +57; forks gained: +5.
Language: TypeScript
Topics: batch, dag, data-engineering, data-integration, data-pipeline, etl
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
GitHub repository with 9,062 stars and 776 forks.
Trending score: 1.62; stars gained: +2; forks gained: +0.
Language: Rust
Topics: apache-iceberg, data-engineering, database, etl-pipeline, event-streaming, kafka