Reliable Python message queuing with Redis and built-in deduplication. Publish once, process once, recover from crashes - across any number of producers and consumers.
GitHub repository with 5 stars and 1 forks.
Language: Python
Topics: data-processing, deduplication, exactly-once, exactly-once-processing, horizontal-scaling, idempotent, message-acknowledgement, message-broker, message-queue, pub-sub
2026-06-05: 5 stars and 1 forks.
Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
GitHub repository with 6,487 stars and 373 forks.
Trending score: 1.18; stars gained: +16; forks gained: -1.
Language: Python
Topics: data, data-analysis, data-pipeline, data-processing, data-science, data-visualization
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
GitHub repository with 63,100 stars and 1,679 forks.
Trending score: 0.46; stars gained: -30; forks gained: +4.
Language: Python
Topics: batch-processing, data-analytics, data-pipelines, data-processing, dataflow, etl
Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.
GitHub repository with 1,208 stars and 60 forks.
Trending score: 0.30; stars gained: +1; forks gained: +0.
Language: Python
Topics: artificial-intelligence, aws, data-science, gcp, machine-learning, python
A light-weight, flexible, and expressive statistical data testing library
GitHub repository with 4,361 stars and 401 forks.
Trending score: 0.28; stars gained: +0; forks gained: +0.
Language: Python
Topics: pandas, validation, schema, dataframes, testing, pandas-validation
Open pixelated STEM framework
GitHub repository with 126 stars and 73 forks.
Trending score: 0.28; stars gained: +0; forks gained: +0.
Language: Python
Topics: electron-microscopy, data-processing, image-processing, python
A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀
GitHub repository with 365 stars and 122 forks.
Trending score: 0.09; stars gained: +0; forks gained: +0.
Language: Python
Topics: python, big-data, data-processing, multicloud, big-data-analytics, serverless
The agent that grows with you
GitHub repository with 181,357 stars and 31,120 forks.
Trending score: 5.95; stars gained: +1,867; forks gained: +361.
Language: Python
Topics: ai, ai-agent, ai-agents, anthropic, chatgpt, claude
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 60-95% fewer tokens, same answers. Library, proxy, MCP server.
GitHub repository with 12,942 stars and 833 forks.
Trending score: 5.69; stars gained: +2,829; forks gained: +175.
Language: Python
Topics: agent, ai, anthropic, claude-code, compression, context-engineering
Academic Research Skills for Claude Code: research → write → review → revise → finalize
GitHub repository with 27,327 stars and 2,249 forks.
Trending score: 5.52; stars gained: +1,079; forks gained: +89.
Language: Python
Topics: academic-pipeline, academic-writing, ai-research, claude, claude-code, literature-review
GitHub repository with 29,986 stars and 4,219 forks.
Trending score: 4.88; stars gained: +688; forks gained: +114.
Language: Python
Turn any technical book PDF into a Claude Code skill — ready to study, reference, and use while you work.
GitHub repository with 4,221 stars and 528 forks.
Trending score: 4.88; stars gained: +476; forks gained: +68.
Language: Python
An opinionated list of Python frameworks, libraries, tools, and resources
GitHub repository with 301,341 stars and 28,044 forks.
Trending score: 4.60; stars gained: +518; forks gained: +24.
Language: Python
Topics: awesome, python, collections, python-frameworks, python-libraries, python-tools
Incremental engine for long horizon agents 🌟 Star if you like it!
GitHub repository with 10,180 stars and 799 forks.
Trending score: 2.57; stars gained: +20; forks gained: +3.
Language: Rust
Topics: ai, change-data-capture, data-indexing, etl, indexing, python
High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE. Includes VS Code extension, TypeScript/Python SDKs, and Docker deployment.
GitHub repository with 3,742 stars and 1,222 forks.
Trending score: 2.16; stars gained: +7; forks gained: +8.
Language: C++
Topics: ai, cpp, data-pipeline, data-processing, machine-learning, mcp
Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
GitHub repository with 6,487 stars and 373 forks.
Trending score: 1.18; stars gained: +16; forks gained: -1.
Language: Python
Topics: data, data-analysis, data-pipeline, data-processing, data-science, data-visualization
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
GitHub repository with 63,100 stars and 1,679 forks.
Trending score: 0.46; stars gained: -30; forks gained: +4.
Language: Python
Topics: batch-processing, data-analytics, data-pipelines, data-processing, dataflow, etl
Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.
GitHub repository with 1,208 stars and 60 forks.
Trending score: 0.30; stars gained: +1; forks gained: +0.
Language: Python
Topics: artificial-intelligence, aws, data-science, gcp, machine-learning, python
A light-weight, flexible, and expressive statistical data testing library
GitHub repository with 4,361 stars and 401 forks.
Trending score: 0.28; stars gained: +0; forks gained: +0.
Language: Python
Topics: pandas, validation, schema, dataframes, testing, pandas-validation