pyspark repositories
Trending repositories tagged pyspark, ranked by trending score. Updated daily.
-
1
sparklabx/sparklabx
Self-hosted multi-user Apache Spark notebooks — PySpark + Scala, per-user kernel pods, MinIO IAM storage, OAuth. One Helm install. Apache-2.0.
GitHub · TypeScript · 32 stars · 9 forks · +4 stars in 24 hours · Trending score 1.06
-
2
MrPowers/chispa
PySpark test helper methods with beautiful error messages
GitHub · Python · 778 stars · 86 forks · +2 stars in 24 hours · Trending score 1.04
-
3
ibis-project/ibis
the portable Python dataframe library
GitHub · Python · 6,705 stars · 790 forks · +1 stars in 24 hours · Trending score 0.96
-
4
narwhals-dev/narwhals
Lightweight and extensible compatibility layer between dataframe libraries!
GitHub · Python · 1,751 stars · 276 forks · 0 stars in 24 hours · Trending score 0.75
-
5
lakehq/sail
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
GitHub · Rust · 3,504 stars · 243 forks · 0 stars in 24 hours · Trending score 0.69
-
6
sparkdq-community/sparkdq
A lightweight, declarative PySpark framework for data quality validation — check columns, rows, and entire datasets directly in your Spark pipelines
GitHub · Python · 77 stars · 10 forks · +1 stars in 24 hours · Trending score 0.61
-
7
ssanjaychandra123/data-engineering-patterns
600+ free patterns and concepts for data engineers on Azure Databricks, Microsoft Fabric, and PySpark. 12 books covering the full stack. Free forever.
GitHub · 93 stars · 39 forks · 0 stars in 24 hours · Trending score 0.58
-
8
DataFabric-Academy/ms-fabric-data-science
Hands-on Microsoft Fabric Data Science labs (FreshMart churn) with Thai study notes, PySpark/MLflow, and DP-604-aligned learning path.
GitHub · Jupyter Notebook · 99 stars · 3 forks · 0 stars in 24 hours · Trending score 0.14
-
9
opentargets/gentropy
Open Targets python framework for post-GWAS analysis
GitHub · Python · 62 stars · 17 forks · 0 stars in 24 hours · Trending score 0.10
-
10
andfanilo/pyspark-tutorial
Jupyter notebooks for pyspark tutorials given at University
GitHub · Jupyter Notebook · 111 stars · 74 forks · 0 stars in 24 hours · Trending score 0.08
-
11
khuyentran1401/codecut-blog
45+ production-ready tutorials on data science, MLOps, and AI tools. All code is executable and adaptable for real projects.
GitHub · Jupyter Notebook · 42 stars · 12 forks · 0 stars in 24 hours · Trending score 0.05
-
12
Matt17BR/openwrangler
Open Wrangler is an open-source visual dataframe editor for VS Code and editors based on it. Explore files or data loaded in memory, make changes through a GUI, and generate Python or R code.
GitHub · TypeScript · 7 stars · 1 forks · 0 stars in 24 hours · Trending score 0.05
-
13
animetanver-commits/fabric-unified-data-blueprint
Microsoft Fabric Unified Data Foundation with Databricks & Purview 2026
GitHub · HTML · 114 stars · 0 forks · 0 stars in 24 hours · Trending score 0.04
-
14
apache/incubator-graphar
An open source, standard data file format for graph data storage and retrieval.
GitHub · Java · 384 stars · 97 forks · 0 stars in 24 hours · Trending score 0.04
-
15
mahmoudparsian/big-data-modeling
Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University
GitHub · HTML · 167 stars · 144 forks · 0 stars in 24 hours · Trending score 0.04
-
16
datasifaw/databricks-projet-bancaire-lakehouse
Projet Data Engineering bancaire de bout en bout avec Databricks : ingestion des données clients, comptes et transactions depuis SQL Server, traitement en architecture Bronze-Silver-Gold, transformations PySpark, stockage Delta Lake, contrôles de qualité, orchestration via Workflows et création d’indicateurs financiers pour un dashboard BI fiable
GitHub · TSQL · 35 stars · 0 forks · 0 stars in 24 hours · Trending score 0.03
-
17
zaratsian/Spark
Apache Spark (Scala, PySpark, SparkR) Code, Tricks, and References
GitHub · Jupyter Notebook · 69 stars · 37 forks · 0 stars in 24 hours · Trending score 0.03
-
18
maurinruneacco/luau-diagrams-and-defects
Production Data Engineering Cloud Architecture Workflow Automation Portfolio 2026
GitHub · HTML · 69 stars · 0 forks · 0 stars in 24 hours · Trending score 0.02
-
19
pavanbadempet/AI-Healthcare-System
AI Healthcare System & Clinical Data Platform. Combines a Databricks Medallion Lakehouse (OHDSI OMOP CDM v5.4 & FHIR R4) with TabICLv2 tabular foundation models, calibrated tree ensembles (CatBoost, XGBoost), a 10-year coupled ODE digital twin simulator, LangGraph multi-agent RAG, and an offline HIPAA-compliant clinical decision support portal.
GitHub · Python · 66 stars · 16 forks · 0 stars in 24 hours · Trending score 0.01
-
20
mlverse/pysparklyr
Extension to {sparklyr} that allows you to interact with Spark & Databricks Connect
GitHub · R · 19 stars · 10 forks · 0 stars in 24 hours · Trending score 0.00
-
21
mitchelllisle/sparkdantic
✨ A Pydantic to PySpark schema library
GitHub · Python · 134 stars · 22 forks · 0 stars in 24 hours · Trending score 0.00
-
22
JohnSnowLabs/spark-nlp
State of the Art Natural Language Processing
GitHub · Scala · 4,160 stars · 743 forks · 0 stars in 24 hours · Trending score 0.00
-
23
shauryashaurya/The-Silmaril
Practice #ontology engineering with Python (and other languages).
GitHub · Jupyter Notebook · 38 stars · 9 forks · 0 stars in 24 hours · Trending score 0.00
-
24
zorost/AI-Engineering-Lab
A free, self-paced 24-week AI engineering course: Python, machine learning, LLMs, RAG, fine-tuning, agents and MCP, Azure and Vertex and Bedrock, and Databricks. 43 runnable notebooks, one continuous case study. MIT licensed, no signup. By Zorost Intelligence AI Lab.
GitHub · Jupyter Notebook · 316 stars · 256 forks · 0 stars in 24 hours · Trending score 0.00