Notebook

Pandas (without Spark integration)

pandas-json-sample

pySpark

Check Installation

pyspark-intro: basic pySpark operations
pyspark-check-install: check pySpark installation, this notebook must run without errors.
pyspark-apache-arrow: Apache Arrow to integrate Pandas/NumPy data to pySpark.
pyspark-postgres: Reading/Writing data from Postgres

Basic commands

pyspark-dataframe-overview: Spark Dataframe operations

Machine Learning

Titanic Exercise | Solution: Kaggle Competition solved using pySpark
pyspark-nlp: Multi-Class Text Classification Using PySpark, MLlib & Doc2Vec