Skip to content

Commit

Permalink
Merge pull request #167 from GoogleCloudDataproc/bmiro-ai-ml-samples
Browse files Browse the repository at this point in the history
Adding Pytorch Image Classification sample
  • Loading branch information
bradmiro authored Dec 18, 2024
2 parents 22f74f3 + 4139a99 commit 35e5b2f
Show file tree
Hide file tree
Showing 4 changed files with 166 additions and 117 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ This repository contains code and documentation for use with
* `codelabs/opencv-haarcascade` provides the source code for the [OpenCV Dataproc Codelab](https://codelabs.developers.google.com/codelabs/cloud-dataproc-opencv/index.html), which demonstrates a Spark job that adds facial detection to a set of images.
* `codelabs/spark-bigquery` provides the source code for the [PySpark for Preprocessing BigQuery Data Codelab](https://codelabs.developers.google.com/codelabs/pyspark-bigquery/index.html), which demonstrates using PySpark on Cloud Dataproc to process data from BigQuery.
* `codelabs/spark-nlp` provides the source code for the [PySpark for Natural Language Processing Codelab](https://codelabs.developers.google.com/codelabs/spark-nlp/index.html), which demonstrates using [spark-nlp](https://github.com/JohnSnowLabs/spark-nlp) library for Natural Language Processing.
* `notebooks/ai-ml/` provides source code for Spark for AI/ML use cases, including a [PyTorch](https://pytorch.org/) sample for image classification.
* `notebooks/python` provides example Jupyter notebooks to demonstrate using PySpark with the [BigQuery Storage Connector](https://github.com/GoogleCloudPlatform/spark-bigquery-connector) and the [Spark GCS Connector](https://github.com/GoogleCloudPlatform/bigdata-interop/tree/master/gcs)
* `spark-tensorflow` provides an example of using Spark as a preprocessing toolchain for Tensorflow jobs. Optionally,
it demonstrates the [spark-tensorflow-connector](https://github.com/tensorflow/ecosystem/tree/master/spark/spark-tensorflow-connector) to convert CSV files to TFRecords.
Expand Down
3 changes: 1 addition & 2 deletions notebooks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,4 @@ Follow the links below for instructions on how to create a Dataproc Cluster with

* / neo4j / 1.1. Similarity Example, Graph Data Science Client - Python
* / neo4j / 1.2. Similarity Example, Python Client
* / neo4j / 1.3. Similarity Example, BigQuery Storage & Spark Connector & GDS Client - Python

* / neo4j / 1.3. Similarity Example, BigQuery Storage & Spark Connector & GDS Client - Python
Loading

0 comments on commit 35e5b2f

Please sign in to comment.