Use the setup instructions to install the dependencies required to run the notebooks.
This directory has Jupyter notebooks that demonstrate transfer learning with and without Intel® Transfer Learning Tool. All of the notebooks use models from public model repositories and leverage optimized libraries Intel-optimized TensorFlow and Intel Extension for PyTorch.
| Notebook | Domain: Use Case | Framework | Description | Devices |
|---|---|---|---|---|
| BERT Text Classification with TensorFlow using the Intel® Transfer Learning Tool | NLP: Text Classification | TensorFlow and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to fine tune a BERT model from Hugging Face using text classification datasets. | cpu |
| BERT Text Classification with PyTorch using the Intel® Transfer Learning Tool | NLP: Text Classification | PyTorch and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to fine tune a BERT model from Hugging Face using text classification datasets. | cpu |
| Image Classification with TensorFlow using Intel® Transfer Learning Tool | CV: Image Classification | TensorFlow and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do transfer learning for image classification using a TensorFlow model. | cpu |
| Image Classification with TensorFlow using Graph Optimization and Intel® Transfer Learning Tool | CV: Image Classification | TensorFlow and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do transfer learning with graph optimization that increases throughput for image classification using a TensorFlow model. | cpu |
| Image Classification with PyTorch using Intel® Transfer Learning Tool | CV: Image Classification | PyTorch and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do transfer learning for image classification using a PyTorch model. | cpu, Intel® Gaudi® |
| Image Anomaly Detection with PyTorch using Intel® Transfer Learning Tool | CV: Image Anomaly Detection | PyTorch and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do feature extraction and pca analysis using a single function for image anomaly detection using a Torchvision model. | cpu, Intel Gaudi |
| Text Generation with PyTorch using Intel® Transfer Learning Tool | NLP: Text Generation | PyTorch and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do instruction fine-tuning for text generation with a large language model (LLM) from Hugging Face. | cpu |
| Notebook | Domain: Use Case | Framework | Description |
|---|---|---|---|
| BERT SQuAD fine tuning with TF Hub | NLP: Question Answering | TensorFlow | Demonstrates BERT fine tuning using scripts from the TensorFlow Model Garden and the SQuAD dataset. The notebook allows for selecting a BERT large or BERT base model from TF Hub. The fine tuned model is evaluated and exported as a saved model. |
| BERT Text Classification with TF Hub | NLP: Text Classification | TensorFlow | Demonstrates BERT binary text classification fine tuning using the IMDb movie review dataset and multiclass text classification fine tuning using the AG News datasets from TensorFlow Datasets or a custom dataset (for binary classification). The notebook allows for selecting a BERT encoder (BERT large, BERT base, or small BERT) to use along with a preprocessor from TF Hub. The fine tuned model is evaluated and exported as a saved model. |
| Text Classifier fine tuning with PyTorch & Hugging Face | NLP: Text Classification | PyTorch | Demonstrates fine tuning Hugging Face models to do sentiment analysis using the IMDb movie review dataset from Hugging Face Datasets or a custom dataset with Intel® Extension for PyTorch* |
| Image Classification with TF Hub | CV: Image Classification | TensorFlow | Demonstrates transfer learning with multiple TF Hub image classifiers, TF datasets, and custom image datasets |
| Image Classification with PyTorch & Torchvision | CV: Image Classification | PyTorch | Demonstrates transfer learning with multiple Torchvision image classification models, Torchvision datasets, and custom datasets |
| Image Classification with Hugging Face | CV: Image Classification | PyTorch, TensorFlow | Demonstrates transfer learning with Hugging Face models for image classification using either TensorFlow or PyTorch as a backend. |
| LLM instruction tuning with PyTorch & Hugging Face | NLP: Text Generation | PyTorch | Demonstrates instruction tuning Hugging Face Causal Language Models to do text generation using Hugging Face Datasets or a custom dataset |
| Notebook | Domain: Use Case | Framework | Description |
|---|---|---|---|
| Document-Level Sentiment Analysis (SST2) using PyTorch and the Intel® Transfer Learning Tool API | NLP: Text Classification | PyTorch and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do transfer learning for text classification using a PyTorch model from Hugging Face for a document-level sentiment analysis workflow. |
| Medical Imaging Classification (Colorectal histology) using TensorFlow and the Intel® Transfer Learning Tool API | CV: Image Classification | TensorFlow and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do transfer learning for image classification using a TensorFlow model for a medical imaging classification application. |
| Remote Sensing Image Scene Classification (Resisc) using TensorFlow and the Intel® Transfer Learning Tool API | CV: Image Classification | TensorFlow and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do transfer learning for image classification using a TensorFlow model for a remote sensing image scene classification application. |
| Multimodal Cancer Detection using TensorFlow, PyTorch, and the Intel® Transfer Learning Tool API | CV: Image Classification NLP: Text Classification |
TensorFlow, PyTorch, and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do transfer learning for a late fusion multimodal ensemble application using both NLP and computer vision models from PyTorch and Tensorflow, respectively. |
| Anomaly Detection with PyTorch using Intel® Transfer Learning Tool | CV: Image Anomaly Detection | PyTorch and the Intel Transfer Learning Tool API | Demonstrates how to use the Intel Transfer Learning Tool API to do feature extraction and pca analysis using dedicated function calls for image anomaly detection using a Torchvision model. |
| Notebook | Domain: Use Case | Framework | Description |
|---|---|---|---|
| Performance Comparison: Image Classification Transfer Learning with TensorFlow and the Intel Transfer Learning Tool | CV: Image Classification | TensorFlow and the Intel Transfer Learning Tool API | Compares training and evaluation metrics and performance for image classification transfer learning using TensorFlow libraries and the Intel Transfer Learning Tool. |
| Performance Comparison: Text Classification Transfer Learning with Hugging Face and the Intel Transfer Learning Tool | NLP: Text Classification | Hugging Face, PyTorch, and the Intel Transfer Learning Tool API | Compares training and evaluation metrics for text classification transfer learning using the Hugging Face Trainer and the Intel Transfer Learning Tool. |
All notebooks provide standard evaluation metrics such as accuracy and loss for validation/test/train sets. While important, it's essential to acknowledge that these metrics may not explicitly capture biases. Users should be cautious and consider potential biases by analyzing disparities in the data and model prediction. Techniques such as confusion matrices, PR curves, ROC curves, local attribution-based and gradCAM explanations, can all be good indicators for bias. Clear documentation of model behavior and performance is also crucial for iterative bias mitigation. Intel® Explainable AI Tools provides components that demonstrate the aformentioned techniques with Explainer, a simple API providing post-hoc model distillation and visualization methods, as well as The Model Card Generator which provides an interactive HTML report that containing these workflows and demonstrations of model behavior.