Model Zoo for Intel® Architecture

This repository contains links to pre-trained models, sample scripts, best practices, and step-by-step tutorials for many popular open-source machine learning models optimized by Intel to run on Intel® Xeon® Scalable processors and Intel® Data Center GPUs.

Model packages and containers for running the Model Zoo's workloads can be found at the Intel® Developer Catalog.

Purpose of the Model Zoo

• Demonstrate the AI workloads and deep learning models Intel has optimized and validated to run on Intel hardware
• Show how to efficiently execute, train, and deploy Intel-optimized models
• Make it easy to get started running Intel-optimized models on Intel hardware in the cloud or on bare metal

DISCLAIMER: These scripts are not intended for benchmarking Intel platforms. For any performance and/or benchmarking information on specific Intel platforms, visit https://www.intel.ai/blog.

Use cases

The model documentation in the tables below have information on the prerequisites to run each model. The model scripts run on Linux. Certain models are also able to run using bare metal on Windows. For more information and a list of models that are supported on Windows, see the documentation here.

For best performance on Intel® Data Center GPU Flex Series, please check the dGPU Int8 instructions for some image recognition and object detection models. It runs inference using Intel(R) Extension for PyTorch or Intel(R) Extension for TensorFlow.

Instructions available to run on Sapphire Rapids.

Image Recognition

Model	Framework	Mode	Model Documentation	Benchmark/Test Dataset
DenseNet169	TensorFlow	Inference	FP32	ImageNet 2012
Inception V3	TensorFlow	Inference	Int8 FP32	ImageNet 2012
Inception V4	TensorFlow	Inference	Int8 FP32	ImageNet 2012
MobileNet V1*	TensorFlow	Inference	Int8 FP32 BFloat16	ImageNet 2012
MobileNet V1* Sapphire Rapids	TensorFlow	Inference	Int8 FP32 BFloat16 BFloat32	ImageNet 2012
ResNet 101	TensorFlow	Inference	Int8 FP32	ImageNet 2012
ResNet 50	TensorFlow	Inference	Int8 FP32	ImageNet 2012
ResNet 50v1.5	TensorFlow	Inference	Int8 FP32 BFloat16 dGPU Int8	ImageNet 2012
ResNet 50v1.5 Sapphire Rapids	TensorFlow	Inference	Int8 FP32 BFloat16 BFloat32	ImageNet 2012
ResNet 50v1.5	TensorFlow	Training	FP32 BFloat16	ImageNet 2012
ResNet 50v1.5 Sapphire Rapids	TensorFlow	Training	FP32 BFloat16 BFloat32	ImageNet 2012
Inception V3	TensorFlow Serving	Inference	FP32	Synthetic Data
ResNet 50v1.5	TensorFlow Serving	Inference	FP32	Synthetic Data
GoogLeNet	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
Inception v3	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
MNASNet 0.5	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
MNASNet 1.0	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
ResNet 50	PyTorch	Inference	FP32 BFloat16 BFloat32	ImageNet 2012
ResNet 50	PyTorch	Training	FP32 BFloat16 BFloat32	ImageNet 2012
ResNet 101	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
ResNet 152	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
ResNext 32x4d	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
ResNext 32x16d	PyTorch	Inference	FP32 BFloat16 BFloat32	ImageNet 2012
VGG-11	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
VGG-11 with batch normalization	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
Wide ResNet-50-2	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
Wide ResNet-101-2	PyTorch	Inference	FP32 BFloat16	ImageNet 2012
ResNet 50 v1.5	PyTorch	Inference	dGPU Int8	ImageNet 2012

Image Segmentation

Model	Framework	Mode	Model Documentation	Benchmark/Test Dataset
3D U-Net	TensorFlow	Inference	FP32	BRATS 2018
3D U-Net MLPerf*	TensorFlow	Inference	FP32 BFloat16 Int8	BRATS 2019
3D U-Net MLPerf* Sapphire Rapids	Tensorflow	Inference	FP32 BFloat16 Int8 BFloat32	BRATS 2019
MaskRCNN	TensorFlow	Inference	FP32	MS COCO 2014
UNet	TensorFlow	Inference	FP32

Language Modeling

Model	Framework	Mode	Model Documentation	Benchmark/Test Dataset
BERT	TensorFlow	Inference	FP32 BFloat16	SQuAD
BERT	TensorFlow	Training	FP32 BFloat16	SQuAD and MRPC
BERT Sapphire Rapids	Tensorflow	Inference	FP32 BFloat16 Int8 BFloat32	SQuAD
BERT Sapphire Rapids	Tensorflow	Training	FP32 BFloat16 BFloat32	SQuAD
DistilBERT base	Tensorflow	Inference	FP32 BFloat16 INT8	SST-2
BERT base	PyTorch	Inference	FP32 BFloat16	BERT Base SQuAD1.1
BERT large	PyTorch	Inference	FP32 Int8 BFloat16 BFloat32	BERT Large SQuAD1.1
BERT large	PyTorch	Training	FP32 BFloat16 BFloat32	preprocessed text dataset
DistilBERT base	PyTorch	Inference	FP32 Int8 BFloat16 BFloat32	DistilBERT Base SQuAD1.1
RNN-T	PyTorch	Inference	FP32 BFloat16 BFloat32	RNN-T dataset
RNN-T	PyTorch	Training	FP32 BFloat16 BFloat32	RNN-T dataset
RoBERTa base	PyTorch	Inference	FP32 BFloat16	RoBERTa Base SQuAD 2.0
T5	PyTorch	Inference	FP32 Int8

Language Translation

Model	Framework	Mode	Model Documentation	Benchmark/Test Dataset
BERT	TensorFlow	Inference	FP32	MRPC
GNMT*	TensorFlow	Inference	FP32	MLPerf GNMT model benchmarking dataset
Transformer_LT_mlperf*	TensorFlow	Inference	FP32 BFloat16 Int8	WMT English-German data
Transformer_LT_mlperf* Sapphire Rapids	Tensorflow	Inference	FP32 BFloat16 Int8 BFloat32	WMT English-German dataset
Transformer_LT_mlperf*	TensorFlow	Training	FP32 BFloat16	WMT English-German dataset
Transformer_LT_mlperf* Sapphire Rapids	Tensorflow	Training	FP32 BFloat16 BFloat32	WMT English-German dataset
Transformer_LT_Official	TensorFlow	Inference	FP32	WMT English-German dataset
Transformer_LT_Official	TensorFlow Serving	Inference	FP32

Object Detection

Model	Framework	Mode	Model Documentation	Benchmark/Test Dataset
Faster R-CNN	TensorFlow	Inference	Int8 FP32	COCO 2017 validation dataset
R-FCN	TensorFlow	Inference	Int8 FP32	COCO 2017 validation dataset
SSD-MobileNet*	TensorFlow	Inference	Int8 FP32 BFloat16 dGPU Int8	COCO 2017 validation dataset
SSD-MobileNet* Sapphire Rapids	TensorFlow	Inference	Int8 FP32 BFloat16 BFloat32	COCO 2017 validation dataset
SSD-ResNet34*	TensorFlow	Inference	Int8 FP32 BFloat16	COCO 2017 validation dataset
SSD-ResNet34* Sapphire Rapids	TensorFlow	Inference	Int8 FP32 BFloat16 BFloat32	COCO 2017 validation dataset
SSD-ResNet34	TensorFlow	Training	FP32 BFloat16	COCO 2017 training dataset
SSD-ResNet34 Sapphire Rapids	TensorFlow	Training	FP32 BFloat16 BFloat32	COCO 2017 training dataset
SSD-MobileNet	TensorFlow Serving	Inference	FP32
Faster R-CNN ResNet50 FPN	PyTorch	Inference	FP32 BFloat16	COCO 2017
Mask R-CNN	PyTorch	Inference	FP32 BFloat16 BFloat32	COCO 2017
Mask R-CNN	PyTorch	Training	FP32 BFloat16 BFloat32	COCO 2017
Mask R-CNN ResNet50 FPN	PyTorch	Inference	FP32 BFloat16	COCO 2017
RetinaNet ResNet-50 FPN	PyTorch	Inference	FP32 BFloat16	COCO 2017
SSD-ResNet34	PyTorch	Inference	FP32 Int8 BFloat16 BFloat32	COCO 2017
SSD-ResNet34	PyTorch	Training	FP32 BFloat16 BFloat32	COCO 2017
SSD-MobileNet	Pytorch	Inference	dGPU Int8	COCO 2017
Yolo V4	Pytorch	Inference	dGPU Int8	COCO 2017

Recommendation

Model	Framework	Mode	Model Documentation	Benchmark/Test Dataset
DIEN	TensorFlow	Inference	FP32 BFloat16	DIEN dataset
DIEN Sapphire Rapids	TensorFlow	Inference	FP32 BFloat16 BFloat32	DIEN dataset
DIEN	TensorFlow	Training	FP32	DIEN dataset
DIEN Sapphire Rapids	TensorFlow	Training	FP32 BFloat16 BFloat32	DIEN dataset
NCF	TensorFlow	Inference	FP32	MovieLens 1M
Wide & Deep	TensorFlow	Inference	FP32	Census Income dataset
Wide & Deep Large Dataset	TensorFlow	Inference	Int8 FP32	Large Kaggle Display Advertising Challenge dataset
Wide & Deep Large Dataset	TensorFlow	Training	FP32	Large Kaggle Display Advertising Challenge dataset
DLRM	PyTorch	Inference	FP32 Int8 BFloat16 BFloat32	Criteo Terabyte
DLRM	PyTorch	Training	FP32 BFloat16 BFloat32	Criteo Terabyte

Text-to-Speech

Model	Framework	Mode	Model Documentation	Benchmark/Test Dataset
WaveNet	TensorFlow	Inference	FP32

Shot Boundary Detection

Model	Framework	Mode	Model Documentation	Benchmark/Test Dataset
TransNetV2	PyTorch	Inference	FP32 BFloat16	Synthetic Data

AI Drug Design (AIDD)

Model	Framework	Mode	Model Documentation	Benchmark/Test Dataset
AlphaFold2	PyTorch	Inference	FP32	AF2Dataset

*Means the model belongs to MLPerf models and will be supported long-term.

How to Contribute

If you would like to add a new benchmarking script, please use this guide.