Skip to content

Latest commit

 

History

History
143 lines (123 loc) · 6.52 KB

README.md

File metadata and controls

143 lines (123 loc) · 6.52 KB
Raw

ResNet50 v1.5 Int8 inference

Description

This document has instructions for running ResNet50 v1.5 Int8 inference using Intel-optimized TensorFlow.

Datasets

Download and preprocess the ImageNet dataset using the instructions here. After running the conversion script you should have a directory with the ImageNet dataset in the TF records format.

Set the DATASET_DIR to point to the TF records directory when running ResNet50 v1.5.

Quick Start Scripts

Script name Description
int8_online_inference.sh Runs online inference (batch_size=1).
int8_batch_inference.sh Runs batch inference (batch_size=128).
int8_accuracy.sh Measures the model accuracy (batch_size=100).
multi_instance_batch_inference.sh Uses numactl to run batch inference (batch_size=128) with one instance per socket for 1500 steps and 50 warmup steps. If no DATASET_DIR is set, synthetic data is used. The script waits for all instances to complete, then prints a summarized throughput value.
multi_instance_online_inference.sh Uses numactl to run online inference (batch_size=1) using four cores per instance for 1500 steps and 50 warmup steps. If no DATASET_DIR is set, synthetic data is used. The script waits for all instances to complete, then prints a summarized throughput value.

Run the model

Setup your environment using the instructions below, depending on if you are using AI Kit:

Setup using AI Kit on Linux Setup without AI Kit on Linux Setup without AI Kit on Windows

To run using AI Kit on Linux you will need:

  • numactl
  • wget
  • openmpi-bin (only required for multi-instance)
  • openmpi-common (only required for multi-instance)
  • openssh-client (only required for multi-instance)
  • openssh-server (only required for multi-instance)
  • libopenmpi-dev (only required for multi-instance)
  • horovod==0.21.0 (only required for multi-instance)
  • Activate the tensorflow conda environment 
    conda activate tensorflow

To run without AI Kit on Linux you will need:

  • Python 3
  • intel-tensorflow>=2.5.0
  • git
  • numactl
  • wget
  • openmpi-bin (only required for multi-instance)
  • openmpi-common (only required for multi-instance)
  • openssh-client (only required for multi-instance)
  • openssh-server (only required for multi-instance)
  • libopenmpi-dev (only required for multi-instance)
  • horovod==0.21.0 (only required for multi-instance)
  • A clone of the Model Zoo repo
    git clone https://github.com/IntelAI/models.git

To run without AI Kit on Windows you will need:

After finishing the setup above, download the pretrained model and set the PRETRAINED_MODEL environment var to the path to the frozen graph. If you run on Windows, please use a browser to download the pretrained model using the link below. For Linux, run:

wget https://storage.googleapis.com/intel-optimized-tensorflow/models/v1_8/resnet50v1_5_int8_pretrained_model.pb
export PRETRAINED_MODEL=$(pwd)/resnet50v1_5_int8_pretrained_model.pb

Set environment variables for the path to your DATASET_DIR for ImageNet and an OUTPUT_DIR where log files will be written. Navigate to your model zoo directory and then run a quickstart script on either Linux or Windows.

Run on Linux

# cd to your model zoo directory
cd models

export PRETRAINED_MODEL=<path to the frozen graph downloaded above>
export DATASET_DIR=<path to the ImageNet TF records>
export OUTPUT_DIR=<path to the directory where log files and checkpoints will be written>
# For a custom batch size, set env var `BATCH_SIZE` or it will run with a default value.
export BATCH_SIZE=<customized batch size value>

./quickstart/image_recognition/tensorflow/resnet50v1_5/inference/cpu/int8/<script name>.sh

Run on Windows

Using cmd.exe, run:

# cd to your model zoo directory
cd models

set PRETRAINED_MODEL=<path to the frozen graph downloaded above>
set DATASET_DIR=<path to the ImageNet TF records>
set OUTPUT_DIR=<directory where log files will be written>
# For a custom batch size, set env var `BATCH_SIZE` or it will run with a default value.
set BATCH_SIZE=<customized batch size value>

bash quickstart\image_recognition\tensorflow\resnet50v1_5\inference\cpu\int8\<script name>.sh

Note: You may use cygpath to convert the Windows paths to Unix paths before setting the environment variables. As an example, if the dataset location on Windows is D:\user\ImageNet, convert the Windows path to Unix as shown:

cygpath D:\user\ImageNet
/d/user/ImageNet

Then, set the DATASET_DIR environment variable set DATASET_DIR=/d/user/ImageNet.

Additional Resources