diff --git a/docs/QATv2.md b/docs/QATv2.md index 2bf5b243d..1360cf397 100644 --- a/docs/QATv2.md +++ b/docs/QATv2.md @@ -1,7 +1,7 @@ # Quantization Aware Training (QAT) This document aims to explain Quantization Aware Training framework for MAX7800x series microcontrollers. QAT for MAX7800x consists of four main stages: -Activation Statistics Collection, Activation Threshold Determination, Scale Ajustments, and Weights Quantization. +Activation Statistics Collection, Activation Threshold Determination, Scale Adjustments, and Weights Quantization. ## Activation Statistics Collection @@ -12,7 +12,7 @@ To train a quantization-aware model, the first step is to collect activation sta After collecting the activation statistics, the next step is to determine the activation thresholds. To do this, first, an outlier removal step based on z-score is applied to the activation statistics. The default z-score is 8.0, and it can be changed by defining a z-score on the qat policy file. A threshold is then selected to encompass the full activation range, followed by an iterative algorithm [1] that minimizes quantization error by adjusting the threshold to balance range and resolution. Scales are calculated as powers of two, making the scaling-down operation more computationally efficient by defining them as bit shift operations at the edge hardware. -## Scale Ajustments +## Scale Adjustments To implement the threshold-based quantization, the scales of the layers are adjusted. The scales are adjusted based on the type of operation that is performed on the layers. The scale adjustments are made for residual additions, concatenations, and layer sharing. In the f