Skip to content

Commit

Permalink
Spelling fix
Browse files Browse the repository at this point in the history
  • Loading branch information
oguzhanbsolak committed Nov 11, 2024
1 parent bc3f53c commit 55d1df8
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/QATv2.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Quantization Aware Training (QAT)

This document aims to explain Quantization Aware Training framework for MAX7800x series microcontrollers. QAT for MAX7800x consists of four main stages:
Activation Statistics Collection, Activation Threshold Determination, Scale Ajustments, and Weights Quantization.
Activation Statistics Collection, Activation Threshold Determination, Scale Adjustments, and Weights Quantization.

## Activation Statistics Collection

Expand All @@ -12,7 +12,7 @@ To train a quantization-aware model, the first step is to collect activation sta
After collecting the activation statistics, the next step is to determine the activation thresholds. To do this, first, an outlier removal step based on z-score is applied to the activation statistics. The default z-score is 8.0, and it can be changed by defining a z-score on the qat policy file. A threshold is then selected to encompass the full activation range, followed by an iterative algorithm [1] that minimizes quantization error by adjusting the threshold to balance range and resolution. Scales are calculated as powers of two, making the scaling-down operation more computationally efficient by defining them as bit shift operations at the edge hardware.


## Scale Ajustments
## Scale Adjustments

To implement the threshold-based quantization, the scales of the layers are adjusted. The scales are adjusted based on the type of operation that is performed on the layers. The scale adjustments are made for residual additions, concatenations, and layer sharing. In the f

Expand Down

0 comments on commit 55d1df8

Please sign in to comment.