ReadMe update and QATv2 Refactoring

[oguzhanbsolak]

resolves #340

1- quantile function replaced with torch.quantile()
2- Improved Distributed mode compatibility
3- Fixed the issue caused when loading best_ckpt with compile mode

[ermanok]

Looks good,

[yousefbilal]

using incoming qatv2 branch,the KeyError is fixed, but when I run evaluate_cifar100_mobilenetv2 I am faced with the error:
Log file for this run: /mnt/d/ai8x/ai8x-training-qatv2/logs/2024.11.27-225820/2024.11.27-225820.log
Traceback (most recent call last):
File "/mnt/d/ai8x/ai8x-training-qatv2/train.py", line 1577, in
main()
File "/mnt/d/ai8x/ai8x-training-qatv2/train.py", line 409, in main
model = apputils.load_lean_checkpoint(model, args.load_model_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/d/ai8x/ai8x-training-qatv2/distiller/build/editable.distiller-0.4.0rc0-py3-none-any/distiller/apputils/checkpoint.py", line 92, in load_lean_checkpoint
return load_checkpoint(model, chkpt_file, model_device=model_device,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/d/ai8x/ai8x-training-qatv2/distiller/build/editable.distiller-0.4.0rc0-py3-none-any/distiller/apputils/checkpoint.py", line 249, in load_checkpoint
raise ValueError("The loaded checkpoint (%s) is missing %d state keys" %
ValueError: The loaded checkpoint (./logs/2024.11.27-223950/qat_best.pth.tar) is missing 136 state keys

[oguzhanbsolak]

using incoming qatv2 branch,the KeyError is fixed, but when I run evaluate_cifar100_mobilenetv2 I am faced with the error: Log file for this run: /mnt/d/ai8x/ai8x-training-qatv2/logs/2024.11.27-225820/2024.11.27-225820.log Traceback (most recent call last): File "/mnt/d/ai8x/ai8x-training-qatv2/train.py", line 1577, in main() File "/mnt/d/ai8x/ai8x-training-qatv2/train.py", line 409, in main model = apputils.load_lean_checkpoint(model, args.load_model_path, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/d/ai8x/ai8x-training-qatv2/distiller/build/editable.distiller-0.4.0rc0-py3-none-any/distiller/apputils/checkpoint.py", line 92, in load_lean_checkpoint return load_checkpoint(model, chkpt_file, model_device=model_device, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/d/ai8x/ai8x-training-qatv2/distiller/build/editable.distiller-0.4.0rc0-py3-none-any/distiller/apputils/checkpoint.py", line 249, in load_checkpoint raise ValueError("The loaded checkpoint (%s) is missing %d state keys" % ValueError: The loaded checkpoint (./logs/2024.11.27-223950/qat_best.pth.tar) is missing 136 state keys

Hi, thanks for getting in touch. Could you please share the ckpt file(2024.11.27-223950/qat_best.pth.tar) with us? Also, please indicate which script you are using: "evaluate_cifar100_mobilenet_v2_0.5.sh" or "evaluate_cifar100_mobilenet_v2_0.75.sh."

[yousefbilal]

using incoming qatv2 branch,the KeyError is fixed, but when I run evaluate_cifar100_mobilenetv2 I am faced with the error: Log file for this run: /mnt/d/ai8x/ai8x-training-qatv2/logs/2024.11.27-225820/2024.11.27-225820.log Traceback (most recent call last): File "/mnt/d/ai8x/ai8x-training-qatv2/train.py", line 1577, in main() File "/mnt/d/ai8x/ai8x-training-qatv2/train.py", line 409, in main model = apputils.load_lean_checkpoint(model, args.load_model_path, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/d/ai8x/ai8x-training-qatv2/distiller/build/editable.distiller-0.4.0rc0-py3-none-any/distiller/apputils/checkpoint.py", line 92, in load_lean_checkpoint return load_checkpoint(model, chkpt_file, model_device=model_device, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/d/ai8x/ai8x-training-qatv2/distiller/build/editable.distiller-0.4.0rc0-py3-none-any/distiller/apputils/checkpoint.py", line 249, in load_checkpoint raise ValueError("The loaded checkpoint (%s) is missing %d state keys" % ValueError: The loaded checkpoint (./logs/2024.11.27-223950/qat_best.pth.tar) is missing 136 state keys

Hi, thanks for getting in touch. Could you please share the ckpt file(2024.11.27-223950/qat_best.pth.tar) with us? Also, please indicate which script you are using: "evaluate_cifar100_mobilenet_v2_0.5.sh" or "evaluate_cifar100_mobilenet_v2_0.75.sh."

I ran evaluate_cifar100_mobilenet_v2_0.75.sh but I believe the same issue will still take place with evaluate_cifar100_mobilenet_v2_0.5.
qat_best.pth.tar.zip

[oguzhanbsolak]

using incoming qatv2 branch,the KeyError is fixed, but when I run evaluate_cifar100_mobilenetv2 I am faced with the error: Log file for this run: /mnt/d/ai8x/ai8x-training-qatv2/logs/2024.11.27-225820/2024.11.27-225820.log Traceback (most recent call last): File "/mnt/d/ai8x/ai8x-training-qatv2/train.py", line 1577, in main() File "/mnt/d/ai8x/ai8x-training-qatv2/train.py", line 409, in main model = apputils.load_lean_checkpoint(model, args.load_model_path, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/d/ai8x/ai8x-training-qatv2/distiller/build/editable.distiller-0.4.0rc0-py3-none-any/distiller/apputils/checkpoint.py", line 92, in load_lean_checkpoint return load_checkpoint(model, chkpt_file, model_device=model_device, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/d/ai8x/ai8x-training-qatv2/distiller/build/editable.distiller-0.4.0rc0-py3-none-any/distiller/apputils/checkpoint.py", line 249, in load_checkpoint raise ValueError("The loaded checkpoint (%s) is missing %d state keys" % ValueError: The loaded checkpoint (./logs/2024.11.27-223950/qat_best.pth.tar) is missing 136 state keys

Hi, thanks for getting in touch. Could you please share the ckpt file(2024.11.27-223950/qat_best.pth.tar) with us? Also, please indicate which script you are using: "evaluate_cifar100_mobilenet_v2_0.5.sh" or "evaluate_cifar100_mobilenet_v2_0.75.sh."

I ran evaluate_cifar100_mobilenet_v2_0.75.sh but I believe the same issue will still take place with evaluate_cifar100_mobilenet_v2_0.5. qat_best.pth.tar.zip

Please update your evaluation script to include the QAT policy used during training by specifying it as --qat-policy policy_used_in_training.yaml. Once QAT starts, batch normalization parameters are fused, and as a result, QAT checkpoints do not include batch normalization parameters.

If the QAT policy is not specified in the evaluation script, the default policy is applied, where QAT begins at epoch 30. Since your checkpoint is from epoch 19, failing to declare the correct policy will result in a mismatch: the model will still contain batch normalization layers, but the checkpoint will lack the corresponding parameters.

P.S.: Another thing that can be missed is the -8 parameter in the evaluation script. The -8 parameter is intended for evaluating a quantized checkpoint. Since your current checkpoint is from QAT, you must remove this parameter to properly evaluate the model. After completing the synthesizer steps, you can reintroduce the -8 parameter to evaluate the fully quantized checkpoint.