Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

模型编码器tensorrt测试是用的fp16模式吗 #142

Open
13213085 opened this issue Oct 21, 2024 · 1 comment
Open

模型编码器tensorrt测试是用的fp16模式吗 #142

13213085 opened this issue Oct 21, 2024 · 1 comment
Assignees

Comments

@13213085
Copy link

mkdir -p assets/export_models/efficientvit_sam/tensorrt/

Export Encoder

trtexec --onnx=assets/export_models/efficientvit_sam/onnx/efficientvit_sam_xl1_encoder.onnx --minShapes=input_image:1x3x1024x1024 --optShapes=input_image:4x3x1024x1024 --maxShapes=input_image:4x3x1024x1024 --saveEngine=assets/export_models/efficientvit_sam/tensorrt/efficientvit_sam_xl1_encoder.engine
部署tensorrt模型的时候导入编码器的时候没有指定fp16模式,是否用fp16模式测过,结果是否正确,论文里面测试的fp16结果编码器用的是fp16模型吗,还是只有解码器用fp16模型

@zhuoyang20
Copy link
Collaborator

Hi,

Setting all parameters to FP16 in the encoder leads to overflow in the LayerNorm layers. We suggest you to enable FP16 mode while forcing the LayerNorm layers to FP32 precision. This approach has been verified by us. I hope this addresses your question. Please feel free to reach out if you have additional concerns.

Best,
Zhuoyang

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants