convert float32 model to float16, but memory usage has not decreased

[kukugpt]

Tasks

Preview Give feedback

No tasks being tracked yet.

[titaiwangms]

Hi @kukugpt Could you share a minimum repro, or your original code?

[kukugpt]

of course, my task listed in below:

import numpy as np
from onnx import load_model, save_model
from onnxmltools.utils import float16_converter
from utils import check_model_consistency

if name == "main":
onnx_model_path = "float32_optim.onnx"
save_model_path = "_ptim_float16.onnx"
onnx_model = load_model(onnx_model_path)
trans_model = float16_converter.convert_float_to_float16(onnx_model)
save_model(trans_model, save_model_path)
check_model_consistency(onnx_model_path, save_model_path, input_type1=np.float32, input_type2=np.float16, diff=1/255.)
my model like a framework of Unet.

[kukugpt]

@titaiwangms

[titaiwangms]

For tool float16_converter
cc @xiaowuhu @xadupre

[xadupre]

Can you be more specific about the memory usage? Is it when you run it? What is the model size before and after...