FP8 support?

[wuchaooooo]

Hi! Is adding FP8 transformer engine (H100) speedup to inference planned?
If not, could you please give me an outline of what needs to be done in order for me to work on that?

Thank you!

[zRzRzRzRzRzRzR]

Not yet, because the machine is currently tight, and I cannot test it on the H100 with FP8 inference capabilities. I don’t have much advice on related content yet, because I haven’t conducted an in-depth test in this scenario.