CUDA OOM Error while importing

[DingYX0731]

It is so weird that CUDA OOM error happens when importing package!!! My GPU is L20 48GB, which should satisfy the requirement of the experiment. Please help...
When importing from ChatDrug.task_and_evaluation.Conversational_LLMs_utils import complete and from utils import ( construct_PDDS_prompt, load_retrieval_DB, retrieve_and_feedback, load_thredhold ):

[DingYX0731]

Note that I have change the directory of pytorch_model_ss3.bin because the path is not correct after download checkpoint according to README. However, it may not be the cause of oom error. I have observed that before loading model checkpoint, the usage of GPU suddenly comes to more than 95% and then cuda oom when the loaded model is put to the device.

Will the problem related to the tensorflow related warning? I am not familar with tensorflow so I hope someone could give me some hint/instruction. Thanks!

[chao1224]

Hi @DingYX0731,

Apologize for the delay. I just tried the codes, and they work well.
May I ask if you can add the CUDA_LAUNCH_BLOCKING=1 and print the exception messages?