'triton.compiler' has no attribute 'OutOfResources'

[davidray222]

Thank you for this excellent work!!
may I ask some question?

I use llama-7b to quantize and use qalora.py,and occur
problem:
except triton.compiler.OutOfResources:
AttributeError: module 'triton.compiler' has no attribute 'OutOfResources'

my triton vision:
Version: 2.2.0
Summary: A language and compiler for custom Deep Learning operations
Home-page: https://github.com/openai/triton/
Author: Philippe Tillet
Author-email: [email protected]

[yuhuixu1993]

Hi, this problem seems to be caused by the inconsistent version of triton. May you try other versions? I found similar issues on GitHub triton-lang/triton#1735.

…

---- Replied Message ---- | From | ***@***.***> | | Date | 12/31/2024 16:58 | | To | yuhuixu1993/qa-lora ***@***.***> | | Cc | Subscribed ***@***.***> | | Subject | [yuhuixu1993/qa-lora] 'triton.compiler' has no attribute 'OutOfResources' (Issue #41) | Thank you for this excellent work!! may I ask some question? I use llama-7b to quantize and use qalora.py,and occur problem: except triton.compiler.OutOfResources: AttributeError: module 'triton.compiler' has no attribute 'OutOfResources' my triton vision: Version: 2.2.0 Summary: A language and compiler for custom Deep Learning operations Home-page: https://github.com/openai/triton/ Author: Philippe Tillet Author-email: ***@***.*** — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[davidray222]

thank you very much!!!!
I have successfully run it, based on other versions of Triton.

the next step is to use merge.py,right?
Thank you for taking the time to answer!

this is my merge code:
"import torch
model_path = "/mnt/8tb_raid/david_model/qa-lora/AutoGPTQ/examples/quantization/llama7b-quant4bit-g32/gptq_model-4bit-32g.bin"
lora_path = "/mnt/8tb_raid/david_model/qa-lora/output/checkpoint-10000/optimizer.pt"
merged_path = "/mnt/8tb_raid/david_model/qa-lora/lora_save/merged_model.bin"
scale = 16 /64
group_size = 128

model = torch.load(model_path, map_location='cpu')
lora = torch.load(lora_path, map_location='cpu')
tmp_keys = [key[17:-14] for key in lora.keys() if 'lora_A' in key]
for tmp_key in tmp_keys:
model[tmp_key+'.qzeros'] -= (lora['base_model.model.'+tmp_key+'.lora_B.weight'] @ lora['base_model.model.'+tmp_key+'.lora_A.weight']).t() * scale / group_size /model[tmp_key+'.scales']

torch.save(model, merged_path)

"