Evaluation：UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

[manlenzzz]

Thank you very much for doing such great open-source work!

i try:
CUDA_VISIBLE_DEVICES=X bash scripts/evaluate.sh PATH_OR_NAME_TO_BASE_MODEL PATH_TO_SAVE_TUNE_MODEL PATH_TO_PRUNE_MODEL EPOCHS_YOU_WANT_TO_EVALUATE

but get the result:
Selected Tasks: ['piqa', 'boolq', 'arc_challenge', 'hellaswag', 'openbookqa', 'arc_easy', 'winogrande']
Load from Pruned Model: prune_log/llama_prune/pytorch_model.bin
Config: LoraConfig(peft_type='LORA', base_model_name_or_path='baffo32/decapoda-research-llama-7B-hf', task_type='CAUSAL_LM', inference_mode=True, r=8, target_modules=['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'down_proj', 'up_proj'], lora_alpha=16, lora_dropout=0.05, fan_in_fan_out=False, bias='none', modules_to_save=None, init_lora_weights=True)
Load from adapter: adapter_model.bin
Traceback (most recent call last):
File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/main.py", line 115, in
main()
File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/main.py", line 81, in main
results = evaluator.simple_evaluate(
File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/utils.py", line 182, in _wrapper
return fn(*args, **kwargs)
File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/evaluator.py", line 81, in simple_evaluate
task_dict = lm_eval.tasks.get_task_dict(tasks)
File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/tasks/init.py", line 344, in get_task_dict
task_name_dict = {
File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/tasks/init.py", line 345, in
task_name: get_task(task_name)()
File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/base.py", line 456, in init
self.download(data_dir, cache_dir, download_mode)
File "/mnt/data/user/zhouchanghai/LLM-Pruner/lm-evaluation-harness/lm_eval/base.py", line 485, in download
self.dataset = datasets.load_dataset(
File "/mnt/data/user/zhouchanghai/LLM-Pruner/LLM-Pruner/lib/python3.10/site-packages/datasets/load.py", line 2587, in load_dataset
builder_instance = load_dataset_builder(
File "/mnt/data/user/zhouchanghai/LLM-Pruner/LLM-Pruner/lib/python3.10/site-packages/datasets/load.py", line 2259, in load_dataset_builder
dataset_module = dataset_module_factory(
File "/mnt/data/user/zhouchanghai/LLM-Pruner/LLM-Pruner/lib/python3.10/site-packages/datasets/load.py", line 1910, in dataset_module_factory
raise e1 from None
File "/mnt/data/user/zhouchanghai/LLM-Pruner/LLM-Pruner/lib/python3.10/site-packages/datasets/load.py", line 1862, in dataset_module_factory
can_load_config_from_parquet_export = "DEFAULT_CONFIG_NAME" not in f.read()
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

[BrownTan]

I think you didn't use a magic.