update experiment results of nlvr2

rhymes-ai · Oct 4, 2024 · bfd36fa · bfd36fa
1 parent 1b2d41b
commit bfd36fa
Show file tree

Hide file tree

Showing 5 changed files with 5 additions and 5 deletions.
diff --git a/assets/nlvr2_loss_490_full.png b/assets/nlvr2_loss_490_full.png
diff --git a/assets/nlvr2_loss_980_full.png b/assets/nlvr2_loss_980_full.png
diff --git a/examples/nlvr2/README.md b/examples/nlvr2/README.md
@@ -63,8 +63,8 @@ CUDA_VISIBLE_DEVICES=0 python examples/nlvr2/evaluation.py \
 The `Accuracy`:
 |        | Aria                           | LoRA SFT               | Full Params SFT  |
 |:--------:|:-------------------------------------:|:-------------------------:|:-------:|
-|490 |86.56 | 91.32 | 92.24 |
-|980 |87.03 | 91.61 | 92.33 |
+|490 |86.56 | 91.32 | 88.39 |
+|980 |87.03 | 91.61 | 89.44 |
 
 # Loss Curve
 These are the loss curves of `LoRA Finetuning` (left) and `Full Params Finetuning` (right) with 490 and 980 `max_image_size`:

diff --git a/examples/nlvr2/config_full.yaml b/examples/nlvr2/config_full.yaml
@@ -20,7 +20,7 @@ resume_from_checkpoint: false # resume from the latest checkpoint in output_dir
 ## hyper params
 num_train_epochs: 1
 eval_strategy: 'no'
-learning_rate: 5e-6
+learning_rate: 5e-7
 weight_decay: 0.1
 adam_beta2: 0.95
 warmup_ratio: 0.01

diff --git a/examples/nlvr2/eval.sh b/examples/nlvr2/eval.sh
@@ -1,7 +1,7 @@
 #!/bin/bash
 
-BASE_MODEL=/root/workspace/models/aria
-TOKENIZER=/root/workspace/models/aria
+BASE_MODEL="[YOUR_ARIA_PATH]"
+TOKENIZER=rhymes-ai/Aria
 LORA=""
 IMG_SIZE=980
 SAVE_ROOT=./eval/nlvr2__$(basename $BASE_MODEL)__$(basename $LORA)__$IMG_SIZE