Skip to content

Latest commit

 

History

History
53 lines (41 loc) · 1.87 KB

README-ReaSCAN.md

File metadata and controls

53 lines (41 loc) · 1.87 KB

NeSyCoCo

ReaSCAN Dataset (Original)

Step 1: Prepare the dataset. Download from here. To replicate the experiments, you need to prepare your dataset as the following.

data/ReaSCAN
├── ReaSCAN-compositional
│   ├── train.json
│   ├── dev.json
│   └── test.json
├── ReaSCAN-compositional-a1
│   └── test.json
├── ReaSCAN-compositional-a2
│   └── test.json
├── ReaSCAN-compositional-a3
.
.
.

Step 2: Preprocess ReaSCAN Queries

python scripts/reascan_preprocess.py

To use the program generated by LLaMa 3.1, use the files inside data/ReaSCAN. To generate those data-parses files, run the commands inside the prompts/ directory. You need to install transformers pip install transformers before running the command.

jac-run run-llama-prompt.py --dataset reascan --questions <PATH_TO_REASCAN_JSON_FILE>  --output reascan-parsing.pkl --prompt prompts-reascan.txt

Step 3: Generate ReaSCAN Images

python  scripts/reascan_image_generator.py

Step 4: Training .

jac-crun 0 scripts/trainval-reascan.py --desc experiments/desc_reascan_nesycoco.py --data-dir data/ReaSCAN/ --validation-data-dir data/ReaSCAN/ --data-parses <prompts> --curriculum all --expr reascan --validation-interval 5 --data-tvsplit 0.95 --save-best-on validation/validation/acc/refexp --lr 1e-4 --val-split dev_comp_3500

Step 5: Evaluation. To evaluate each of the test splits

jac-crun 0 scripts/trainval-reascan.py --desc experiments/desc_reascan_nesycoco.py --data-dir data/ReaSCAN/ --validation-data-dir data/ReaSCAN/ --data-parses <prompts> --curriculum all --expr reascan --validation-interval 5 --data-tvsplit 0.95 --save-best-on validation/validation/acc/refexp --lr 1e-4 --val-split <SPLIT>
  --load <TRAINED_CHECKPOINT_FILE> --evaluate