update readme

README.md

@@ -179,6 +179,16 @@ deepspeed_config:
   zero_stage: 3
 ```
 
+#### Inference with Your Trained Model
+
+First, you need to extract the FP32 consolidated weights from ZeRO 1, 2, or 3 DeepSpeed checkpoints:
+```bash
+cd /path/to/your/output/dir
+python zero_to_fp32.py . pytorch_model.bin
+```
+
+See [inference.md](docs/inference.md) for instructions on how to perform inference with the fine-tuned model.
+
 ## Citation
 If you find our work helpful, please consider citing.
 ```

examples/nextqa/README.md

@@ -30,6 +30,12 @@ accelerate launch --config_file recipes/accelerate_configs/zero3_offload.yaml ar
 ```
 
 # Evaluation and Results
+> **Note:** If you train full params with DeepSpeed ZeRO, you need to extract the fp32 consolidated weights from ZeRO 1, 2, or 3 DeepSpeed checkpoints:
+> ```bash
+> cd /path/to/your/output/dir
+> python zero_to_fp32.py . pytorch_model.bin
+> ```
+
 After modifying the dataset paths in [NextQA-Evaluation](../../examples/nextqa/evaluation.py#L45), run::
 ```bash
 CUDA_VISIBLE_DEVICES=0 python examples/nextqa/evaluation.py \

examples/nlvr2/README.md

@@ -44,6 +44,12 @@ accelerate launch --config_file recipes/accelerate_configs/zero3_offload.yaml ar
 ```
 
 # Evaluation and Results
+> **Note:** If you train full params with DeepSpeed ZeRO, you need to extract the fp32 consolidated weights from ZeRO 1, 2, or 3 DeepSpeed checkpoints:
+> ```bash
+> cd /path/to/your/output/dir
+> python zero_to_fp32.py . pytorch_model.bin
+> ```
+
 After modifying the dataset paths in [NLVR2-Evaluation](../../examples/nlvr2/evaluation.py#L45), you can run:
 ```bash
 CUDA_VISIBLE_DEVICES=0 python examples/nlvr2/evaluation.py \

examples/refcoco/README.md

@@ -29,6 +29,12 @@ accelerate launch --config_file recipes/accelerate_configs/zero3_offload.yaml ar
 ```
 
 # Inference
+> **Note:** If you train full params with DeepSpeed ZeRO, you need to extract the fp32 consolidated weights from ZeRO 1, 2, or 3 DeepSpeed checkpoints:
+> ```bash
+> cd /path/to/your/output/dir
+> python zero_to_fp32.py . pytorch_model.bin
+> ```
+
 We provide an [infernece script](./inference.py) to predict bounding box coordinates according to the input description of reference object, as shown:
 ![](../../assets/refcoco_example1.png)
 
@@ -45,6 +51,12 @@ CUDA_VISIBIE_DEVICES=0 python examples/refcoco/inference.py \
 
 
 # Evaluation and Results
+> **Note:** If you train full params with DeepSpeed ZeRO, you need to extract the fp32 consolidated weights from ZeRO 1, 2, or 3 DeepSpeed checkpoints:
+> ```bash
+> cd /path/to/your/output/dir
+> python zero_to_fp32.py . pytorch_model.bin
+> ```
+
 After modifying the dataset paths in [RefCOCO-Evaluation](../../examples/refcoco/evaluation.py#L47), run:
 ```bash
 CUDA_VISIBLE_DEVICES=0 python examples/refcoco/evaluation.py \