-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
0fe2b5f
commit d4002f6
Showing
1 changed file
with
6 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -17,10 +17,10 @@ We propose a paradigm **Bro**wse and Concentra**te** (**Brote**) for incorporati | |
|
||
## Instructions For Training and Inference | ||
|
||
### Data | ||
### 1. Data | ||
Please refer to the data format described in [MIC](https://github.com/HaozheZhao/MIC). | ||
|
||
#### 1. Data for pretraining. | ||
#### 1.1 Data for pretraining. | ||
|
||
We create a dataset of 56k fewshot data samples, resulting in 191k training instances (one image per instance). These instances are supposed to contain question-aware and cross-image information. The data construction pipeline is illustrated in the following figure. | ||
|
||
|
@@ -30,24 +30,24 @@ We create a dataset of 56k fewshot data samples, resulting in 191k training inst | |
|
||
The pretraining data will be available by the end of this week (08 Dec). | ||
|
||
#### 2. Data for finetuning. | ||
#### 1.2 Data for finetuning. | ||
|
||
We sampled about 500k data from MIC for model finetuning. | ||
|
||
### Environment | ||
### 2. Environment | ||
``` | ||
pip install -r requirements.txt | ||
``` | ||
|
||
### Training | ||
### 3. Training | ||
~~coming soon~~ | ||
|
||
~~(Please raise issues regarding the training scripts, or reach out at [this email](mailto:[email protected]).~~ | ||
|
||
The training scripts will be available by the end of this week (08 Dec). | ||
|
||
|
||
### Inference | ||
### 4. Inference | ||
Please refer to the test.py file; files under the **model** dir are for test only, and will be updated soon for training. | ||
|
||
To run the test script (ensure the required libraries are properly installed): | ||
|