Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support visdrone dataset #704

Open
wants to merge 17 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/en/common_usage/amp_training.md
Original file line number Diff line number Diff line change
@@ -1 +1,13 @@
# Automatic mixed precision(AMP)training

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

记录不太对,这个不应该有才是

To enable Automatic Mixing Precision (AMP) training, add `--amp` to the end of the training command, which is as follows:

```shell
python tools/train.py python ./tools/train.py ${CONFIG} --amp
```

Specific examples are as follows:

```shell
python tools/train.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py --amp
```
8 changes: 8 additions & 0 deletions docs/en/common_usage/resume_training.md
Original file line number Diff line number Diff line change
@@ -1 +1,9 @@
# Resume training

Resume training means to continue training from the state saved from one of the previous trainings, where the state includes the model weights, the state of the optimizer and the optimizer parameter adjustment strategy.

The user can add `--resume` at the end of the training command to resume training, and the program will automatically load the latest weight file from `work_dirs` to resume training. If there is an updated checkpoint in `work_dir` (e.g. the training was interrupted during the last training), the training will be resumed from that checkpoint, otherwise (e.g. the last training did not have time to save the checkpoint or a new training task was started) the training will be restarted. Here is an example of resuming training:

```shell
python tools/train.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py --resume
```
12 changes: 12 additions & 0 deletions docs/zh_cn/common_usage/amp_training.md
Original file line number Diff line number Diff line change
@@ -1 +1,13 @@
# 自动混合精度(AMP)训练

如果要开启自动混合精度(AMP)训练,在训练命令最后加上 `--amp` 即可, 命令如下:

```shell
python tools/train.py python ./tools/train.py ${CONFIG} --amp
```

具体例子如下:

```shell
python tools/train.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py --amp
```
8 changes: 8 additions & 0 deletions docs/zh_cn/common_usage/resume_training.md
Original file line number Diff line number Diff line change
@@ -1 +1,9 @@
# 恢复训练

恢复训练是指从之前某次训练保存下来的状态开始继续训练,这里的状态包括模型的权重、优化器和优化器参数调整策略的状态。

用户可以在训练命令最后加上 `--resume` 恢复训练,程序会自动从 `work_dirs` 中加载最新的权重文件恢复训练。如果 `work_dir` 中有最新的 checkpoint(例如该训练在上一次训练时被中断),则会从该 checkpoint 恢复训练,否则(例如上一次训练还没来得及保存 checkpoint 或者启动了新的训练任务)会重新开始训练。下面是一个恢复训练的示例:

```shell
python tools/train.py configs/yolov5/yolov5_s-v61_syncbn_8xb16-300e_coco.py --resume
```
107 changes: 107 additions & 0 deletions tools/dataset_converters/visdrone2coco.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
import json
import os

import cv2
from tqdm import tqdm

# Recommended dataset download address:
# https://cdn.opendatalab.com/Visdrone_DET/raw/Visdrone_DET.tar.gz
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可以修改 download_dataset 脚本,然后用户直接用里面下载

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可以修改 download_dataset 脚本,然后用户直接用里面下载

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

并且在 docs/zh_cn/recommended_topics/dataset_preparation.md 里面详细说明整个数据准备过程。



def convert_visdrone_to_coco(ann_file, out_file, image_prefix):
id_num = 0
img_num = 0
categories = [{
'id': 0,
'name': 'ignored regions'
}, {
'id': 1,
'name': 'pedestrian'
}, {
'id': 2,
'name': 'people'
}, {
'id': 3,
'name': 'bicycle'
}, {
'id': 4,
'name': 'car'
}, {
'id': 5,
'name': 'van'
}, {
'id': 6,
'name': 'truck'
}, {
'id': 7,
'name': 'tricycle'
}, {
'id': 8,
'name': 'awning-tricycle'
}, {
'id': 9,
'name': 'bus'
}, {
'id': 10,
'name': 'motor'
}, {
'id': 11,
'name': 'others'
}]
images = []
annotations = []
print('start loading data')
set = os.listdir(ann_file)
annotations_path = ann_file
images_path = image_prefix
for i in tqdm(set):
f = open(annotations_path + '/' + i)
name = i.replace('.txt', '')
image = {}
height, width = cv2.imread(images_path + '/' + name + '.jpg').shape[:2]
file_name = name + '.jpg'
image['file_name'] = file_name
image['height'] = height
image['width'] = width
image['id'] = img_num
images.append(image)
for line in f.readlines():
annotation = {}
line = line.replace('\n', '')
if line.endswith(','):
line = line.rstrip(',')
line_list = [int(i) for i in line.split(',')]
bbox_xywh = [
line_list[0], line_list[1], line_list[2], line_list[3]
]
annotation['image_id'] = img_num
annotation['score'] = line_list[4]
annotation['bbox'] = bbox_xywh
annotation['category_id'] = int(line_list[5])
annotation['id'] = id_num
annotation['iscrowd'] = 0
annotation['segmentation'] = []
annotation['area'] = bbox_xywh[2] * bbox_xywh[3]
id_num += 1
annotations.append(annotation)
img_num += 1
dataset_dict = {}
dataset_dict['images'] = images
dataset_dict['annotations'] = annotations
dataset_dict['categories'] = categories
json_str = json.dumps(dataset_dict)
with open(out_file, 'w') as json_file:
json_file.write(json_str)


print('json file write done')

if __name__ == '__main__':
convert_visdrone_to_coco(
'../data/visdrone/VisDrone2019-DET-train/annotations',
'../data/visdrone/train_coco.json',
'../data/visdrone/VisDrone2019-DET-train/images')
convert_visdrone_to_coco(
'../data/visdrone/VisDrone2019-DET-val/annotations',
'../data/visdrone/val_coco.json',
'../data/visdrone/VisDrone2019-DET-val/images')