[CodeCamp2023-470] Runner supports setting the number of iterations for each epoch

[ShuRaymond]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

One of Openmmlab codecamp task.

Modification

Modified _flexible_runner.py and runner.py so that FlexibleRunner supports setting the number of rounds per epoch iteration to save debugging time.

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMCls.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[CLAassistant]

All committers have signed the CLA.

[HAOCHENYE]

Hi, we should also update a unit test to validate this feature works as expected 😄

[zhouzaida]

Hi @ShuRaymond , thanks for your contribution.

Here are several comments:

1. No need to modify FlexibleRunner, as num_batch_per_epoch can be passed to the loop directly through train_cfg, val_cfg, or test_cfg.
2. No need to update IterBasedTrainLoop
3. Unit tests need to be added.

   • mmengine/tests/test_runner/test_runner.py

     Line 1304 in 398d229

     def test_build_train_loop(self):

   • mmengine/tests/test_runner/test_runner.py

     Line 1350 in 398d229

     def test_build_val_loop(self):

   • mmengine/tests/test_runner/test_runner.py

     Line 1377 in 398d229

     def test_build_test_loop(self):

4. Documentation needs to be updated.
   • docs/zh_cn/common_usage/debug_tricks.md
   • docs/en/common_usage/debug_tricks.md

[ShuRaymond]

Hi, we should also update a unit test to validate this feature works as expected 😄

thanks for reminding, I am doing it.

[ShuRaymond]

Hi @ShuRaymond , thanks for your contribution.

Here are several comments:

1. No need to modify FlexibleRunner, as num_batch_per_epoch can be passed to the loop directly through train_cfg, val_cfg, or test_cfg.

2. No need to update IterBasedTrainLoop

3. Unit tests need to be added.

   • mmengine/tests/test_runner/test_runner.py

     Line 1304 in 398d229

     def test_build_train_loop(self):

   • mmengine/tests/test_runner/test_runner.py

     Line 1350 in 398d229

     def test_build_val_loop(self):

   • mmengine/tests/test_runner/test_runner.py

     Line 1377 in 398d229

     def test_build_test_loop(self):

4. Documentation needs to be updated.

   • docs/zh_cn/common_usage/debug_tricks.md
   • docs/en/common_usage/debug_tricks.md

thanks for reminding and teaching, just done it.

[zhouzaida]

Hi, also need to add several unit tests (whether the num_batch_per_epoch works as expected) in the following methods:

mmengine/tests/test_runner/test_runner.py

Line 1432 in bbd416a

def test_train(self):

mmengine/tests/test_runner/test_runner.py

Line 1841 in bbd416a

def test_val(self):

mmengine/tests/test_runner/test_runner.py

Line 1920 in bbd416a

def test_test(self):

• test_train

def test_train(self):
    # 15 test num_batch_per_epoch
    cfg = copy.deepcopy(self.epoch_based_cfg)
    cfg.train_cfg = dict(
        by_epoch=True,
        max_epochs=3,
        num_batch_per_epoch=2,
    )
    runner = Runner.from_cfg(cfg)
    runner.train()
    self.assertEqual(runner.iter, 3 * 2)

[zhouzaida]

Suggested change

[zhouzaida]

Also, need to update docstring in

mmengine/mmengine/runner/loops.py

Line 33 in bbd416a

corresponding milestone. Defaults to None.

num_batch_per_epoch (int, optional): 

[zhouzaida]

Adding a new parameter in the middle position may cause a bc issue. Suggest moving it to the end.

[HAOCHENYE]

Suggested change

# Copyright (c) OpenMMLab. All rights reserved.

[HAOCHENYE]

Suggested change

	During the process of debugging code, sometimes it is necessary to train for several epochs, such as debugging the validation process or checking whether the checkpoint saving meets expectations. However, if the dataset is too large, it may take a long time to complete one epoch, in which case the cfg parameter can be added.
	During the process of debugging code, sometimes it is necessary to train for several epochs, such as debugging the validation process or checking whether the checkpoint saving meets expectations. However, if the dataset is too large, it may take a long time to complete one epoch, in which case the `num_batch_per_epoch` could be configured:

[HAOCHENYE]

Why should we configure this env variable?

[ShuRaymond]

This one is to solve my conda environment and torch package conflict. I will delete it in commit.

[HAOCHENYE]

Suggested change

	Take `MMEngine` as an example（Refer to the [documentation](https://mmengine.readthedocs.io/zh_CN/latest/get_started/installation.html)for installing MMEngine）。
	Example of a training script

[HAOCHENYE]

Suggested change

	Fast debugging is achieved by adding the `num_batch_per_epoch` parameter to `train_dataloader` and `val_dataloader`.
	Fast debugging is achieved by configuring the `num_batch_per_epoch` in `train_dataloader` and `val_dataloader`. You can quickly debug the code of the validation after just 5 training iterations,

[HAOCHENYE]

Suggested change

Run the training script. You can see that after running each epoch run 5 batch is over. Compared to the original, debugging is faster and more flexible.

[zhouzaida]

FlexibleRunner supports setting the number of iterations for each epoch