Merge branch 'open-mmlab:main' into main

docs/en/api/visualization.rst

@@ -35,3 +35,4 @@ visualization Backend
    TensorboardVisBackend
    WandbVisBackend
    ClearMLVisBackend
+   NeptuneVisBackend

docs/en/common_usage/visualize_training_log.md

@@ -1,12 +1,12 @@
 # Visualize Training Logs
 
-MMEngine integrates experiment management tools such as [TensorBoard](https://www.tensorflow.org/tensorboard), [Weights & Biases (WandB)](https://docs.wandb.ai/), [MLflow](https://mlflow.org/docs/latest/index.html) and [ClearML](https://clear.ml/docs/latest/docs), making it easy to track and visualize metrics like loss and accuracy.
+MMEngine integrates experiment management tools such as [TensorBoard](https://www.tensorflow.org/tensorboard), [Weights & Biases (WandB)](https://docs.wandb.ai/), [MLflow](https://mlflow.org/docs/latest/index.html), [ClearML](https://clear.ml/docs/latest/docs) and [Neptune](https://docs.neptune.ai/), making it easy to track and visualize metrics like loss and accuracy.
 
 Below, we'll show you how to configure an experiment management tool in just one line, based on the example from [15 minutes to get started with MMEngine](../get_started/15_minutes.md).
 
 ## TensorBoard
 
-Configure the `visualizer` in the initialization parameters of the Runner, and set `vis_backends` to `TensorboardVisBackend`.
+Configure the `visualizer` in the initialization parameters of the Runner, and set `vis_backends` to [TensorboardVisBackend](mmengine.visualization.TensorboardVisBackend).
 
 ```python
 runner = Runner(
@@ -32,7 +32,7 @@ pip install wandb
 wandb login
 ```
 
-Configure the `visualizer` in the initialization parameters of the Runner, and set `vis_backends` to `WandbVisBackend`.
+Configure the `visualizer` in the initialization parameters of the Runner, and set `vis_backends` to [WandbVisBackend](mmengine.visualization.WandbVisBackend).
 
 ```python
 runner = Runner(
@@ -81,7 +81,7 @@ pip install clearml
 clearml-init
 ```
 
-Configure the `visualizer` in the initialization parameters of the Runner, and set `vis_backends` to `ClearMLVisBackend`.
+Configure the `visualizer` in the initialization parameters of the Runner, and set `vis_backends` to [ClearMLVisBackend](mmengine.visualization.ClearMLVisBackend).
 
 ```python
 runner = Runner(
@@ -99,3 +99,53 @@ runner.train()
 ```
 
 ![image](https://github.com/open-mmlab/mmengine/assets/58739961/d68e1dd2-9e82-40fb-ad81-00a647549adc)
+
+## Neptune
+
+Before using Neptune, you need to install `neptune` dependency library and refer to [Neptune.AI](https://docs.neptune.ai/) for configuration.
+
+```bash
+pip install neptune
+```
+
+Configure the `Runner` in the initialization parameters of the Runner, and set `vis_backends` to [NeptuneVisBackend](mmengine.visualization.NeptuneVisBackend).
+
+```python
+runner = Runner(
+    model=MMResNet50(),
+    work_dir='./work_dir',
+    train_dataloader=train_dataloader,
+    optim_wrapper=dict(optimizer=dict(type=SGD, lr=0.001, momentum=0.9)),
+    train_cfg=dict(by_epoch=True, max_epochs=5, val_interval=1),
+    val_dataloader=val_dataloader,
+    val_cfg=dict(),
+    val_evaluator=dict(type=Accuracy),
+    visualizer=dict(type='Visualizer', vis_backends=[dict(type='NeptuneVisBackend')]),
+)
+runner.train()
+```
+
+![image](https://github.com/open-mmlab/mmengine/assets/58739961/9122e2ac-cc4f-43b2-bad3-ae33faa64043)
+
+Please note: If the `project` and `api_token` are not specified, neptune will be set to offline mode and the generated files will be saved to the local `.neptune` file.
+It is recommended to specify the `project` and `api_token` during initialization as shown below.
+
+```python
+runner = Runner(
+    ...
+    visualizer=dict(
+        type='Visualizer',
+        vis_backends=[
+            dict(
+                type='NeptuneVisBackend',
+                init_kwargs=dict(project='workspace-name/project-name',
+                                 api_token='your api token')
+            ),
+        ],
+    ),
+    ...
+)
+runner.train()
+```
+
+More initialization configuration parameters are available at [neptune.init_run API](https://docs.neptune.ai/api/neptune/#init_run).

docs/zh_cn/api/visualization.rst

@@ -35,3 +35,4 @@ visualization Backend
    TensorboardVisBackend
    WandbVisBackend
    ClearMLVisBackend
+   NeptuneVisBackend

docs/zh_cn/common_usage/visualize_training_log.md

@@ -1,12 +1,12 @@
 # 可视化训练日志
 
-MMEngine 集成了 [TensorBoard](https://www.tensorflow.org/tensorboard?hl=zh-cn)、[Weights & Biases (WandB)](https://docs.wandb.ai/)、[MLflow](https://mlflow.org/docs/latest/index.html) 和 [ClearML](https://clear.ml/docs/latest/docs) 实验管理工具，你可以很方便地跟踪和可视化损失及准确率等指标。
+MMEngine 集成了 [TensorBoard](https://www.tensorflow.org/tensorboard?hl=zh-cn)、[Weights & Biases (WandB)](https://docs.wandb.ai/)、[MLflow](https://mlflow.org/docs/latest/index.html) 、[ClearML](https://clear.ml/docs/latest/docs) 和 [Neptune](https://docs.neptune.ai/) 实验管理工具，你可以很方便地跟踪和可视化损失及准确率等指标。
 
 下面基于[15 分钟上手 MMENGINE](../get_started/15_minutes.md)中的例子介绍如何一行配置实验管理工具。
 
 ## TensorBoard
 
-设置 `Runner` 初始化参数中的 `visualizer`，并将 `vis_backends` 设置为 `TensorboardVisBackend`。
+设置 `Runner` 初始化参数中的 `visualizer`，并将 `vis_backends` 设置为 [TensorboardVisBackend](mmengine.visualization.TensorboardVisBackend)。
 
 ```python
 runner = Runner(
@@ -32,7 +32,7 @@ pip install wandb
 wandb login
 ```
 
-设置 `Runner` 初始化参数中的 `visualizer`，并将 `vis_backends` 设置为 `WandbVisBackend`。
+设置 `Runner` 初始化参数中的 `visualizer`，并将 `vis_backends` 设置为 [WandbVisBackend](mmengine.visualization.WandbVisBackend)。
 
 ```python
 runner = Runner(
@@ -81,7 +81,7 @@ pip install clearml
 clearml-init
 ```
 
-设置 `Runner` 初始化参数中的 `visualizer`，并将 `vis_backends` 设置为 `ClearMLVisBackend`。
+设置 `Runner` 初始化参数中的 `visualizer`，并将 `vis_backends` 设置为 [ClearMLVisBackend](mmengine.visualization.ClearMLVisBackend)。
 
 ```python
 runner = Runner(
@@ -99,3 +99,53 @@ runner.train()
 ```
 
 ![image](https://github.com/open-mmlab/mmengine/assets/58739961/d68e1dd2-9e82-40fb-ad81-00a647549adc)
+
+## Neptune
+
+使用 Neptune 前需先安装依赖库 `neptune` 并登录 [Neptune.AI](https://docs.neptune.ai/) 进行配置。
+
+```bash
+pip install neptune
+```
+
+设置 `Runner` 初始化参数中的 `visualizer`，并将 `vis_backends` 设置为 [NeptuneVisBackend](mmengine.visualization.NeptuneVisBackend)。
+
+```python
+runner = Runner(
+    model=MMResNet50(),
+    work_dir='./work_dir',
+    train_dataloader=train_dataloader,
+    optim_wrapper=dict(optimizer=dict(type=SGD, lr=0.001, momentum=0.9)),
+    train_cfg=dict(by_epoch=True, max_epochs=5, val_interval=1),
+    val_dataloader=val_dataloader,
+    val_cfg=dict(),
+    val_evaluator=dict(type=Accuracy),
+    visualizer=dict(type='Visualizer', vis_backends=[dict(type='NeptuneVisBackend')]),
+)
+runner.train()
+```
+
+![image](https://github.com/open-mmlab/mmengine/assets/58739961/9122e2ac-cc4f-43b2-bad3-ae33faa64043)
+
+请注意：若未提供 `project` 和 `api_token` ，neptune 将被设置成离线模式，产生的文件将保存到本地 `.neptune` 文件下。
+推荐在初始化时提供 `project` 和 `api_token` ，具体方法如下所示：
+
+```python
+runner = Runner(
+    ...
+    visualizer=dict(
+        type='Visualizer',
+        vis_backends=[
+            dict(
+                type='NeptuneVisBackend',
+                init_kwargs=dict(project='workspace-name/project-name',
+                                 api_token='your api token')
+            ),
+        ],
+    ),
+    ...
+)
+runner.train()
+```
+
+更多初始化配置参数可点击 [neptune.init_run API](https://docs.neptune.ai/api/neptune/#init_run) 查询。

mmengine/visualization/__init__.py

@@ -1,10 +1,11 @@
 # Copyright (c) OpenMMLab. All rights reserved.
 from .vis_backend import (BaseVisBackend, ClearMLVisBackend, LocalVisBackend,
-                          MLflowVisBackend, TensorboardVisBackend,
-                          WandbVisBackend)
+                          MLflowVisBackend, NeptuneVisBackend,
+                          TensorboardVisBackend, WandbVisBackend)
 from .visualizer import Visualizer
 
 __all__ = [
     'Visualizer', 'BaseVisBackend', 'LocalVisBackend', 'WandbVisBackend',
-    'TensorboardVisBackend', 'MLflowVisBackend', 'ClearMLVisBackend'
+    'TensorboardVisBackend', 'MLflowVisBackend', 'ClearMLVisBackend',
+    'NeptuneVisBackend'
 ]

mmengine/visualization/vis_backend.py

@@ -986,3 +986,147 @@ def close(self) -> None:
         for file_path in file_paths:
             self._task.upload_artifact(os.path.basename(file_path), file_path)
         self._task.close()
+
+
+@VISBACKENDS.register_module()
+class NeptuneVisBackend(BaseVisBackend):
+    """Neptune visualization backend class.
+
+    Examples:
+        >>> from mmengine.visualization import NeptuneVisBackend
+        >>> from mmengine import Config
+        >>> import numpy as np
+        >>> init_kwargs = {'project': 'your_project_name'}
+        >>> neptune_vis_backend = NeptuneVisBackend(init_kwargs=init_kwargs)
+        >>> img = np.random.randint(0, 256, size=(10, 10, 3))
+        >>> neptune_vis_backend.add_image('img', img)
+        >>> neptune_vis_backend.add_scalar('mAP', 0.6)
+        >>> neptune_vis_backend.add_scalars({'loss': 0.1, 'acc': 0.8})
+        >>> cfg = Config(dict(a=1, b=dict(b1=[0, 1])))
+        >>> neptune_vis_backend.add_config(cfg)
+
+    Note:
+        `New in version 0.8.5.`
+
+    Args:
+        save_dir (str, optional): The root directory to save the files
+            produced by the visualizer. NeptuneVisBackend does
+            not require this argument. Defaults to None.
+        init_kwargs (dict, optional): Neptune initialization parameters.
+            Defaults to None.
+
+            - project (str): Name of a project in a form of
+              `namespace/project_name`. If `project` is not specified,
+              the value of `NEPTUNE_PROJECT` environment variable
+              will be taken.
+            - api_token (str): User's API token. If api_token is not api_token,
+              the value of `NEPTUNE_API_TOKEN` environment variable will
+              be taken. Note: It is strongly recommended to use
+              `NEPTUNE_API_TOKEN` environment variable rather than
+              placing your API token here.
+
+            If 'project' and 'api_token are not specified in `init_kwargs`,
+            the 'mode' will be set to 'offline'.
+            See `neptune.init_run
+            <https://docs.neptune.ai/api/neptune/#init_run>`_ for
+            details.
+    """
+
+    def __init__(self,
+                 save_dir: Optional[str] = None,
+                 init_kwargs: Optional[dict] = None):
+        super().__init__(save_dir)  # type:ignore
+        self._init_kwargs = init_kwargs
+
+    def _init_env(self):
+        """Setup env for neptune."""
+        try:
+            import neptune
+        except ImportError:
+            raise ImportError(
+                'Please run "pip install -U neptune" to install neptune')
+        if self._init_kwargs is None:
+            self._init_kwargs = {'mode': 'offline'}
+
+        self._neptune = neptune.init_run(**self._init_kwargs)
+
+    @property  # type: ignore
+    @force_init_env
+    def experiment(self):
+        """Return Neptune object."""
+        return self._neptune
+
+    @force_init_env
+    def add_config(self, config: Config, **kwargs) -> None:
+        """Record the config to neptune.
+
+        Args:
+            config (Config): The Config object
+        """
+        from neptune.types import File
+        self._neptune['config'].upload(File.from_content(config.pretty_text))
+
+    @force_init_env
+    def add_image(self,
+                  name: str,
+                  image: np.ndarray,
+                  step: int = 0,
+                  **kwargs) -> None:
+        """Record the image.
+
+        Args:
+            name (str): The image identifier.
+            image (np.ndarray): The image to be saved. The format
+                should be RGB. Defaults to None.
+            step (int): Global step value to record. Defaults to 0.
+        """
+        from neptune.types import File
+
+        # values in the array need to be in the [0, 1] range
+        img = image.astype(np.float32) / 255.0
+        self._neptune['images'].append(
+            File.as_image(img), name=name, step=step)
+
+    @force_init_env
+    def add_scalar(self,
+                   name: str,
+                   value: Union[int, float],
+                   step: int = 0,
+                   **kwargs) -> None:
+        """Record the scalar.
+
+        Args:
+            name (str): The scalar identifier.
+            value (int, float): Value to save.
+            step (int): Global step value to record. Defaults to 0.
+        """
+        self._neptune[name].append(value, step=step)
+
+    @force_init_env
+    def add_scalars(self,
+                    scalar_dict: dict,
+                    step: int = 0,
+                    file_path: Optional[str] = None,
+                    **kwargs) -> None:
+        """Record the scalars' data.
+
+        Args:
+            scalar_dict (dict): Key-value pair storing the tag and
+                corresponding values.
+            step (int): Global step value to record. Defaults to 0.
+            file_path (str, optional): The scalar's data will be
+                saved to the `file_path` file at the same time
+                if the `file_path` parameter is specified.
+                Defaults to None.
+        """
+        assert isinstance(scalar_dict, dict)
+        assert 'step' not in scalar_dict, 'Please set it directly ' \
+                                          'through the step parameter'
+
+        for k, v in scalar_dict.items():
+            self._neptune[k].append(v, step=step)
+
+    def close(self) -> None:
+        """close an opened object."""
+        if hasattr(self, '_neptune'):
+            self._neptune.stop()

requirements/tests.txt

@@ -4,5 +4,6 @@ dadaptation
 lion-pytorch
 lmdb
 mlflow
+neptune
 parameterized
 pytest