update name; doc

README.md

@@ -1,6 +1,6 @@
 # DI-smartcross
 
-<img src="./docs/figs/di-smartcross_logo.png" width="200" alt="icon"/>
+<img src="./docs/figs/di-smartcross_banner.png" alt="icon"/>
 
 DI-smartcross - Decision Intelligence Platform for Traffic Crossing Signal Control.
 
@@ -84,6 +84,7 @@ We appreciate all contributions to improve DI-smartcross, both algorithms and sy
 DI-smartcross released under the Apache 2.0 license.
 
 ## Citation
+
 ```latex
 @misc{smartcross,
     title={{DI-smartcross: OpenDILab} Decision Intelligence platform for Traffic Crossing Signal Control},

docs/source/envs/cf_grid_env.rst

@@ -0,0 +1,2 @@
+CityFlow Grid Env
+#####################

docs/source/faq.rst

@@ -0,0 +1,15 @@
+FAQ
+##############
+
+.. toctree::
+    :maxdepth: 2
+
+
+Q1: SUMO environment always showing `Retrying in 1 seconds`
+------------------------------------------------------------------
+
+:A1:
+    SUMO environments and `traci` lib is slow to reset when running with large roadnets.
+    It only check the collection after reset for 1 sec. DI-smartcross provides an easy way
+    to change the retry timeout for `traci`. You can run `modify_traci_connect_timeout.sh`
+    file. It will automatically

docs/source/index.rst

@@ -1,4 +1,4 @@
-.. DI-drive documentation master file, created by
+.. DI-smartcross documentation master file, created by
    sphinx-quickstart on Mon Jan 25 13:49:15 2021.
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
@@ -14,15 +14,14 @@ DI-smartcross Documentation
    installation
    quick_start
    rl_environments
+   faq
 
-
-.. figure:: ../figs/di-smartcross_logo.png
+.. figure:: ../figs/di-smartcross_banner.png
    :alt: DI-smartcross
-   :width: 500px
 
 Decision Intelligence Platform for Traffic Crossing Signal Control.
 
-Last updated on 2022.03.20
+Last updated on 2022.04.16
 
 -----
 
@@ -50,10 +49,13 @@ Content
 ==============
 
 `Installation <installation.html>`_
---------------------------------------
+------------------------------------------
+
+`Quick Start <quickstart.html>`_
+-------------------------------------
 
-`Quick Start <quickstart>`_
------------------------------
+`RL Environments <rl_environments.html>`_
+-------------------------------------------------
 
-`RL Environments <rl_environments>`_
-----------------------------------------
+`FAQ <faq.html>`_
+--------------------

docs/source/installation.rst

@@ -4,10 +4,17 @@ Installation
 .. toctree::
     :maxdepth: 2
 
+Here we provide easy installation for **DI-smartcross** and all simulators supported.
+
+.. note::
+
+    You can choose one of the simulators to run your experiments. Only chosen one needs to be installed.
+
+
 SUMO installation
 =====================
 
-**DI-smartcross** support SUMO version >= 1.6.0. Here we show two easy guides
+**DI-smartcross** supports SUMO version >= 1.6.0. Here we show two easy guides
 of SUMO installation on Linux.
 
 Install SUMO via apt-get or homebrew
@@ -92,6 +99,23 @@ If successful, the following message will be shown in the shell.
     License EPL-2.0: Eclipse Public License Version 2 <https://eclipse.org/legal/epl-v20.html>
     Use --help to get the list of options.
 
+
+CityFlow Installation
+==========================
+
+CityFlow simulator can be installed from source code via `CMake <https://cmake.org>`_.
+Please make sure it is correctly worked in your system.
+
+Simply download their source code and run ``pip install`` in the root folder to install CityFlow.
+
+.. code:: bash
+
+    git clone https://github.com/cityflow-project/CityFlow.git
+    cd CityFlow
+    pip install .
+
+You can check installation by running ``import cityflow`` in python.
+
 Install DI-smartcross
 ==========================
 

docs/source/quick_start.rst

@@ -15,11 +15,13 @@ to get detail instructions of these configs.
 train RL policies
 --------------------
 
+The type of policy can be automatically parsed from the config file.
+
 .. code::
 
     usage: sumo_train [-h] -d DING_CFG -e ENV_CFG [-s SEED] [--dynamic-flow]
-                  [-cn COLLECT_ENV_NUM] [-en EVALUATE_ENV_NUM]
-                  [--exp-name EXP_NAME]
+                      [-cn COLLECT_ENV_NUM] [-en EVALUATE_ENV_NUM]
+                      [--exp-name EXP_NAME]
 
     DI-smartcross training script
 
@@ -40,18 +42,25 @@ train RL policies
 
 Example of running DQN in wj3 env with default config.
 
+.. note:: 
+
+    Running with dynamic flow is only supported for arterial7 env currently.
+
 .. code:: bash
 
     sumo_train -e smartcross/envs/sumo_wj3_default_config.yaml -d entry/config/sumo_wj3_dqn_default_config.py
 
 evaluate existing policies
 --------------------------------
 
+We provide two eval policies: random and fixed-time. You can choose one to evaluate
+as comparison. It is suggested to use the `eval_default_config` for each env.
+
 .. code:: 
 
     usage: sumo_eval [-h] [-d DING_CFG] -e ENV_CFG [-s SEED]
-                 [-p {random,fix,dqn,rainbow,ppo}] [--dynamic-flow]
-                 [-n ENV_NUM] [--gui] [-c CKPT_PATH]
+                     [-p {random,fix,dqn,rainbow,ppo}] [--dynamic-flow]
+                     [-n ENV_NUM] [--gui] [-c CKPT_PATH]
 
     DI-smartcross testing script
 
@@ -76,4 +85,63 @@ Example of running random policy in wj3 env.
 
 .. code:: bash
 
-    sumo_eval -p random -e smartcross/envs/sumo_wj3_default_config.yaml
+    sumo_eval -p random -e smartcross/envs/sumo_wj3_default_config.yaml
+
+
+CityFlow Entries
+=================
+
+**DI-smartcross** provides a simple DQN and Off-policy PPO demo for CityFlow env. Each
+policy comes with a default **DI-engine** configs is provided for each policy. You can
+check the document of DI-engine to get detail instructions of these configs.
+
+train RL policies
+--------------------
+
+.. code::
+
+    usage: cityflow_train [-h] -d DING_CFG -e ENV_CFG [-s SEED]
+                          [-cn COLLECT_ENV_NUM] [-en EVALUATE_ENV_NUM]
+                          [--exp-name EXP_NAME]
+
+    DI-smartcross training script
+
+    optional arguments:
+    -h, --help            show this help message and exit
+    -d DING_CFG, --ding-cfg DING_CFG
+                            DI-engine configuration path
+    -e ENV_CFG, --env-cfg ENV_CFG
+                            cityflow json configuration path
+    -s SEED, --seed SEED  random seed
+    -cn COLLECT_ENV_NUM, --collect-env-num COLLECT_ENV_NUM
+                            collector env num for training
+    -en EVALUATE_ENV_NUM, --evaluate-env-num EVALUATE_ENV_NUM
+                            evaluator env num for training
+    --exp-name EXP_NAME   experiment name to save log and ckpt
+
+evaluate existing policies
+--------------------------------
+
+Note that CityFlow will run in fixed-time mode by default when not in rl mode.
+So the fix policy runs with an `auto_config.json`.
+
+.. code::
+
+    usage: cityflow_eval [-h] [-d DING_CFG] -e ENV_CFG [-s SEED]
+                         [-p {fix,dqn,ppo}] [-n ENV_NUM] [-c CKPT_PATH]
+
+    DI-smartcross training script
+
+    optional arguments:
+    -h, --help            show this help message and exit
+    -d DING_CFG, --ding-cfg DING_CFG
+                            DI-engine configuration path
+    -e ENV_CFG, --env-cfg ENV_CFG
+                            sumo environment configuration path
+    -s SEED, --seed SEED  random seed for sumo
+    -p {fix,dqn,ppo}, --policy-type {fix,dqn,ppo}
+                            RL policy type
+    -n ENV_NUM, --env-num ENV_NUM
+                            sumo env num for evaluation
+    -c CKPT_PATH, --ckpt-path CKPT_PATH
+                            model ckpt path

docs/source/rl_environments.rst

@@ -82,3 +82,54 @@ Roadnets
 
     envs/wj3_env
     envs/rl_arterial7_env
+
+
+CityFlow environments
+=============================
+
+configuration
+-----------------
+
+CityFlow simulator has its own config `json` file, with roadnet file, flow file and replay file defined in it.
+DI-smartcross adds some extra configs together with CityFlow's config file path in DI-engine's env config.
+
+.. code:: python
+
+    main_config = dict(
+        env=dict(
+            obs_type=['phase', 'lane_vehicle_num', 'lane_waiting_vehicle_num'],
+            max_episode_duration=1000,
+            green_duration=30,
+            yellow_duration=5,
+            red_duration=0,
+            ...
+        ),
+        ...
+    )
+
+Observation
+----------------
+
+We provide several types of observations of each cross.
+
+- phase: One-hot phase vector of current cross signal
+- lane_vehicle_num: vehicle nums of each incoming lane
+- lane_waiting_vehicle_num: waiting vehicle nums of each incoming lane
+
+Action
+-------------
+
+CityFlow environment supports changing cross signal to target phase. The action space is set to multi-discrete for each cross to reduce action num.
+
+Reward
+-------------
+
+CityFlow environment uses pressure of each cross as reward
+
+Roadnets
+-------------
+
+.. toctree::
+    :maxdepth: 2
+
+    envs/cf_grid_env

smartcross/envs/action/sumo_action.py

@@ -22,7 +22,7 @@ def _init(self, env: BaseEnv, cfg: Dict) -> None:
         Overview:
             init the sumo action environment with the given config file
         Arguments:
-            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo/sumo_env_default_config.yaml`
+            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo_wj3_default_config.yaml`
         """
         self._env = env
         self._cfg = cfg

smartcross/envs/action/sumo_action_runner.py

@@ -15,7 +15,7 @@ def _init(self, engine: BaseEnv, cfg: Dict) -> None:
         Overview:
             init the sumo observation helper with the given config file
         Arguments:
-            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo/sumo_env_default_config.yaml`
+            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo_wj3_default_config.yaml`
         """
         # set self._core and other state variable
         self._engine = engine

smartcross/envs/obs/sumo_obs_runner.py

@@ -20,7 +20,7 @@ def _init(self, engine: BaseEnv, cfg: dict) -> None:
         Overview:
             init the sumo observation helper with the given config file
         Arguments:
-            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo/sumo_env_default_config.yaml`
+            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo_wj3_default_config.yaml`
         """
         # set self._core and other state variable
         self._engine = engine

smartcross/envs/reward/sumo_reward.py

@@ -22,7 +22,7 @@ def _init(self, env: BaseEnv, cfg: Dict) -> None:
         Overview:
             init the sumo reward environment with the given config file
         Arguments:
-            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo/sumo_env_default_config.yaml`
+            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo_wj3_default_config.yaml`
         """
         self._env = env
         self._cfg = cfg

smartcross/envs/reward/sumo_reward_runner.py

@@ -21,7 +21,7 @@ def _init(self, engine: BaseEnv, cfg: dict) -> None:
         Overview:
             init the sumo reward environment with the given config file
         Arguments:
-            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo/sumo_env_default_config.yaml`
+            - cfg(:obj:`EasyDict`): config, you can refer to `envs/sumo_wj3_default_config.yaml`
         """
         self._engine = engine
         self._core = SumoReward(engine, cfg)