docs: rewrite README; deprecate manually written TOC (#3179)

Deprecate per discussion. --------- Signed-off-by: Jinzhe Zeng <[email protected]>
deepmodeling · Jan 26, 2024 · 5c545f7 · 5c545f7
1 parent 3618702
commit 5c545f7
Show file tree

Hide file tree

Showing 14 changed files with 46 additions and 1,812 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -38,7 +38,7 @@ Currently, we maintain two main branch:
 - devel :  branch for developers
 
 ### Developer guide
-See [here](doc/development/index.md) for coding conventions, API and other needs-to-know of the code.
+See [documentation](https://deepmd.readthedocs.io/) for coding conventions, API and other needs-to-know of the code.
 
 ## How to contribute
 Please perform the following steps to create your Pull Request to this repository. If don't like to use commands, you can also use [GitHub Desktop](https://desktop.github.com/), which is easier to get started. Go to [git documentation](https://git-scm.com/doc) if you want to really master git.

diff --git a/README.md b/README.md
@@ -2,48 +2,28 @@
 
 --------------------------------------------------------------------------------
 
-<span style="font-size:larger;">DeePMD-kit Manual</span>
-========
+# DeePMD-kit
+
 [![GitHub release](https://img.shields.io/github/release/deepmodeling/deepmd-kit.svg?maxAge=86400)](https://github.com/deepmodeling/deepmd-kit/releases)
 [![offline packages](https://img.shields.io/github/downloads/deepmodeling/deepmd-kit/total?label=offline%20packages)](https://github.com/deepmodeling/deepmd-kit/releases)
 [![conda-forge](https://img.shields.io/conda/dn/conda-forge/deepmd-kit?color=red&label=conda-forge&logo=conda-forge)](https://anaconda.org/conda-forge/deepmd-kit)
 [![pip install](https://img.shields.io/pypi/dm/deepmd-kit?label=pip%20install)](https://pypi.org/project/deepmd-kit)
 [![docker pull](https://img.shields.io/docker/pulls/deepmodeling/deepmd-kit)](https://hub.docker.com/r/deepmodeling/deepmd-kit)
 [![Documentation Status](https://readthedocs.org/projects/deepmd/badge/)](https://deepmd.readthedocs.io/)
 
-# Table of contents
-- [About DeePMD-kit](#about-deepmd-kit)
- 	- [Highlights in v2.0](#highlights-in-deepmd-kit-v2.0)
- 	- [Highlighted features](#highlighted-features)
- 	- [License and credits](#license-and-credits)
- 	- [Deep Potential in a nutshell](#deep-potential-in-a-nutshell)
-- [Download and install](#download-and-install)
-- [Use DeePMD-kit](#use-deepmd-kit)
-- [Code structure](#code-structure)
-- [Troubleshooting](#troubleshooting)
-
-# About DeePMD-kit
+## About DeePMD-kit
 DeePMD-kit is a package written in Python/C++, designed to minimize the effort required to build deep learning-based model of interatomic potential energy and force field and to perform molecular dynamics (MD). This brings new hopes to addressing the accuracy-versus-efficiency dilemma in molecular simulations. Applications of DeePMD-kit span from finite molecules to extended systems and from metallic systems to chemically bonded systems.
 
 For more information, check the [documentation](https://deepmd.readthedocs.io/).
 
-# Highlights in DeePMD-kit v2.0
-* [Model compression](doc/freeze/compress.md). Accelerate the efficiency of model inference 4-15 times.
-* [New descriptors](doc/model/overall.md). Including [`se_e2_r`](doc/model/train-se-e2-r.md) and [`se_e3`](doc/model/train-se-e3.md).
-* [Hybridization of descriptors](doc/model/train-hybrid.md). Hybrid descriptor constructed from the concatenation of several descriptors.
-* [Atom type embedding](doc/model/train-se-e2-a-tebd.md). Enable atom-type embedding to decline training complexity and refine performance.
-* Training and inference of the dipole (vector) and polarizability (matrix).
-* Split of training and validation dataset.
-* Optimized training on GPUs.
-
-## Highlighted features
-* **interfaced with TensorFlow**, one of the most popular deep learning frameworks, making the training process highly automatic and efficient, in addition, Tensorboard can be used to visualize training procedures.
-* **interfaced with high-performance classical MD and quantum (path-integral) MD packages**, i.e., LAMMPS and i-PI, respectively.
-* **implements the Deep Potential series models**, which have been successfully applied to finite and extended systems including organic molecules, metals, semiconductors, insulators, etc.
+### Highlighted features
+* **interfaced with multiple backends**, including TensorFlow and PyTorch, the most popular deep learning frameworks, making the training process highly automatic and efficient.
+* **interfaced with high-performance classical MD and quantum (path-integral) MD packages**, including LAMMPS, i-PI, AMBER, CP2K, GROMACS, OpenMM, and ABUCUS.
+* **implements the Deep Potential series models**, which have been successfully applied to finite and extended systems, including organic molecules, metals, semiconductors, insulators, etc.
 * **implements MPI and GPU supports**, making it highly efficient for high-performance parallel and distributed computing.
 * **highly modularized**, easy to adapt to different descriptors for deep learning-based potential energy models.
 
-## License and credits
+### License and credits
 The project DeePMD-kit is licensed under [GNU LGPLv3.0](./LICENSE).
 If you use this code in any future publications, please cite the following publications for general purpose:
 - Han Wang, Linfeng Zhang, Jiequn Han, and Weinan E. "DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamics." Computer Physics Communications 228 (2018): 178-184.
@@ -55,7 +35,9 @@ If you use this code in any future publications, please cite the following publi
 
 In addition, please follow [the bib file](CITATIONS.bib) to cite the methods you used.
 
-## Deep Potential in a nutshell
+### Highlights in major versions
+
+#### Initial version
 The goal of Deep Potential is to employ deep learning techniques and realize an inter-atomic potential energy model that is general, accurate, computationally efficient and scalable. The key component is to respect the extensive and symmetry-invariant properties of a potential energy model by assigning a local reference frame and a local environment to each atom. Each environment contains a finite number of atoms, whose local coordinates are arranged in a symmetry-preserving way. These local coordinates are then transformed, through a sub-network, to so-called *atomic energy*. Summing up all the atomic energies gives the potential energy of the system.
 
 The initial proof of concept is in the [Deep Potential][1] paper, which employed an approach that was devised to train the neural network model with the potential energy only. With typical *ab initio* molecular dynamics (AIMD) datasets this is insufficient to reproduce the trajectories. The Deep Potential Molecular Dynamics ([DeePMD][2]) model overcomes this limitation. In addition, the learning process in DeePMD improves significantly over the Deep Potential method thanks to the introduction of a flexible family of loss functions. The NN potential constructed in this way reproduces accurately the AIMD trajectories, both classical and quantum (path integral), in extended and finite systems, at a cost that scales linearly with system size and is always several orders of magnitude lower than that of equivalent AIMD simulations.
@@ -64,110 +46,48 @@ Although highly efficient, the original Deep Potential model satisfies the exten
 
 In addition to building up potential energy models, DeePMD-kit can also be used to build up coarse-grained models. In these models, the quantity that we want to parameterize is the free energy, or the coarse-grained potential, of the coarse-grained particles. See the [DeePCG paper][4] for more details.
 
-See [our latest paper](https://doi.org/10.48550/arXiv.2304.09409) for details of all features.
-
-# Download and install
-
-Please follow our [GitHub](https://github.com/deepmodeling/deepmd-kit) webpage to download the [latest released version](https://github.com/deepmodeling/deepmd-kit/tree/master) and [development version](https://github.com/deepmodeling/deepmd-kit/tree/devel).
-
-DeePMD-kit offers multiple installation methods. It is recommended to use easy methods like [offline packages](doc/install/easy-install.md#offline-packages), [conda](doc/install/easy-install.md#with-conda) and [docker](doc/install/easy-install.md#with-docker).
-
-One may manually install DeePMD-kit by following the instructions on [installing the Python interface](doc/install/install-from-source.md#install-the-python-interface) and [installing the C++ interface](doc/install/install-from-source.md#install-the-c-interface). The C++ interface is necessary when using DeePMD-kit with LAMMPS, i-PI or GROMACS.
-
-
-# Use DeePMD-kit
-
-A quick start on using DeePMD-kit can be found [here](doc/getting-started/quick_start.ipynb).
-
-A full [document](doc/train/train-input-auto.rst) on options in the training input script is available.
-
-# Advanced
-
-- [Installation](doc/install/index.md)
-    - [Easy install](doc/install/easy-install.md)
-    - [Install from source code](doc/install/install-from-source.md)
-    - [Install from pre-compiled C library](doc/install/install-from-c-library.md)
-    - [Install LAMMPS](doc/install/install-lammps.md)
-    - [Install i-PI](doc/install/install-ipi.md)
-    - [Install GROMACS](doc/install/install-gromacs.md)
-    - [Building conda packages](doc/install/build-conda.md)
-    - [Install Node.js interface](doc/install/install-nodejs.md)
-    - [Easy install the latest development version](doc/install/easy-install-dev.md)
-- [Data](doc/data/index.md)
-    - [System](doc/data/system.md)
-    - [Formats of a system](doc/data/data-conv.md)
-    - [Prepare data with dpdata](doc/data/dpdata.md)
-- [Model](doc/model/index.md)
-    - [Overall](doc/model/overall.md)
-    - [Descriptor `"se_e2_a"`](doc/model/train-se-e2-a.md)
-    - [Descriptor `"se_e2_r"`](doc/model/train-se-e2-r.md)
-    - [Descriptor `"se_e3"`](doc/model/train-se-e3.md)
-    - [Descriptor `"se_atten"`](doc/model/train-se-atten.md)
-    - [Descriptor `"se_atten_v2"`](doc/model/train-se-atten.md#descriptor-se_atten_v2)
-    - [Descriptor `"hybrid"`](doc/model/train-hybrid.md)
-    - [Descriptor `sel`](doc/model/sel.md)
-    - [Fit energy](doc/model/train-energy.md)
-    - [Fit spin energy](doc/model/train-energy-spin.md)
-    - [Fit `tensor` like `Dipole` and `Polarizability`](doc/model/train-fitting-tensor.md)
-    - [Fit electronic density of states (DOS)](doc/model/train-fitting-dos.md)
-    - [Train a Deep Potential model using `type embedding` approach](doc/model/train-se-e2-a-tebd.md)
-    - [Deep potential long-range](doc/model/dplr.md)
-    - [Deep Potential - Range Correction (DPRc)](doc/model/dprc.md)
-    - [Linear model](doc/model/linear.md)
-    - [Interpolation or combination with a pairwise potential](doc/model/pairtab.md)
-- [Training](doc/train/index.md)
-    - [Training a model](doc/train/training.md)
-    - [Advanced options](doc/train/training-advanced.md)
-    - [Parallel training](doc/train/parallel-training.md)
-    - [Multi-task training](doc/train/multi-task-training.md)
-    - [TensorBoard Usage](doc/train/tensorboard.md)
-    - [Known limitations of using GPUs](doc/train/gpu-limitations.md)
-    - [Training Parameters](doc/train-input-auto.rst)
-- [Freeze and Compress](doc/freeze/index.rst)
-    - [Freeze a model](doc/freeze/freeze.md)
-    - [Compress a model](doc/freeze/compress.md)
-- [Test](doc/test/index.rst)
-    - [Test a model](doc/test/test.md)
-    - [Calculate Model Deviation](doc/test/model-deviation.md)
-- [Inference](doc/inference/index.rst)
-    - [Python interface](doc/inference/python.md)
-    - [C++ interface](doc/inference/cxx.md)
-    - [Node.js interface](doc/inference/nodejs.md)
-- [Integrate with third-party packages](doc/third-party/index.rst)
-    - [Use deep potential with dpdata](doc/third-party/dpdata.md)
-    - [Use deep potential with ASE](doc/third-party/ase.md)
-    - [Run MD with LAMMPS](doc/third-party/lammps-command.md)
-    - [Run path-integral MD with i-PI](doc/third-party/ipi.md)
-    - [Run MD with GROMACS](doc/third-party/gromacs.md)
-    - [Interfaces out of DeePMD-kit](doc/third-party/out-of-deepmd-kit.md)
-- [Use NVNMD](doc/nvnmd/index.md)
-
-# Code structure
+#### v1
+
+* Code refactor to make it highly modularized.
+* GPU support for descriptors.
+
+#### v2
+
+* Model compression. Accelerate the efficiency of model inference 4-15 times.
+* New descriptors. Including `se_e2_r`, `se_e3`, and `se_atten` (DPA-1).
+* Hybridization of descriptors. Hybrid descriptor constructed from the concatenation of several descriptors.
+* Atom type embedding. Enable atom-type embedding to decline training complexity and refine performance.
+* Training and inference of the dipole (vector) and polarizability (matrix).
+* Split of training and validation dataset.
+* Optimized training on GPUs, including CUDA and ROCm.
+* Non-von-Neumann.
+* C API to interface with the third-party packages.
+
+See [our latest paper](https://doi.org/10.1063/5.0155600) for details of all features until v2.2.3.
+
+#### v3
+
+* Multiple backends supported. Add a PyTorch backend.
+* The DPA-2 model.
+
+## Install and use DeePMD-kit
+
+Please read the [online documentation](https://deepmd.readthedocs.io/) for how to install and use DeePMD-kit.
+
+## Code structure
 
 The code is organized as follows:
 
-* `data/raw`: tools manipulating the raw data files.
 * `examples`: examples.
 * `deepmd`: DeePMD-kit python modules.
+* `source/lib`: source code of the core library.
+* `source/op`: Operator (OP) implementation.
 * `source/api_cc`: source code of DeePMD-kit C++ API.
+* `source/api_c`: source code of the C API.
+* `source/nodejs`: source code of the Node.js API.
 * `source/ipi`: source code of i-PI client.
-* `source/lib`: source code of DeePMD-kit library.
 * `source/lmp`: source code of Lammps module.
 * `source/gmx`: source code of Gromacs plugin.
-* `source/op`: TensorFlow op implementation. working with the library.
-
-
-# Troubleshooting
-
-- [Model compatibility](doc/troubleshooting/model_compatability.md)
-- [Installation](doc/troubleshooting/installation.md)
-- [The temperature undulates violently during the early stages of MD](doc/troubleshooting/md_energy_undulation.md)
-- [MD: cannot run LAMMPS after installing a new version of DeePMD-kit](doc/troubleshooting/md_version_compatibility.md)
-- [Do we need to set rcut < half boxsize?](doc/troubleshooting/howtoset_rcut.md)
-- [How to set sel?](doc/troubleshooting/howtoset_sel.md)
-- [How to control the parallelism of a job?](doc/troubleshooting/howtoset_num_nodes.md)
-- [How to tune Fitting/embedding-net size?](doc/troubleshooting/howtoset_netsize.md)
-- [Why does a model have low precision?](doc/troubleshooting/precision.md)
 
 # Contributing
 

diff --git a/doc/conf.py b/doc/conf.py
@@ -28,96 +28,6 @@
 sys.path.append(os.path.dirname(__file__))
 import sphinx_contrib_exhale_multiproject  # noqa: F401
 
-
-def mkindex(dirname):
-    dirname = dirname + "/"
-    oldfindex = open(dirname + "index.md")
-    oldlist = oldfindex.readlines()
-    oldfindex.close()
-
-    oldnames = []
-    for entry in oldlist:
-        _name = entry[entry.find("(") + 1 : entry.find(")")]
-        oldnames.append(_name)
-
-    newfindex = open(dirname + "index.md", "a")
-    for root, dirs, files in os.walk(dirname, topdown=False):
-        newnames = [
-            name for name in files if "index.md" not in name and name not in oldnames
-        ]
-        for name in newnames:
-            f = open(dirname + name)
-            _lines = f.readlines()
-            for _headline in _lines:
-                _headline = _headline.strip("#")
-                headline = _headline.strip()
-                if len(headline) == 0 or headline[0] == "." or headline[0] == "=":
-                    continue
-                else:
-                    break
-            longname = "- [" + headline + "]" + "(" + name + ")\n"
-            newfindex.write(longname)
-
-    newfindex.close()
-
-
-def classify_index_TS():
-    dirname = "troubleshooting/"
-    oldfindex = open(dirname + "index.md")
-    oldlist = oldfindex.readlines()
-    oldfindex.close()
-
-    oldnames = []
-    sub_titles = []
-    heads = []
-    while len(oldlist) > 0:
-        entry = oldlist.pop(0)
-        if entry.find("(") >= 0:
-            _name = entry[entry.find("(") + 1 : entry.find(")")]
-            oldnames.append(_name)
-            continue
-        if entry.find("##") >= 0:
-            _name = entry[entry.find("##") + 3 : -1]
-            sub_titles.append(_name)
-            continue
-        entry.strip()
-        if entry != "\n":
-            heads.append(entry)
-
-    newfindex = open(dirname + "index.md", "w")
-    for entry in heads:
-        newfindex.write(entry)
-    newfindex.write("\n")
-    sub_lists = [[], []]
-    for root, dirs, files in os.walk(dirname, topdown=False):
-        newnames = [name for name in files if "index.md" not in name]
-        for name in newnames:
-            f = open(dirname + name)
-            _lines = f.readlines()
-            f.close()
-            for _headline in _lines:
-                _headline = _headline.strip("#")
-                headline = _headline.strip()
-                if len(headline) == 0 or headline[0] == "." or headline[0] == "=":
-                    continue
-                else:
-                    break
-            longname = "- [" + headline + "]" + "(" + name + ")\n"
-            if "howtoset_" in name:
-                sub_lists[1].append(longname)
-            else:
-                sub_lists[0].append(longname)
-
-    newfindex.write("## Trouble shooting\n")
-    for entry in sub_lists[0]:
-        newfindex.write(entry)
-    newfindex.write("\n")
-    newfindex.write("## Parameters setting\n")
-    for entry in sub_lists[1]:
-        newfindex.write(entry)
-    newfindex.close()
-
-
 # -- Project information -----------------------------------------------------
 
 project = "DeePMD-kit"
@@ -169,10 +79,6 @@ def setup(app):
 #     'sphinx.ext.autosummary'
 # ]
 
-# mkindex("troubleshooting")
-# mkindex("development")
-# classify_index_TS()
-
 extensions = [
     "deepmodeling_sphinx",
     "dargs.sphinx",

diff --git a/doc/data/index.md b/doc/data/index.md
diff --git a/doc/freeze/index.md b/doc/freeze/index.md
diff --git a/doc/getting-started/quick_start.ipynb b/doc/getting-started/quick_start.ipynb
@@ -239,7 +239,7 @@
    "id": "a999f41b-e343-4dc2-8499-84fee6e52221",
    "metadata": {},
    "source": [
-    "The DeePMD-kit adopts a compressed data format. All training data should first be converted into this format and can then be used by DeePMD-kit. The data format is explained in detail in the DeePMD-kit manual that can be found in [the DeePMD-kit Data Introduction](../data/index.md)."
+    "The DeePMD-kit adopts a compressed data format. All training data should first be converted into this format and can then be used by DeePMD-kit. The data format is explained in detail in the DeePMD-kit manual that can be found in [the DeePMD-kit Data Introduction](../data/system.md)."
    ]
   },
   {

diff --git a/doc/inference/index.md b/doc/inference/index.md
diff --git a/doc/install/index.md b/doc/install/index.md