Enable use of CVs defined by PyTorch neural network models (#570)

Squashed commit of PR #570

* In the process of implementing torchann colvar

* Implementing torchann

* Implemented calc_gradients and apply_forces

* Start working

* Handling periodicity; test torchANN with phi angle

* Add output_index to specify the component of output layer that is used

* up

* Update gromacs-2021.6 patch

* modify patch files for namd to support LibTorch

* Add test for torchann

* minor changes

* Modified test for torchann component

* Add figures for AutoEncoder CV

* Convert unit of atom coordinates to angstrom before feeding to torch model

* Bug fixed in gradient of torchann

* Add Torch in lammps' cmake

* Added test files for torchann in NAMD

* Minor changes in torchann component

* Updated patches for Gromacs 2020.x and 2022.x

* Updated torchann in header file; Removed test for torchann component

* Add tests for Gromacs and NAMD

* Add tests for Gromacs and NAMD

* Updated torchann-namd test and references

* Reimplement torchann as a derived class of LinearCombination

* declare torchann cvc as scalar

* allocate memory of cv_force before use

* c++17 instead of c++14

* removed file to correct place

* removed file to correct place

* Minor correction in apply_force of torchann

* Add tests for gromacs and NAMD

* Modified the definition of cv_force such that it contains the information on the size

* Add option to set data type in torchann

* Add option to set data type in torchann

* Enable GPU support; Further tests are needed.

* updated patch for gromacs-2021.x

* updated patch for gromacs-2022

* Updated patch for gromacs-2020.x

* Update namd config to support GPU

* Do not require libtorch in lammps by default

* Minor changes (spaces/indent) in code; Removed tests for namd and gromacs backend

* update namd config patch to make namd3 work

* updated patch for gromacs 2023 to support torchann

* merge torchann squash

* Do not use GPU

* merged with master and changed torchann class based on HanatoK's suggestions

* Added torchann tests for gromacs and namd (only when libtorch is linked)

* Add torch option to library CMake file

* Use the same code from distanceZ to handle nearest-image convection

* Squash-merge of master branch

* Integrate libTorch CMake recipe into GROMACS main

* Fix NAMD build dependencies

* Restore componentDisabled and make it more generally usable

* Integrate cvc base class improvements with torchANN

* Fix CMakeFile for Gromacs 2024

Also allow torchANN components to be declared as periodic to fix the regtest

* Set PRNG seed in Gromacs regtests

* Update patches for GMX CMake files

Had to make separate patch for the mdmodules version

* Add patch for gmxManageColvars.cmake

* Use Clang and ccache more often when available

* Find PyTorch through CMake and switch to -DCOLVARS_TORCH as macro

* Update CentOS9-devel container recipe

* Use libTorch from container when available

* Remove pre-C++11 stuff

* Add torchann test for standalone library

* Disable torchann test when not building with libTorch

* Enable Torch interface only when the requested C++ standard supports it

* Raise the CMAKE_CXX_STANDARD to 17 if libtorch is used and
CMAKE_CXX_STANDARD is less than 17

* Squash-merge of master branch

* Remove patch files for old and unsupported GROMACS versions

* Merge branch 'master'

* - Isolated torch-related code
- Removed componentDisabled

* minor fix

* Add torch header file to dependence file of backends

* fixed bug when torch is not available

* Merge branch 'master'

* Enable Torch in GROMACS builds when available from CI container

* Do not duplicate libTorch C++ flags

* Document apptainer push

* Let GROMACS use the newer CMake when available

* Squash-merge of master branch

* Squash-merge of master branch

* Set damping=0 in gromacs regtest for torchann class.
Updated reference files accordingly.

* Updated references files generated by Gromacs with double precision

* Minor corrections in reference files.

* Squashed merge of master branch

---------

Co-authored-by: Zhang <[email protected]>
Co-authored-by: Zhang <[email protected]>
Co-authored-by: Jérôme Hénin <[email protected]>
Co-authored-by: Giacomo Fiorin <[email protected]>
Co-authored-by: HanatoK <[email protected]>
Co-authored-by: Zhang <[email protected]>

.github/workflows/test-library.yml

@@ -51,7 +51,11 @@ jobs:
               exit 1
           fi
 
-      - name: Build and test library (GCC)
+      - name: Build and test library (Clang)
+        env:
+          CMAKE_BUILD_DIR: build
+          CXX: clang++
+          CC: clang
         run: cmake -P devel-tools/build_test_library.cmake
 
       - name: Check documentation of command-line scripting interface
@@ -64,11 +68,11 @@ jobs:
               exit 1
           fi
 
-      - name: Build and test library (Clang)
+      - name: Build and test library (GCC)
         env:
-          CMAKE_BUILD_DIR: build-clang
-          CXX: clang++
-          CC: clang
+          CMAKE_BUILD_DIR: build-gcc
+          CXX: g++
+          CC: gcc
         run: cmake -P devel-tools/build_test_library.cmake
 
       - name: Build library with debug code enabled (GCC)

cmake/CMakeLists.txt

@@ -14,10 +14,6 @@ set(CMAKE_CXX_EXTENSIONS OFF)
 if(NOT DEFINED CMAKE_CXX_STANDARD)
   set(CMAKE_CXX_STANDARD 11)
   set(CMAKE_CXX_STANDARD_REQUIRED ON)
-else()
-  if(${CMAKE_CXX_STANDARD} STREQUAL "98")
-    message(FATAL_ERROR "CMAKE_CXX_STANDARD must be 11 or later")
-  endif()
 endif()
 
 option(WARNINGS_ARE_ERRORS "Treats warnings as errors" OFF)
@@ -88,11 +84,10 @@ option(COLVARS_OPENMP "Build Colvars with (still limited) OpenMP support" ${COLV
 if(COLVARS_OPENMP)
   find_package(OpenMP)
   if(OpenMP_CXX_FOUND)
-    target_link_libraries(colvars OpenMP::OpenMP_CXX)
+    target_link_libraries(colvars PRIVATE OpenMP::OpenMP_CXX)
   endif()
 endif()
 
-
 include(buildColvarsLepton)
 
 
@@ -137,13 +132,35 @@ if(COLVARS_TCL)
   else()
     target_compile_options(colvars PRIVATE -DCOLVARS_TCL)
   endif()
-  target_link_libraries(colvars ${TCL_LIBRARY})
+  target_link_libraries(colvars PUBLIC ${TCL_LIBRARY})
+endif()
+
+option(COLVARS_TORCH "Link against the Torch library" OFF)
+
+if(COLVARS_TORCH)
+  if (CMAKE_CXX_STANDARD LESS 17)
+    message(STATUS "Increase the CMAKE_CXX_STANDARD to 17")
+    set_property(TARGET colvars PROPERTY CXX_STANDARD 17)
+  else()
+    message(STATUS "Keep the CMAKE_CXX_STANDARD as ${CMAKE_CXX_STANDARD}")
+  endif()
+  if(DEFINED LIBTORCH_PREFIX)
+    find_package(Torch REQUIRED PATHS file(JOIN ${LIBTORCH_PREFIX} "share/cmake/Torch"))
+  else()
+    find_package(Torch REQUIRED)
+  endif()
+  if(Torch_FOUND)
+    target_compile_definitions(colvars PRIVATE -DCOLVARS_TORCH)
+    target_compile_options(colvars PRIVATE ${TORCH_CXX_FLAGS})
+    target_include_directories(colvars PRIVATE ${TORCH_INCLUDE_DIRS})
+    target_link_libraries(colvars PUBLIC ${TORCH_LIBRARIES})
+  endif()
 endif()
 
 option(BUILD_TOOLS "Build standalone tools" ON)
 
 option(BUILD_TESTS "Build tests" ON)
-
+ 
 if(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
   # TODO fix linkage errors with Visual Studio...
   set(BUILD_TOOLS OFF)

cmake/buildColvarsLepton.cmake

@@ -18,7 +18,7 @@ if(COLVARS_LEPTON)
     target_compile_options(colvars PRIVATE -DLEPTON)
   endif()
   target_include_directories(colvars PRIVATE ${LEPTON_DIR}/include)
-  target_link_libraries(colvars lepton)
+  target_link_libraries(colvars PRIVATE lepton)
 
   # Silence warnings for Lepton
   target_compile_options(lepton PRIVATE $<$<CXX_COMPILER_ID:Clang>:-Wno-tautological-undefined-compare -Wno-unknown-warning-option>)

cmake/buildColvarsTests.cmake

@@ -15,6 +15,11 @@ if(BUILD_TESTS)
 
       # Copy the Colvars configuration files
       file(GLOB TEST_CONFIG_FILES ${COLVARS_SOURCE_DIR}/tests/input_files/*/test.in)
+      if(NOT COLVARS_TORCH)
+        # TODO create a way to detect test dependencies at some point
+        list(REMOVE_ITEM TEST_CONFIG_FILES ${COLVARS_SOURCE_DIR}/tests/input_files/torchann-dihedral_harmonic-fixed/test.in)
+      endif()
+
       foreach(TEST_CONFIG_FILE ${TEST_CONFIG_FILES})
         get_filename_component(TEST_NAME ${TEST_CONFIG_FILE} DIRECTORY)
         get_filename_component(TEST_NAME ${TEST_NAME} NAME)

devel-tools/build_test_library.cmake

@@ -7,13 +7,19 @@ else()
   get_filename_component(COLVARS_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR} DIRECTORY)
 endif()
 
-set(COLVARS_LEPTON ON)
+if(EXISTS "/opt/libtorch")
+  if(NOT DEFINED CMAKE_CXX_STANDARD)
+    set(CMAKE_CXX_STANDARD 17)
+  endif()
+  if(${CMAKE_CXX_STANDARD} GREATER_EQUAL 17)
+    message("Enabling Torch interface, using library at /opt/libtorch")
+    set(DEFINE_TORCH "-DCOLVARS_TORCH=ON")
+    set(DEFINE_TORCH_PREFIX "-DLIBTORCH_PREFIX=/opt/libtorch")
+  endif()
+endif()
+
 if(NOT DEFINED CMAKE_CXX_STANDARD)
   set(CMAKE_CXX_STANDARD 11)
-else()
-  if(${CMAKE_CXX_STANDARD} GREATER 70)
-    set(COLVARS_LEPTON OFF)
-  endif()
 endif()
 
 if(NOT DEFINED BUILD_SHARED_LIBS)
@@ -54,6 +60,8 @@ if(COLVARS_TCL AND DEFINED TCL_DIR)
 endif()
 
 
+set(COLVARS_LEPTON ON)
+
 if(COLVARS_LEPTON)
   if(NOT DEFINED LEPTON_DIR)
     set(LEPTON_DIR "${COLVARS_SOURCE_DIR}/openmm-source/libraries/lepton")
@@ -82,8 +90,27 @@ else()
 endif()
 
 if(DEFINED ENV{CCACHE})
-  set(DEFINE_CC_CCACHE "-DCMAKE_C_COMPILER_LAUNCHER=$ENV{CCACHE}")
-  set(DEFINE_CXX_CCACHE "-DCMAKE_CXX_COMPILER_LAUNCHER=$ENV{CCACHE}")
+  set(CCACHE $ENV{CCACHE})
+else()
+  find_program(CCACHE "ccache")
+  if(CCACHE)
+    set(CCACHE "ccache")
+  endif()
+endif()
+
+if(NOT ${CCACHE} STREQUAL "CCACHE-NOTFOUND")
+  message(STATUS "Using ${CCACHE} as the compiler launcher")
+  set(DEFINE_CC_CCACHE "-DCMAKE_C_COMPILER_LAUNCHER=${CCACHE}")
+  set(DEFINE_CXX_CCACHE "-DCMAKE_CXX_COMPILER_LAUNCHER=${CCACHE}")
+endif()
+
+if(NOT DEFINED ENV{CXX})
+  # Use Clang if available to catch more errors
+  find_program(CLANG "clang++")
+  if(CLANG)
+    message(STATUS "Clang is available, using it as the default compiler")
+    set(DEFINE_CXX_COMPILER "-DCMAKE_CXX_COMPILER=clang++")
+  endif()
 endif()
 
 execute_process(
@@ -96,13 +123,18 @@ execute_process(
   -D WARNINGS_ARE_ERRORS=ON
   -D CMAKE_VERBOSE_MAKEFILE=ON
   -D CMAKE_CXX_STANDARD=${CMAKE_CXX_STANDARD}
+  ${DEFINE_CXX_COMPILER}
   ${DEFINE_CC_CCACHE}
   ${DEFINE_CXX_CCACHE}
+  ${DEFINE_PYTHON}
   -D COLVARS_TCL=${COLVARS_TCL}
   ${DEFINE_TCL_DIR}
   ${DEFINE_TCL_LIBRARY}
+  ${DEFINE_TORCH}
+  ${DEFINE_TORCH_PREFIX}
   -D COLVARS_LEPTON=${COLVARS_LEPTON}
   -D LEPTON_DIR=${LEPTON_DIR}
+  -D CMAKE_PREFIX_PATH="/opt/libtorch/share/cmake"
   RESULT_VARIABLE result
   )
 

devel-tools/compile-gromacs.sh

@@ -69,6 +69,10 @@ compile_gromacs_target() {
         fi
     fi
 
+    if [ -d "/opt/libtorch" ] ; then
+        GMX_BUILD_OPTS+=(-DGMX_NNPOT=TORCH -DTorch_DIR=/opt/libtorch/share/cmake/Torch)
+    fi
+
     # When on GitHub Actions, build the tests as well
     if [ -n "${GITHUB_ACTION}" ] ; then
         export GROMACS_BUILD_TESTS=1

doc/colvars-code-refs.bib

@@ -389,5 +389,6 @@ @article{Invernizzi2022
 % coordNum pairlist
 % Custom functions (Lepton)
 % Scripted functions (Tcl)
+% torchANN colvar component 
 % ABMD bias
 % --- END ---