Using LATTE through LAMMPS

DFTB Based Quantum Molecular Dynamics Simulation using LAMMPS and LATTE: Step-by-step tutorial

This notes (LA-UR number ’LA-UR-17-27187’) are intended to give a step-by-step “howto” run a Density Functional Tight-Binding (DFTB) based Quantum Molecular Dynamics (QMD) calculation using LAMMPS and LATTE computational codes. Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) is an intensively used molecular dynamics code written in C++ Plimpton1995. Los Alamos Transferable Tight-binding for Energetics (LATTE) is a LANL Fortran code used for quantum chemistry based on the tight-binding method. See Cawkwell2012 and Niklasson2012. The LAMMPS documentation can be found in http://lammps.sandia.gov/doc/Manual.html. The LATTE documentation can be found in https://github.com/lanl/LATTE/blob/master/Manual/LATTE_manual.pdf. With the help of an interface we are now able to drive a molecular dynamics (MD) simulation with LAMMPS using the forces computed at the DFTB level of theory with LATTE. A Description of the latte-lammps interface can be found in http://lammps.sandia.gov/doc/fix_latte.html This interface has been developed by C. F. A. Negre, M. J. Cawkwell and S. J. Plimpton as part of the EXAALT (Molecular Dynamics at the Exascale: Spanning the Accuracy, Length and Time Scales for Critical Problems in Materials Science) ECP (Exascale Computing Project).

Requirements

In order to follow this tutorial, we will assume that the reader have a LINUX or MAC operative system with the following packages properly installed:

• The git program for cloning the codes.

• A C/C++ compiler (gcc and g++ for example)

• A Fortran compiler (gfortran for example)

• The LAPACK and BLAS libraries (GNU libblas and liblapack for example)

• The python interpreter (not essential).

• The pkgconfig and cmake programs (not essential).

On an x86_64 GNU/Linux Ubuntu 16.04 distribution the commands to be typed are the following:

      $ sudo apt-get update   
      $ sudo apt-get --yes --force-yes install gfortran gcc g++
      $ sudo apt-get --yes --force-yes install libblas-dev liblapack-dev   
      $ sudo apt-get --yes --force-yes install cmake pkg-config cmake-data 
      $ sudo apt-get --yes --force-yes install git python             

NOTE: Through the course of this tutorial we will assume that the follower will work and install the programs in the home directory ($HOME).

Download and installation

Download:

We will need to clone two essential repositories. One for LAMMPS:

      $ cd; git clone https://github.com/lammps/lammps.git

And another for LATTE:

      $ cd; git clone https://github.com/lanl/LATTE.git

NOTE: Make sure your public ssh key is added to your github profile in order to be able to clone this repositories. Alternative to the git command, these two codes can be Downloaded directly as a .zip file from the github webpage clicking on the “clone or download button” and choosing the “download zip” option. For the case of LAMMPS go to: https://github.com/lammps/lammps and for LATTE go to: https://github.com/lanl/LATTE.

Compiling LATTE:

We recommend to compile LATTE with PROGRESS and BML libraries. We explain how to get and install these libraries in a section below. The first code that must be compiled is the LATTE code because it has to be compiled as a library so that it can be linked with LAMMPS. In order to compile the LATTE code we proceed as following. We enter the LATTE directory (cd ~/LATTE) and modify the makefile.CHOICES file according to the operative system, libraries, compiler, etc. There are several examples of makefile.CHOICES files inside the makefiles directory (~/LATTE/makefiles) that could be used as a template to replace the makefile.CHOICES that is located in the main directory. IMPORTANT: Make sure that the MAKELIB variable is set ot ON (MAKELIB = ON) to be able to compile LATTE as a library.

If you are running on x86_64 GNU/Linux Ubuntu 16.04 with the packages that we suggested to install, then the actual makefile.CHOICES should work.

To build the code just type make inside the main directory as follows:

      $ cd; cd ~/LATTE
      $ make; make test

The latter should build the code and test it with some examples that are located in ~/LATTE/tests. A file called liblatte.a should appear after building the code. NOTE: Make sure to remove the liblatte.a any time the code has to be rebuilt.

Compiling LAMMPS:

To build the LAMMPS code and link it with LATTE, got to the LAMMPS folder and enter in the lib/latte directory (cd ~/lammps/lib/latte). Copy one of the Makefiles to Makefile.lammps as follows: cp Makefile.lammps.gfortran Makefile.lammps and edit the Makefile.lammps file according to your system settings, compilers and LATTE location. There are more instructions in the README file inside ~/lammps/lib/latte. After this is done go to the source folder (cd ~/lammps/src) and type the following series of commands:

      $ cd ~/lammps/src
      $ make yes-latte
      $ make yes-molecule 
      $ make serial

A test for the LATTE-LAMMPS insterface can be done within the LATTE foder. For this, check that the appropriate path to both LATTE and LAMMPS are set in /LATTE/tests/test_lmp.sh. Then while being inside the LATTE folder we can type: make test_lmp. A series of tests testing latte-lammps interface will follow.

Running an example calculation

We will assume you have a pdb/xyz file containing the coordinates of your system. For the purpose of this tutorial we will use a simple system consisting on eight water molecules in a box.

We will make a new directory to be able to run so that we can separate the running files from program files.

      $ mkdir ~/example_water
      $ cd ~/example_water
      $ cp ~/LATTE/examples/lammps/water/wat.pdb .

Now we need to transform the coordinates to the LAMMPS data file which is formated as follows:

      LAMMPS Description
      
      24 atoms
      
      2 atom types
      
      0.0000000000000000        6.2670000000000003      xlo xhi
      0.0000000000000000        6.2670000000000003      ylo yhi
      0.0000000000000000        6.2670000000000003      zlo zhi
      
      Masses
      
      1   15.994915008544922     
      2   1.0078250169754028     
      
      Atoms
      
      1    1    1   0.0   3.08800   3.70000   3.12400
      2    1    2   0.0   4.05800   3.70000   3.12400
      3    1    2   0.0   2.76400   3.13200   3.84100
      4    1    1   0.0   2.47000   0.39000   1.36000    

The format of the file is self explanatory but one can consult the LAMMPS data format for more information: http://lammps.sandia.gov/doc/2001/data_format.html. For this example we are providing an already formated file which is located in:
~/LATTE/examples/lammps/water/wat.lmp

If a particular mass needs to be changed, this has to be done, not only on the .lmp file, but also inside the “electrons.dat” file . This is a file located in TBparams/electrons.dat.

There is a tool that can transform pdb to lmp formats back and forth which is part of the PROGRESS library located in: https://github.com/lanl/qmd-progress If installed, a tool can be used to transform coordinates as follows:

      $ ~/qmd-progress/build/changecoords mycoords.pdb mycoords.lmp

There is another tool in openbabel library that can convert from any format to the lammps format but it cannot convert back from lammps to any other format. See source code.

A general LAMMPS input file will have to be created. An example input file can be found in ~/LATTE/examples/lammps/water/in.md. This file has to be copied to the running folder to be used in this tutorial.

      $ cp ~/LATTE/examples/lammps/water/in.md ~/example_water/

For a precise definition of the keywords here used please refer to:
http://lammps.sandia.gov/doc/Manual.html. There are some important points to notice:

• units must be set to metal

• read_data must be followed by the name of our data file. In this case this is: wat.lmp.

• atom_modify must be followed by sort 0 0.0 in order to avoid an error from resorting the coordinates.

• fix 2 all latte NULL will call the LATTE program to compute the forces.

A file that controls all the LATTE keywords is provided and needs to be copied to the running folder and renamed latte.in. A detailed Description of all these keywords can be found in:
https://github.com/lanl/LATTE/blob/master/Manual/LATTE_manual.pdf

      $ cp ~/LATTE/examples/lammps/water/latte.fullscf.in ~/example_water/latte.in

Finally, we need to copy the Tight-binding parameters to the running folder. These TB parameters are located in ~/LATTE/TBparam

      $ cp -r ~/LATTE/TBparam  ~/example_water

In order to run the calculation we just need to type the following lines:

      $ cd ~/example_water
      $ ~/lammps/src/lmp_serial < in.wat

If we want to control the number of threads that LATTE is using we can just set export omp_num_threads=N where N is the number of threads that we require.

If we want to print the log.lammps with more frequency we can add the following lines to the in.wat file:

      thermo          1
      thermo_modify flush yes

Restarting a simulation

In order to restart a lammps-latte simulation, calculations have to be run with a special restart key in both program. This key tells us how often the program will be writing into files. To do this, the following line must be added in the lammps input file:

      restart 10 restart.*.md

This means that the program will write each 10 steps in a file called restart.*.md. In the latte.in file we need to add the following flags and values:

      MDCONTROL{
        RSFREQ= 10
        RSLEVEL= 2
      }

This means that we will be writing in file every 10 steps, same as lammps. Note that the “Restarting frequency” must coincide with lammps. Also make sure that RESTARTLIB is set to 0 under the CONTROL list. The RSLEVEL flag tells us how much we want to write to file. If set to 0, only “one-index” arrays will be written. If set to 1 both one and two index arrays will be written. If set to 2 or larger, one, two and three index array will be written.

After the run, there are many files produced both by latte and lamps. Let’s say now we would like to restart at step 100. For doing this, the steps are the following. Copy the restart.latte.100.dat file into restart.latte.dat. In the lammps input file do the following changes:

      #read_data data.0scf.lmp
      read_restart restart.100.md
      
      #velocity all create 0.0 87287 loop geom

In the latte input set the flag RESTARTLIB to 1, this is RESTARTLIB= 1. Now the program can be rerun as usual.

Compiling LATTE with PROGRESS and BML libraries

The latte program can benefit a lot when run using both PROGRESS and BML libraries. In this section we will explain how to install both of these libraries.

Installing the libraries

Clone the BML library (in your home directory) by doing[^1]:

      $ cd 
      $ git clone [email protected]:lanl/bml.git

Take a look at the example_build.sh file which has a set of instructions for configuring. Configure the installation by typing:

      $ sh example_build.sh

The build.sh script is called and the installation is configured by creating the build directory. Go into the build directory and type:

      $ make
      $ make install

To ensure bml is installed correctly type $ make tests or $ make test ARGS="-V" to see details of the output. Series of tests results should follow.

After BML is installed, return to you home folder and “clone” the PROGRESS repository. To do this type:

      $ cd 
      $ git clone [email protected]:lanl/qmd-progress.git

Once the folder is cloned, cd into that folder and use the example_build.sh file to configure the installation by following the same steps as for the bml library.

      $ sh example_build.sh
      $ cd build
      $ make; make install 

You can test the installation by typing $ make tests in the same way as it is done for BML.

Compiling with LATTE

Open the makefile.CHOICES file in the LATTE folder and uncomment the following lines:

      PROGRESS = ON
      PROGRESS_PATH= $(HOME)/qmd-progress/install/lib
      BML_PATH= $(HOME)/bml/install/lib

Make sure that the path where the PROGRESS and BML libraries are installed are the correct ones. NOTE: Sometimes, depending on the architecture the libraries are installed in /lib64 instead of /lib. After the aforementioned changes are done to the makefile.CHOICES file, proceed compiling with the “make” command as before.

Compiling with LAMMPS-LATTE

Once latte is properly compiled with PROGRESS and BML and the tests are passing. Proceed to go inside the /lammps/lib/latte folder and add the following lines to the Makefile.lammps file:

      latte_SYSINC  =  -I${bml_PATH}/install/include -I${progress_PATH}/install/include         
      latte_SYSLIB  += -L${progress_PATH}/install/lib -lprogress -L${bml_PATH}/install/lib -lbml_fortran -lbml 
      latte_SYSLIB  += -L${metis_PATH}/install -lmetis

Once this lines are added, recompile lammps as before.

Note on BLAS and LAPACK:

LATTE is heavily relying on these libraries and so are the BML and PROGRESS libraries. Having these libraries properly linked with the code is very important for the performance of the simulations. Right now we are supporting both the GNU and Intel version of the libraries. The Intel version of the library can also be compiled with gfortran without problems since they are provided by the vendors. Sometimes even if we are using GNU compilers we can still benefit from the speed of the Intel LAPACK/BLAS implementation. So far, the best combination we have seen is to use the GNU compilers and the MKL(Intel) version of LAPACK/BLAS libraries.

Note on Nudged elastic band (NEB) calculation

To run NEB with lammps-latte we need the to install the following packages:

      $ make yes-molecule
      $ make yes-kspace
      $ make yes-replica
      $ make yes-rigid
      $ make mpi 

The program meeds to be compiled with MPI and run with mpirun as follows:

      mpirun -np 14 --map-by node /home/cnegre/lammps/src/lmp_mpi -partition 14x1 -in in.neb | tee out

In this case we are optimizing 14 replicas.

Contacts:

• For problems with LAMMPS please contact Steve Plimpton, email: [email protected]

• For problems with LAMMPS-LATTE interface please contact Christian Negre, email: [email protected]

• For problems with LATTE please contact Marc Cawkwell, email: [email protected]

[^1]: In order to have access to the repository you should have a github account and make sure to add your public ssh key is added in the configuration windows of github account.