New Gromacs User, Many Errors

[Ntr9999]

Hi,

I am a new Gromacs user and came across your package which has the potential to be very useful to me. Unfortunately, I have had problems using MDPOW. I'm pretty sure Gromacs is installed properly, is my location okay (/usr/local/bin/GMXRC)?
MDAnalysis seems to be causing problems too, when I try to install it, both with MDPOW install and individually, something doesn't work, I don't know if this is because there is a new release soon or something. I managed to fudge the packages together using conda and pip for MDA and MDAT respectively, because it is the only thing that worked. The MDAnalysisTests were failing...Most of the time I have 140 errors in 5 seconds and it stopped... then I think I changed my matplotlib version to <3.3 and it fixed it partially (TypeError: use() got an unexpected keyword argument 'warn')? I was able to run the tests through to completion.
But then when I try to use the MDPOW package, it tells me I don't gave MDA installed.. even though I could check the version and output 1.0.0. Then somehow I managed to get it to acknowledge the package existed (I cannot reproduce this step idk what I did)
When I thought had I finally done it.. I got a syntax error, coming from one of the packages I installed. Unfortunately I do not have the error messages from this last bit, because I changed something and I was unable to reproduce it.
I am quite new to computational biology (as I'm sure you can tell), but I would greatly appreciate any help or pointers on what I might have messed up on. I don't know whether all the global packages are messing things up, I have messed around quite a lot with installations on my laptop. I was doing everything here in a conda env though.

This package looks super useful to me but I cannot for the life of me get it to work.
Again, thanks for any help, anything will be greatly appreciated, I have spent many hours trying to solve this.

[orbeckst]

The MDAnalysis problem is known (see MDAnalysis/mdanalysis#2938) and you found the correct workaround.

Working in a conda environment is an excellent idea.

For anything else, please

• copy and paste your input (what are your doing to install)
• copy and paste your output (anything that says error and what leads up to it) or attach a log file
  Without details it's unlikely that anyone can help.

With installation issues it is important to keep track of everything that you're doing. Copy and paste everything into a text file. In your case, I'd suggest you just start with a fresh conda env with Python 2.7 (Python 3 is not supported, see #84 )

conda create -c conda-forge -n mdpow python=2.7 numpy scipy 'matplotlib<3.3' 'mdanalysis<2' 'mdanalysistests<2' pyyaml
conda activate mdpow  
pip install gromacswrapper
git clone https://github.com/Becksteinlab/MDPOW.git
pip install ./MDPOW

Then see if it installed with

mdpow-equilibrium --help

The installation instructions say that there's a directly pip-installable version available but at the moment there isn't. (EDIT: removed from installation docs). Also, I recommend you install the current development version (from the git repo) because it has a lot of changes/fixes compared to the 0.6.1 release. We wanted to release a 0.7.0 but haven't gotten around to it.

[orbeckst]

I updated the INSTALL instructions, too.

[Ntr9999]

Thank you for your quick response, I greatly appreciate your assistance.
So I started from scratch in a fresh environment, as you suggest:
input:

conda create -c conda-forge -n mdpow python=2.7 numpy scipy 'matplotlib<3.3' 'mdanalysis<2' 'mdanalysistests<2' pyyaml
conda activate mdpow
pip install gromacswrapper
git clone https://github.com/Becksteinlab/MDPOW.git
pip install ./MDPOW
mdpow-equilibrium --help

output: (from the mdpow-equilibirum --help command, everything prior was fine I do believe)

(mdpow) Mark@Marks-MBP ~ % mdpow-equilibrium --help
Traceback (most recent call last):
  File "/usr/local/bin/mdpow-equilibrium", line 4, in <module>
    __import__('pkg_resources').run_script('MDPOW==0.7.0.dev0', 'mdpow-equilibrium')
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 3241, in <module>
    @_call_aside
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 3225, in _call_aside
    f(*args, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'mdanalysis' distribution was not found and is required by MDPOW

This is what has been confusing me, because I have checked to see if MDAnalysis is installed, and it appears as though it is:

python
>>> import MDAnalysis
from MDAnalysis.tests.datafiles import PSF, DCD
print(MDAnalysis.Universe(PSF, DCD))
>>> from MDAnalysis.tests.datafiles import PSF, DCD
>>> print(MDAnalysis.Universe(PSF, DCD))
<Universe with 3341 atoms>
>>> print(MDAnalysis.__version__)
0.20.1
>>> 

I also checked to see if mdpow was installed properly and it looks like it is too:
python

>>> import mdpow
>>> help(mdpow)

Help on package mdpow:

NAME
    mdpow

FILE
    /Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/mdpow/__init__.py

DESCRIPTION
    # POW package __init__.py
    # Copyright (c) 2010 Oliver Beckstein <[email protected]>
    # Released under the GNU Public License 3 (or higher, your choice)
    # See the file COPYING for details.

PACKAGE CONTENTS
    config
    equil
    fep
    filelock
    forcefields
    log
    restart
:

Any idea why it doesn't want to acknowledge the MDAnalysis package?

Thanks again. :)

[orbeckst]

There's an old installation of MDPOW (and possibly other stuff) in /usr/local as indicated by /usr/local/bin/mdpow-equilibrium" and /System/Library/Frameworks/.... Somehow your conda environment is not properly isolated from the rest of your system. Are you setting PYTHONPATH or PATH environment variables anywhere?

Look at the output of

echo "PYTHONPATH:=$PYTHONPATH"
echo "PATH=$PATH"
which python
which mdpow-equilibrium

Your MDAnalysis installation is outdated. This should be version 1.0.0. Where is it installed?

import MDAnalysis
print(MDAnalysis.__file__)

should be under your anaconda env.

EDIT: See below – installation of 0.20.1 instead of 1.0.0 is a bug MDAnalysis/mdanalysis#3105

[Ntr9999]

Okay, so I restarted my terminal and went back into the environment and that error no longer occurs (I get the correct -help output).

echo "PYTHONPATH:=$PYTHONPATH"
echo "PATH=$PATH"
which python
which mdpow-equilibrium

PYTHONPATH:=
PATH=/Users/mark/opt/anaconda3/envs/mdpow/bin:/Users/mark/opt/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
/Users/mark/opt/anaconda3/envs/mdpow/bin/python
/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-equilibrium

/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/MDAnalysis/__init__.pyc

Everything looks good with those locations now? So I think that problem with MDAnalysis not being found has solved.

However, my version is still 0.20.1

print(MDAnalysis.__version__)
0.20.1

Why would this be the case?
Thanks again.

[orbeckst]

Maybe

conda update -n mdpow mdanalysis mdanalysistests

?

[orbeckst]

Btw, what operating system are you running?

Can you show the output from starting python (eg mine is

Python 2.7.15 | packaged by conda-forge | (default, Mar  5 2020, 14:58:04)
[GCC Clang 9.0.1 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.

just to see if there's a weird combo of O/S & python for which for some reason no 1.0.0 conda package is available.)

[orbeckst]

And run

conda list mdanalysis
# packages in environment at ~/anaconda3/envs/mdpow:
#
# Name                    Version                   Build  Channel
mdanalysis                0.20.1           py27h4a8c4bd_0    conda-forge
mdanalysistests           0.20.1                   py27_0    conda-forge

[Ntr9999]

I have just realised why it was outputting that error about MDA not found.
When I source my gromacs with "source /usr/local/bin/GMXRC" and then try and run:
mdpow-equilibrium --help,
it outputs the error.

It only works when I don't source my gromacs...

Do you think the location of my gromacs is causing errors? because surely I am going to need to source it at some point :/

[orbeckst]

... oops, I have 0.20.1 too!

I'll have to look into this on MDAnalysis's side.

I guess for right now, 0.20.1 should work... (fingers crossed, it's not super-essential for MDPOW IIRC.)

[orbeckst]

Your source GMXRC changes PATH. In this case it picks up your old installation.

Uninstall your old MDPOW installation (always a good idea).

I'd also install GROMACS in a dedicated directory that's not on the normal PATH, e.g. /opt/gromacs/2020.2 or similar.

[Ntr9999]

Okay, thank you so much for your help!
I will reinstall my GROMACS to a new directory now.

[orbeckst]

I opened MDAnalysis/mdanalysis#3105 for the 0.20.1 version of MDAnalysis.

[Ntr9999]

Okay, so I reinstalled my gromacs and I am able to source it and run the -help mdpow command.

However, when doing the tutorial, I run into error messages that were encountered here: #96
after running this:
export GW_START_LOGGING=True mdpow-equilibrium --solvent water benzene/benzene.yml

I encounter these errors (very similar to that in the previous thread I linked)

Command line:
  gmx mdrun -cpi -stepout 1000 -deffnm md -v -append -nice 19


-------------------------------------------------------
Program:     gmx mdrun, version 2020
Source file: src/gromacs/mdrunutility/handlerestart.cpp (line 238)
Function:    gmx::(anonymous namespace)::StartingBehaviorHandler gmx::(anonymous namespace)::chooseStartingBehavior(const gmx::AppendingBehavior, const int, t_filenm *)

Inconsistency in user input:
Could not do a restart with appending because the checkpoint file was not
found. Either supply the name of the right checkpoint file or do not use
-append

For more information and tips for troubleshooting, please check the GROMACS
website at 
-------------------------------------------------------
gromacs.run : CRITICAL Failure in MDrun, returncode = 1
gromacs.run : CRITICAL Failure in MDrun, returncode = 1
gromacs.run : ERROR    run_check: mdrun failed to complete run
gromacs.run : ERROR    run_check: mdrun failed to complete run
mdpow.run   : CRITICAL Failed MD_relaxed, investigate manually.

To preemptively answer some questions you may ask (similar to the thread I linked):
I find it hard to believe that it hasn't sourced properly, especially since I was able to do the equilibration step.
which mdrun
yields:
mdrun not found

which gmx
yields:
/usr/local/opt/gromacs/bin/gmx

import gromacs gromacs.mdrun(h=True)
yields:

                       :-) GROMACS - gmx mdrun, 2020 (-:

                            GROMACS is written by:
     Emile Apol      Rossen Apostolov      Paul Bauer     Herman J.C. Berendsen
    Par Bjelkmar      Christian Blau   Viacheslav Bolnykh     Kevin Boyd    
 Aldert van Buuren   Rudi van Drunen     Anton Feenstra       Alan Gray     
  Gerrit Groenhof     Anca Hamuraru    Vincent Hindriksen  M. Eric Irrgang  
  Aleksei Iupinov   Christoph Junghans     Joe Jordan     Dimitrios Karkoulis
    Peter Kasson        Jiri Kraus      Carsten Kutzner      Per Larsson    
  Justin A. Lemkul    Viveca Lindahl    Magnus Lundborg     Erik Marklund   
    Pascal Merz     Pieter Meulenhoff    Teemu Murtola       Szilard Pall   
    Sander Pronk      Roland Schulz      Michael Shirts    Alexey Shvetsov  
   Alfons Sijbers     Peter Tieleman      Jon Vincent      Teemu Virolainen 
 Christian Wennberg    Maarten Wolf      Artem Zhmurov   
                           and the project leaders:
        Mark Abraham, Berk Hess, Erik Lindahl, and David van der Spoel

Copyright (c) 1991-2000, University of Groningen, The Netherlands.
Copyright (c) 2001-2019, The GROMACS development team at
Uppsala University, Stockholm University and
the Royal Institute of Technology, Sweden.
check out http://www.gromacs.org for more information.

GROMACS is free software; you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License
as published by the Free Software Foundation; either version 2.1
of the License, or (at your option) any later version.

GROMACS:      gmx mdrun, version 2020
Executable:   /usr/local/opt/gromacs/bin/gmx
Data prefix:  /usr/local/opt/gromacs
Working dir:  /Users/matt/Desktop/benzene
Command line:
  gmx mdrun -h

SYNOPSIS

gmx mdrun [-s [<.tpr>]] [-cpi [<.cpt>]] [-table [<.xvg>]] [-tablep [<.xvg>]]
          [-tableb [<.xvg> [...]]] [-rerun [<.xtc/.trr/...>]] [-ei [<.edi>]]
          [-multidir [<dir> [...]]] [-awh [<.xvg>]] [-membed [<.dat>]]
          [-mp [<.top>]] [-mn [<.ndx>]] [-o [<.trr/.cpt/...>]]
          [-x [<.xtc/.tng>]] [-cpo [<.cpt>]] [-c [<.gro/.g96/...>]]
          [-e [<.edr>]] [-g [<.log>]] [-dhdl [<.xvg>]] [-field [<.xvg>]]
          [-tpi [<.xvg>]] [-tpid [<.xvg>]] [-eo [<.xvg>]] [-px [<.xvg>]]
          [-pf [<.xvg>]] [-ro [<.xvg>]] [-ra [<.log>]] [-rs [<.log>]]
          [-rt [<.log>]] [-mtx [<.mtx>]] [-if [<.xvg>]] [-swap [<.xvg>]]
          [-deffnm <string>] [-xvg <enum>] [-dd <vector>] [-ddorder <enum>]
          [-npme <int>] [-nt <int>] [-ntmpi <int>] [-ntomp <int>]
          [-ntomp_pme <int>] [-pin <enum>] [-pinoffset <int>]
          [-pinstride <int>] [-gpu_id <string>] [-gputasks <string>]
          [-[no]ddcheck] [-rdd <real>] [-rcon <real>] [-dlb <enum>]
          [-dds <real>] [-nb <enum>] [-nstlist <int>] [-[no]tunepme]
          [-pme <enum>] [-pmefft <enum>] [-bonded <enum>] [-update <enum>]
          [-[no]v] [-pforce <real>] [-[no]reprod] [-cpt <real>] [-[no]cpnum]
          [-[no]append] [-nsteps <int>] [-maxh <real>] [-replex <int>]
          [-nex <int>] [-reseed <int>]

DESCRIPTION

gmx mdrun is the main computational chemistry engine within GROMACS.
Obviously, it performs Molecular Dynamics simulations, but it can also perform
Stochastic Dynamics, Energy Minimization, test particle insertion or
(re)calculation of energies. Normal mode analysis is another option. In this
case mdrun builds a Hessian matrix from single conformation. For usual Normal
Modes-like calculations, make sure that the structure provided is properly
energy-minimized. The generated matrix can be diagonalized by gmx nmeig.

The mdrun program reads the run input file (-s) and distributes the topology
over ranks if needed. mdrun produces at least four output files. A single log
file (-g) is written. The trajectory file (-o), contains coordinates,
velocities and optionally forces. The structure file (-c) contains the
coordinates and velocities of the last step. The energy file (-e) contains
energies, the temperature, pressure, etc, a lot of these things are also
printed in the log file. Optionally coordinates can be written to a compressed
trajectory file (-x).

The option -dhdl is only used when free energy calculation is turned on.

Running mdrun efficiently in parallel is a complex topic, many aspects of
which are covered in the online User Guide. You should look there for
practical advice on using many of the options available in mdrun.

ED (essential dynamics) sampling and/or additional flooding potentials are
switched on by using the -ei flag followed by an .edi file. The .edi file can
be produced with the make_edi tool or by using options in the essdyn menu of
the WHAT IF program. mdrun produces a .xvg output file that contains
projections of positions, velocities and forces onto selected eigenvectors.

When user-defined potential functions have been selected in the .mdp file the
-table option is used to pass mdrun a formatted table with potential
functions. The file is read from either the current directory or from the
GMXLIB directory. A number of pre-formatted tables are presented in the GMXLIB
dir, for 6-8, 6-9, 6-10, 6-11, 6-12 Lennard-Jones potentials with normal
Coulomb. When pair interactions are present, a separate table for pair
interaction functions is read using the -tablep option.

When tabulated bonded functions are present in the topology, interaction
functions are read using the -tableb option. For each different tabulated
interaction type used, a table file name must be given. For the topology to
work, a file name given here must match a character sequence before the file
extension. That sequence is: an underscore, then a 'b' for bonds, an 'a' for
angles or a 'd' for dihedrals, and finally the matching table number index
used in the topology. Note that, these options are deprecated, and in future
will be available via grompp.

The options -px and -pf are used for writing pull COM coordinates and forces
when pulling is selected in the .mdp file.

The option -membed does what used to be g_membed, i.e. embed a protein into a
membrane. This module requires a number of settings that are provided in a
data file that is the argument of this option. For more details in membrane
embedding, see the documentation in the user guide. The options -mn and -mp
are used to provide the index and topology files used for the embedding.

The option -pforce is useful when you suspect a simulation crashes due to too
large forces. With this option coordinates and forces of atoms with a force
larger than a certain value will be printed to stderr. It will also terminate
the run when non-finite forces are present.

Checkpoints containing the complete state of the system are written at regular
intervals (option -cpt) to the file -cpo, unless option -cpt is set to -1. The
previous checkpoint is backed up to state_prev.cpt to make sure that a recent
state of the system is always available, even when the simulation is
terminated while writing a checkpoint. With -cpnum all checkpoint files are
kept and appended with the step number. A simulation can be continued by
reading the full state from file with option -cpi. This option is intelligent
in the way that if no checkpoint file is found, GROMACS just assumes a normal
run and starts from the first step of the .tpr file. By default the output
will be appending to the existing output files. The checkpoint file contains
checksums of all output files, such that you will never loose data when some
output files are modified, corrupt or removed. There are three scenarios with
-cpi:

* no files with matching names are present: new output files are written

* all files are present with names and checksums matching those stored in the
checkpoint file: files are appended

* otherwise no files are modified and a fatal error is generated

With -noappend new output files are opened and the simulation part number is
added to all output file names. Note that in all cases the checkpoint file
itself is not renamed and will be overwritten, unless its name does not match
the -cpo option.

With checkpointing the output is appended to previously written output files,
unless -noappend is used or none of the previous output files are present
(except for the checkpoint file). The integrity of the files to be appended is
verified using checksums which are stored in the checkpoint file. This ensures
that output can not be mixed up or corrupted due to file appending. When only
some of the previous output files are present, a fatal error is generated and
no old output files are modified and no new output files are opened. The
result with appending will be the same as from a single run. The contents will
be binary identical, unless you use a different number of ranks or dynamic
load balancing or the FFT library uses optimizations through timing.

With option -maxh a simulation is terminated and a checkpoint file is written
at the first neighbor search step where the run time exceeds -maxh*0.99 hours.
This option is particularly useful in combination with setting nsteps to -1
either in the mdp or using the similarly named command line option (although
the latter is deprecated). This results in an infinite run, terminated only
when the time limit set by -maxh is reached (if any) or upon receiving a
signal.

Interactive molecular dynamics (IMD) can be activated by using at least one of
the three IMD switches: The -imdterm switch allows one to terminate the
simulation from the molecular viewer (e.g. VMD). With -imdwait, mdrun pauses
whenever no IMD client is connected. Pulling from the IMD remote can be turned
on by -imdpull. The port mdrun listens to can be altered by -imdport.The file
pointed to by -if contains atom indices and forces if IMD pulling is used.

OPTIONS

Options to specify input files:

 -s      [<.tpr>]           (topol.tpr)
           Portable xdr run input file
 -cpi    [<.cpt>]           (state.cpt)      (Opt.)
           Checkpoint file
 -table  [<.xvg>]           (table.xvg)      (Opt.)
           xvgr/xmgr file
 -tablep [<.xvg>]           (tablep.xvg)     (Opt.)
           xvgr/xmgr file
 -tableb [<.xvg> [...]]     (table.xvg)      (Opt.)
           xvgr/xmgr file
 -rerun  [<.xtc/.trr/...>]  (rerun.xtc)      (Opt.)
           Trajectory: xtc trr cpt gro g96 pdb tng
 -ei     [<.edi>]           (sam.edi)        (Opt.)
           ED sampling input
 -multidir [<dir> [...]]    (rundir)         (Opt.)
           Run directory
 -awh    [<.xvg>]           (awhinit.xvg)    (Opt.)
           xvgr/xmgr file
 -membed [<.dat>]           (membed.dat)     (Opt.)
           Generic data file
 -mp     [<.top>]           (membed.top)     (Opt.)
           Topology file
 -mn     [<.ndx>]           (membed.ndx)     (Opt.)
           Index file

Options to specify output files:

 -o      [<.trr/.cpt/...>]  (traj.trr)
           Full precision trajectory: trr cpt tng
 -x      [<.xtc/.tng>]      (traj_comp.xtc)  (Opt.)
           Compressed trajectory (tng format or portable xdr format)
 -cpo    [<.cpt>]           (state.cpt)      (Opt.)
           Checkpoint file
 -c      [<.gro/.g96/...>]  (confout.gro)
           Structure file: gro g96 pdb brk ent esp
 -e      [<.edr>]           (ener.edr)
           Energy file
 -g      [<.log>]           (md.log)
           Log file
 -dhdl   [<.xvg>]           (dhdl.xvg)       (Opt.)
           xvgr/xmgr file
 -field  [<.xvg>]           (field.xvg)      (Opt.)
           xvgr/xmgr file
 -tpi    [<.xvg>]           (tpi.xvg)        (Opt.)
           xvgr/xmgr file
 -tpid   [<.xvg>]           (tpidist.xvg)    (Opt.)
           xvgr/xmgr file
 -eo     [<.xvg>]           (edsam.xvg)      (Opt.)
           xvgr/xmgr file
 -px     [<.xvg>]           (pullx.xvg)      (Opt.)
           xvgr/xmgr file
 -pf     [<.xvg>]           (pullf.xvg)      (Opt.)
           xvgr/xmgr file
 -ro     [<.xvg>]           (rotation.xvg)   (Opt.)
           xvgr/xmgr file
 -ra     [<.log>]           (rotangles.log)  (Opt.)
           Log file
 -rs     [<.log>]           (rotslabs.log)   (Opt.)
           Log file
 -rt     [<.log>]           (rottorque.log)  (Opt.)
           Log file
 -mtx    [<.mtx>]           (nm.mtx)         (Opt.)
           Hessian matrix
 -if     [<.xvg>]           (imdforces.xvg)  (Opt.)
           xvgr/xmgr file
 -swap   [<.xvg>]           (swapions.xvg)   (Opt.)
           xvgr/xmgr file

Other options:

 -deffnm <string>
           Set the default filename for all file options
 -xvg    <enum>             (xmgrace)
           xvg plot formatting: xmgrace, xmgr, none
 -dd     <vector>           (0 0 0)
           Domain decomposition grid, 0 is optimize
 -ddorder <enum>            (interleave)
           DD rank order: interleave, pp_pme, cartesian
 -npme   <int>              (-1)
           Number of separate ranks to be used for PME, -1 is guess
 -nt     <int>              (0)
           Total number of threads to start (0 is guess)
 -ntmpi  <int>              (0)
           Number of thread-MPI ranks to start (0 is guess)
 -ntomp  <int>              (0)
           Number of OpenMP threads per MPI rank to start (0 is guess)
 -ntomp_pme <int>           (0)
           Number of OpenMP threads per MPI rank to start (0 is -ntomp)
 -pin    <enum>             (auto)
           Whether mdrun should try to set thread affinities: auto, on, off
 -pinoffset <int>           (0)
           The lowest logical core number to which mdrun should pin the first
           thread
 -pinstride <int>           (0)
           Pinning distance in logical cores for threads, use 0 to minimize
           the number of threads per physical core
 -gpu_id <string>
           List of unique GPU device IDs available to use
 -gputasks <string>
           List of GPU device IDs, mapping each PP task on each node to a
           device
 -[no]ddcheck               (yes)
           Check for all bonded interactions with DD
 -rdd    <real>             (0)
           The maximum distance for bonded interactions with DD (nm), 0 is
           determine from initial coordinates
 -rcon   <real>             (0)
           Maximum distance for P-LINCS (nm), 0 is estimate
 -dlb    <enum>             (auto)
           Dynamic load balancing (with DD): auto, no, yes
 -dds    <real>             (0.8)
           Fraction in (0,1) by whose reciprocal the initial DD cell size will
           be increased in order to provide a margin in which dynamic load
           balancing can act while preserving the minimum cell size.
 -nb     <enum>             (auto)
           Calculate non-bonded interactions on: auto, cpu, gpu
 -nstlist <int>             (0)
           Set nstlist when using a Verlet buffer tolerance (0 is guess)
 -[no]tunepme               (yes)
           Optimize PME load between PP/PME ranks or GPU/CPU
 -pme    <enum>             (auto)
           Perform PME calculations on: auto, cpu, gpu
 -pmefft <enum>             (auto)
           Perform PME FFT calculations on: auto, cpu, gpu
 -bonded <enum>             (auto)
           Perform bonded calculations on: auto, cpu, gpu
 -update <enum>             (auto)
           Perform update and constraints on: auto, cpu, gpu
 -[no]v                     (no)
           Be loud and noisy
 -pforce <real>             (-1)
           Print all forces larger than this (kJ/mol nm)
 -[no]reprod                (no)
           Try to avoid optimizations that affect binary reproducibility
 -cpt    <real>             (15)
           Checkpoint interval (minutes)
 -[no]cpnum                 (no)
           Keep and number checkpoint files
 -[no]append                (yes)
           Append to previous output files when continuing from checkpoint
           instead of adding the simulation part number to all file names
 -nsteps <int>              (-2)
           Run this number of steps (-1 means infinite, -2 means use mdp
           option, smaller is invalid)
 -maxh   <real>             (-1)
           Terminate after 0.99 times this time (hours)
 -replex <int>              (0)
           Attempt replica exchange periodically with this period (steps)
 -nex    <int>              (0)
           Number of random exchanges to carry out each exchange interval (N^3
           is one suggestion).  -nex zero or not specified gives neighbor
           replica exchange.
 -reseed <int>              (-1)
           Seed for replica exchange, -1 is generate a seed

GROMACS reminds you: "Everybody Lie Down On the Floor and Keep Calm" (KLF)

(0, None, None)
>>> 

print(gromacs.mdrun.driver) gmx

Again, I greatly appreciate your assistance in troubleshooting this.
Many thanks.

gromacs.log
mdpow.log

[Ntr9999]

The user in the linked thread said she had installed the package on her desktop and that was causing the errors? I think I built gromacs from my desktop, but I source it from /opt/gromacs/.

Any guidance you can provide will be greatly appreciated.
Thanks

[orbeckst]

@Ntr9999 , it is possible that GROMACS 2020 does not allow you to start a simulation with -cpi -append (see PR #127 and below).

@VOD555 does this error sound familiar to you — have you been running MDPOW with GROMACS 2020?

[orbeckst]

@Ntr9999 could you please try out the code in PR #127 and see if it fixes the problem?

cd MDPOW
git fetch --all
git checkout -b no-append origin/no-append
# then install this version, e.g.,
pip install --upgrade .

Let me know if this worked.

[Ntr9999]

I do believe it worked! Was it because of the gromacs version?

mdpow.checkpoint: INFO checkpoint: MD_NPT_run mdpow.run : INFO Equilibrium simulation phase complete, use 'benzene/water.simulation' to continue.

However the next step has an mdrun error too, would you recommend that I downgrade my gromacs to one of the known supported versions?

Input:
(mdpow) mark@marks-MBP benzene % export GW_START_LOGGING=True mdpow-equilibrium --solvent octanol benzene/benzene.yml

Output (of the step that fails):
`GROMACS: gmx mdrun, version 2020
Executable: /usr/local/opt/gromacs/bin/gmx
Data prefix: /usr/local/opt/gromacs
Working dir: /Users/mark/Desktop/benzene/benzene/Equilibrium/octanol/em
Command line:
gmx mdrun -c em.pdb -deffnm em -stepout 10 -v

Reading file em.tpr, VERSION 2020 (single precision)

---

Program: gmx mdrun, version 2020
Source file: src/gromacs/domdec/domdec.cpp (line 2275)
MPI rank: 0 (out of 4)

Fatal error:
There is no domain decomposition for 4 ranks that is compatible with the given
box and a minimum cell size of 3.9551 nm
Change the number of ranks or mdrun option -rdd or -dds
Look in the log file for details on the domain decomposition

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors

Traceback (most recent call last):
File "/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-equilibrium", line 62, in
S = equilibrium_simulation(cfg, opts.solvent, dirname=opts.dirname)
File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/mdpow/run.py", line 227, in equilibrium_simulation
S.energy_minimize(maxwarn=maxwarn)
File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/mdpow/equil.py", line 431, in energy_minimize
params = gromacs.setup.energy_minimize(**kwargs)
File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/gromacs/setup.py", line 731, in energy_minimize
mdrun(**mdrun_args)
File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/gromacs/core.py", line 398, in call
return self.run(args, **kwargs)
File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/gromacs/core.py", line 178, in run
results, p = self._run_command(_args, **_kwargs)
File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/gromacs/core.py", line 611, in _run_command
self.check_failure(result, command_string=p.command_string)
File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/gromacs/core.py", line 566, in check_failure
raise GromacsError(rc, msg)
gromacs.exceptions.GromacsError: [Errno 1] Gromacs tool failed
Command invocation: gmx mdrun -c em.pdb -deffnm em -stepout 10 -v`

So the error appears to be this: http://www.gromacs.org/Documentation/Errors#There_is_no_domain_decomposition_for_n_nodes_that_is_compatible_with_the_given_box_and_a_minimum_cell_size_of_x_nm

Do you think this is because of my gromacs installation or my computer or just that the simulation box is too small?
I would try to run some fixes, but because the script is automated I don't want to mess things up.
Thank you very much.

Here is the log for the two steps combined:
gromacs.log
mdpow.log

[iorga]

I do believe it worked! Was it because of the gromacs version?

This syntax with -append was correct with previous versions of Gromacs (~4.5-4.6), but is not accepted anymore by more recent versions (e.g. 2020).

However the next step has an mdrun error too, would you recommend that I downgrade my gromacs to one of the known supported versions?

No, you should not downgrade Gromacs, we should make it work with the more recent versions.

Input:
(mdpow) mark@marks-MBP benzene % export GW_START_LOGGING=True mdpow-equilibrium --solvent octanol benzene/benzene.yml

Output (of the step that fails):
`GROMACS: gmx mdrun, version 2020
Executable: /usr/local/opt/gromacs/bin/gmx
Data prefix: /usr/local/opt/gromacs
Working dir: /Users/mark/Desktop/benzene/benzene/Equilibrium/octanol/em
Command line:
gmx mdrun -c em.pdb -deffnm em -stepout 10 -v

Reading file em.tpr, VERSION 2020 (single precision)

Program: gmx mdrun, version 2020
Source file: src/gromacs/domdec/domdec.cpp (line 2275)
MPI rank: 0 (out of 4)

Fatal error:
There is no domain decomposition for 4 ranks that is compatible with the given
box and a minimum cell size of 3.9551 nm
Change the number of ranks or mdrun option -rdd or -dds
Look in the log file for details on the domain decomposition

I've got the same error some time ago, which was reported in #119.

Do you think this is because of my gromacs installation or my computer or just that the simulation box is too small?
I would try to run some fixes, but because the script is automated I don't want to mess things up.
Thank you very much.

For the EM step, Gromacs uses all threads available on your computer. In some cases, the number of threads is not compatible with the default domain decomposition, and currently there is no way to control it. Solving #119 should fix this.

In my case, a workaround was to run the simulation on another machine, with a different number of threads (8 instead of 20) which was compatible with the default domain decomposition.

[Ntr9999]

For the EM step, Gromacs uses all threads available on your computer. In some cases, the number of threads is not compatible with the default domain decomposition, and currently there is no way to control it. Solving #119 should fix this.

In my case, a workaround was to run the simulation on another machine, with a different number of threads (8 instead of 20) which was compatible with the default domain decomposition.

I see, thank you very much for your input.

[Ntr9999]

In the .yml file, there is a section, shown below:
mdrun: stepout: 1000 # how often to report progress (-stepout) verbose: True # report progress verbosely (True is -v) nice: 19 # nicel level maxthreads: None # maximum number of threads (-nt), 0 or None lets mdrun automatically choose
Can't I change max threads here, or is this for something else?

[iorga]

You should definitely try to set this maxthreads option to 2 or even 1. I don't remember exactly what I've got when I tried it, quite a long time ago. I think it applies only to the MD_relaxed and MD_NPT simulations (when run locally), not to the EM step, but I'm not sure. Please check it.

[Ntr9999]

I think it applies only to the MD_relaxed and MD_NPT simulations (when run locally), not to the EM step, but I'm not sure. Please check it.

You're right, it doesn't apply to the EM step.

[Ntr9999]

Hi again 😫.

I'm trying to run the package on quite a small unusual molecule and I believe that my .itp file is correct, however, when I run:

mdpow-equilibrium --solvent water runinput.yml

I get error messages implying that the force field has not been adapted to the new "residue":

ERROR 1 [file diaz.itp, line 26]:
  No default Bond types


ERROR 2 [file diaz.itp, line 27]:
  No default Bond types


ERROR 3 [file diaz.itp, line 30]:
  No default Bond types


ERROR 4 [file diaz.itp, line 53]:
  No default Angle types


ERROR 5 [file diaz.itp, line 54]:
  No default Angle types


ERROR 6 [file diaz.itp, line 57]:
  No default Angle types


ERROR 7 [file diaz.itp, line 58]:
  No default Angle types


ERROR 8 [file diaz.itp, line 59]:
  No default Angle types


ERROR 9 [file diaz.itp, line 60]:
  No default Angle types


ERROR 10 [file diaz.itp, line 62]:
  No default Angle types


ERROR 11 [file diaz.itp, line 64]:
  No default Angle types


ERROR 12 [file diaz.itp, line 66]:
  No default Angle types


ERROR 13 [file diaz.itp, line 67]:
  No default Angle types


ERROR 14 [file diaz.itp, line 68]:
  No default Angle types


ERROR 15 [file diaz.itp, line 69]:
  No default Angle types


ERROR 16 [file diaz.itp, line 70]:
  No default Angle types


ERROR 17 [file diaz.itp, line 71]:
  No default Angle types


ERROR 18 [file diaz.itp, line 76]:
  No default Ryckaert-Bell. types


ERROR 19 [file diaz.itp, line 79]:
  No default Ryckaert-Bell. types


ERROR 20 [file diaz.itp, line 80]:
  No default Ryckaert-Bell. types


ERROR 21 [file diaz.itp, line 82]:
  No default Ryckaert-Bell. types


ERROR 22 [file diaz.itp, line 83]:
  No default Ryckaert-Bell. types

grompp_qtot() failed. See warning and screen output for clues.

I assume grumpp_qtot is failing because of these errors, or should I change my qtot to an integer, I didn't think this was actually essential.

I was also under the impression that mymolecule.itp was incorporated automatically by the package to the top of the topology with #include.
Or am I required to update the force field files myself? If so, are these the ones in the MDPOW directory, or in the gromacs installation in /opt/gromacs/?

Here is my .ipt and .yml file
my .ipt file:

[ moleculetype ]
; Name            nrexcl
DIA                 3

[ atoms ]
;   nr       type  resnr residue  atom   cgnr     charge       mass  typeB    chargeB      massB
     1   opls_135      1    DIA      C      1     -0.180   12.01100                             
     2   opls_906      1    DIA      C      2      0.060   12.01100                             
     3   opls_900      1    DIA      N      3     -0.900   14.00670                             
     4   opls_909      1    DIA      H      3      0.360    1.00800                             
     5   opls_909      1    DIA      H      3      0.360    1.00800                             
     6   opls_900      1    DIA      N      4     -0.900   14.00670                             
     7   opls_909      1    DIA      H      4      0.360    1.00800                             
     8   opls_909      1    DIA      H      4      0.360    1.00800                             
     9   opls_145      1    DIA      C      5     -0.115   12.01100                             
    10   opls_146      1    DIA      H      5      0.115    1.00800                             
    11   opls_154      1    DIA      O      5     -0.683   15.99940                             
    12   opls_155      1    DIA      H      5      0.418    1.00800                             
    13   opls_236      1    DIA      O      5     -0.500   15.99940                             

[ bonds ]
;  ai    aj funct
    1     2     1	;     C     C
    1     3     1	;     C     N
    1     4     1	;     C     H
    1     5     1	;     C     H
    2     6     1	;     C     N
    2     9     1	;     C     C
    2    10     1	;     C     H
    3     6     1	;     N     N
    3     8     1	;     N     H
    6     7     1	;     N     H
    9    11     1	;     C     O
    9    13     1	;     C     O
   11    12     1	;     O     H

[ pairs ]
;  ai    aj funct
    2     6     1	;     C     N
    2     8     1	;     C     H
    1     7     1	;     C     H
    1    11     1	;     C     O
    1    13     1	;     C     O
    1     7     1	;     N     H
    6    11     1	;     N     O
    6    13     1	;     N     O
    2    12     1	;     C     H

[ angles ]
;  ai    aj    ak funct
    2     1     3     1	;     C     C     N
    2     1     4     1	;     C     C     H
    2     1     5     1	;     C     C     H
    1     2     6     1	;     C     C     N
    1     2     9     1	;     C     C     C
    1     2    10     1	;     C     C     H
    3     1     4     1	;     N     C     H
    3     1     5     1	;     N     C     H
    1     3     6     1	;     N     C     N
    1     3     8     1	;     N     C     H
    4     1     5     1	;     H     C     H
    6     2     9     1	;     N     C     C
    6     2    10     1	;     N     C     H
    2     6     7     1	;     N     C     H
    9     2    10     1	;     C     C     H
    2     9    11     1	;     C     C     O
    2     9    13     1	;     C     C     O
    6     3     8     1	;     N     N     H
    3     6     7     1	;     N     N     H
   11     9    13     1	;     O     C     O
    9    11    12     1	;     O     C     H

[ dihedrals ]
;  ai    aj    ak    al funct
    2     1     3     6     3	;     C     C     N     N
    2     1     3     8     3	;     C     C     N     H
    1     2     6     7     3	;     C     C     N     H
    1     2     9    11     3	;     C     C     C     O
    1     2     9    13     3	;     C     C     C     O
    1     3     6     7     3	;     N     C     N     H
    6     2     9    11     3	;     N     C     C     O
    6     2     9    13     3	;     N     C     C     O
    2     9    11    12     3	;     C     C     O     H

[ dihedrals ]
;  aj    ak    ai    al funct
    1     6     3     8     1    improper_Z_N_X_Y	;     H     C     N     N
    2     7     6     3     1    improper_Z_N_X_Y	;     N     C     H     N
    2    13     9    11     1    improper_O_C_X_Y	;     O     C     O     C

My .yml file:

DEFAULT:
  qscripts: local.sh
FEP:
  maxwarn: 0
  mdp: bar_opls.mdp
  method: BAR
  qscript: local.sh
  runlocal: true
  runtime: 5000
FEP_schedule_Coulomb:
  couple_lambda0: vdw-q
  couple_lambda1: vdw
  description: dis-charging vdw+q --> vdw
  label: Coul
  lambdas: 0, 0.25, 0.5, 0.75, 1.0
  name: Coulomb
  sc_alpha: 0
  sc_power: 1
  sc_sigma: 0.3
FEP_schedule_VDW:
  couple_lambda0: vdw
  couple_lambda1: none
  description: decoupling vdw --> none
  label: VDW
  lambdas: 0.0, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9,
    0.95, 1
  name: vdw
  sc_alpha: 0.5
  sc_power: 1
  sc_sigma: 0.3
MD_NPT:
  mdp: NPT_opls.mdp
  qscript: local.sh
  runlocal: true
  runtime: 50000
MD_relaxed:
  mdp: NPT_opls.mdp
  qscript: local.sh
  runlocal: true
  runtime: 5
energy_minimize:
  mdp: em_opls.mdp
mdrun:
  maxthreads: None
  nice: 19
  stepout: 1000
  verbose: true
setup:
  boxtype: dodecahedron
  distance: None
  forcefield: OPLS-AA
  gromacsoutput: false
  itp: "/Users/mark/Desktop/diaz/diaz.itp"
  maxwarn: 0
  molecule: dia
  name: diaz
  prm: None
  solventmodel: tip4p
  structure: "/Users/mark/Desktop/diaz/diaz.pdb"

Thank you again for any assistance you provide; I appreciate that these questions may seem very trivial an experienced user.

EDIT (@orbeckst): formatting

[orbeckst]

Reopening — sorry, PR #127 only solved the append problem.

[orbeckst]

@Ntr9999 did you manage to run the benzene example? Ideally I'd like to hear that this all worked before looking into anything else.

If the maxthreads issue #119 is a problem you can do one of two things: make the box much bigger (so that Gromacs is happy to run it on more cores), by increasing the min distance, or try out the code in branch emin-maxthreads (PR #128).

Can you open a new issue for your own data so that we can deal with different issues in self contained blocks, please?

[Ntr9999]

did you manage to run the benzene example? Ideally I'd like to hear that this all worked before looking into anything else.

I didn't run the FEP, but I guess you're right in that it makes sense to check it works first. I'll do this asap.

or try out the code in branch emin-maxthreads (PR #128)

I think you made a slight error in the code, when I remove the comma at the end of the line it works.

Can you open a new issue for your own data so that we can deal with different issues in self contained blocks, please?

I will do this yes, my apologies.

[Ntr9999]

did you manage to run the benzene example?

It worked and gave me reasonable numerical values, but outputted errors at the end.

For the last step, mdpow-pow benzene, I got the values and output files (energies.txt, pow.txt and DeltaA.png - is there only supposed to be DeltaA.png for the octanol step? Is there a way I can get it for both?), but there was some erroring afterwards referring to Matlab.

Output:

mdpow.fep   : INFO     [BNZ] Octanol solvation free energy (Gibbs) -20.0205 (2.40) kJ/mol
mdpow.fep   : INFO     [BNZ] Octanol solvation free energy (coulomb) 1.52674 (0.27) kJ/mol
mdpow.fep   : INFO     [BNZ] Octanol solvation free energy (vdw) 18.4938 (2.38) kJ/mol
mdpow.fep   : INFO     [BNZ] Values at T = 300 K
mdpow.fep   : INFO     [BNZ] Free energy of transfer water --> octanol: -12.260 (3.707) kJ/mol
mdpow.fep   : INFO     [BNZ] log P_ow: 2.135 (0.645)
mdpow       : INFO     Wrote results to '/Users/mark/Desktop/benzene/pow.txt'
mdpow       : INFO     Wrote solvation energy terms to '/Users/mark/Desktop/benzene/energies.txt'
/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-pow:131: UserWarning: 
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.

The backend was *originally* set to 'MacOSX' by the following code:
  File "/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-pow", line 39, in <module>
    import mdpow.fep
  File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/mdpow/fep.py", line 137, in <module>
    import gromacs, gromacs.utilities
  File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/gromacs/__init__.py", line 183, in <module>
    from . import fileformats
  File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/gromacs/fileformats/__init__.py", line 10, in <module>
    from .xvg import XVG
  File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/gromacs/fileformats/xvg.py", line 203, in <module>
    from matplotlib import pyplot as plt
  File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/matplotlib/pyplot.py", line 71, in <module>
    from matplotlib.backends import pylab_setup
  File "/Users/mark/opt/anaconda3/envs/mdpow/lib/python2.7/site-packages/matplotlib/backends/__init__.py", line 17, in <module>
    line for line in traceback.format_stack()


  matplotlib.use('agg')  # quick non-interactive plotting
mdpow       : CRITICAL Running analysis in directory 'benzene' failed
mdpow       : ERROR    Catastrophic problem occurred, see the stack trace for hints.
Traceback (most recent call last):
  File "/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-pow", line 233, in <module>
    force=opts.force, stride=opts.stride, permissive=opts.permissive)
  File "/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-pow", line 136, in run_pow
    activate_subplot(1)
  File "/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-pow", line 163, in activate_subplot
    return axes(gcf().get_axes()[numPlot])
IndexError: list index out of range
Traceback (most recent call last):
  File "/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-pow", line 233, in <module>
    force=opts.force, stride=opts.stride, permissive=opts.permissive)
  File "/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-pow", line 136, in run_pow
    activate_subplot(1)
  File "/Users/mark/opt/anaconda3/envs/mdpow/bin/mdpow-pow", line 163, in activate_subplot
    return axes(gcf().get_axes()[numPlot])
IndexError: list index out of range

mdpow.log

Thanks again for the assistance.

[orbeckst]

Overall it looks as if the simulation aspects are working but the analysis stage is broken. Admittedly, I don't think that we have been using the analysis scripts a lot lately (@iorga @VOD555 ??); I think we mostly wrote code based on alchemlyb. Our recent paper

S. Fan, B. I. Iorga, and O. Beckstein, “Prediction of octanol-water partition coefficients for the SAMPL6- log P molecules using molecular dynamics simulations with OPLS-AA, AMBER and CHARMM force fields,” Journal of Computer-Aided Molecular Design, vol. 34, pp. 543–560, 2020. doi: 10.1007/s10822-019-00267-z

has more details (and input files at https://github.com/Becksteinlab/SAMPL6_logP_data).

DeltaA should contain plots for water and octanol... IIRC.

I don't think that the backend error is responsible for the crash... but would need to look into it (and it shouldn't happen).

If you raise a new issue for this particular problem then hopefully someone can get to it. Include as much information to reproduce — the more we can copy and paste instructions the more likely someone will attempt to reproduce it.

[orbeckst]

Thank you again for any assistance you provide; I appreciate that these questions may seem very trivial an experienced user.

None of the questions are trivial — it's just that MDPOW is not very polished and it's hard to keep it up-to-date. When we use it "it just works™" (most of the time...). We definitely would like others to be able to use it so thank you for working with us.

[Ntr9999]

I have created two new issues, in order to keep things compartmentalised.
#129 #130
This issue, which was originally regarding installation, can now be closed.