Native Support from JuMP in Julia

[ccoffrin]

In Julia we can currently send MINLPs to MINOTAUR via AMPL NL files. However, there can be advantages in accessing MINOTAUR directly as an NLP solver in Julia, for example is using JuMP's auto-diff instead of AMPL's.

You might consider making MINOTAUR available as a JuliaOpt package. This is the channel for solver implementation discussions, https://gitter.im/JuliaOpt/JuMP-dev.

[odow]

Julia now has a binary build system called Yggdrasil (https://github.com/JuliaPackaging/Yggdrasil), which cross compiles open-source binaries for a wide variety of platforms.

I've been attacking MINLP solvers recently, so now you use something like Bonmin as:

import Pkg; Pkg.add("Bonmin_jll")
using JuMP, Bonmin_jll, AmplNLWriter
model = Model(() -> AmplNLWriter.Optimizer(Bonmin_jll.amplexe))

This will look at the users computer and download the appropriate cross-compiled library. It'd be nice to have the same for minotaur.

I hit a few blockers though:

• Is master safe to build releases off? We can build off a git commit, so we don't necessarily need a tag/branch, but a proper release would be nice. Related issue: Next Release #23
• Is it possible to compile on Mac/Windows? Related issue: Windows binaries #14.
• It seems like the build_third_party downloads binaries like FilterSQP from your website. Does minotaur work without them?
• We already build ASL, Cbc, Ipopt etc, so it would be good to re-use them, rather than compiling fresh versions. This shouldn't be too hard, other than some modifications to the CMAKE files.

[ashutoshmahajan]

Hi Oscar
Sorry for the lack of progress in making releases. Having similar functionality for Minotaur would be great. We are planning the next release in about three weeks. If you would like to have something earlier, you may use the master. It will be as safe as the release.

Building on windows is possible. Some instructions for msys2 here: https://minotaur-solver.github.io/

Minotaur works without Filter and BQPD (both are fortran libraries whose source can not be made open). -f flag of build_third_party will suppress them.

Right now we have a rather simple CMAKE file that just checks for presence of lib and header files. There is a way to specify the directories of each third-party lib to CMAKE in place of the default third-party directory. e.g. -DIPOPT_INC_DIR:PATH=/path/to/ipopt/include -DIPOPT_LIB_DIR:PATH=/path/to/ipopt/lib, -DASL_INC_DIR:PATH, -DASL_LIB_DIR:PATH, etc. I can write all the vars if you want.

[odow]

DIPOPT_INC_DIR:PATH=/path/to/ipopt/include -DIPOPT_LIB_DIR:PATH=/path/to/ipopt/lib, -DASL_INC_DIR:PATH, -DASL_LIB_DIR:PATH

Perfect! This will do nicely. So what is the minimal set of dependencies that we would need? Just ASL, Cbc, Ipopt?

Building on windows is possible

I guess I meant cross-compiling for windows from linux :). I'll have a go and see if I run into problems.

[ashutoshmahajan]

The following seems to work on my computer. You may remove CPPUNIT lines if you do not run tests. I pushed some changes to CMake today. You may want to pull them before trying this.

cmake -DCPPUNIT_INC_DIR:PATH=/home/amahajan/minotaur/tp-2/include/
-DCPPUNIT_LIB_DIR:PATH=/home/amahajan/minotaur/tp-2/lib
-DASL_INC_DIR:PATH=/home/amahajan/minotaur/tp-2/include/asl
-DASL_LIB_DIR:PATH=/home/amahajan/minotaur/tp-2/lib
-DCBC_INC_DIR:PATH=/home/amahajan/minotaur/tp-2/include
-DCBC_LIB_DIR:PATH=/home/amahajan/minotaur/tp-2/lib
-DOSI_INC_DIR:PATH=/home/amahajan/minotaur/tp-2/include
-DOSI_LIB_DIR:PATH=/home/amahajan/minotaur/tp-2/lib
-DOSICLP:BOOL=ON
-DIPOPT_INC_DIR:PATH=/home/amahajan/minotaur/tp-2/include
-DIPOPT_LIB_DIR:PATH=/home/amahajan/minotaur/tp-2/lib
-DBUILD_SHARED_LIBS:BOOL=1 /home/amahajan/minotaur

[odow]

Awesome! I'll take a look.

How stable is minotaur with Cbc/Ipopt compared to the other solvers? Is it possible to link CPLEX at runtime? Or does it have to be compiled?

[ashutoshmahajan]

We don't use Cbc. It is there for some experimental code, and not really used in any solver that gets compiled by default. One can leave it out safely. The main workhorse is CLP for solving LPs. Ipopt is used extensively in Nonlinear Branch-and-Bound and also to a significant extent in other algorithms.

We do not regularly compare to other solvers, so can not comment much on it. We have been trying to fix as many numerical issues arising from our own routines and also from calls to other solvers, and have made significant improvements in the global solver for QCQPs recently. Feedback is this regard (and also others) is most welcome.

Linking with CPLEX is possible. More on this shortly.

[meenarli]

To link to CPLEX, one can use the options -DCPX_INC_DIR= and -DCPX_LIB_DIR= with cmake. For example, we use -DCPX_INC_DIR=/opt/ibm/ILOG/CPLEX_Studio128/cplex/include and
-DCPX_LIB_DIR=/opt/ibm/ILOG/CPLEX_Studio128/cplex/lib/x86-64_linux/static_pic.

CPLEX can be used both for solving LPs and MILPs using Minotaur options --lp_engine Cplex and --milp_engine Cplex, respectively.

[odow]

Thanks for this info! I'll try an open-source version for now and leave CPLEX out of it.

[odow]

I'm chipping away at this again.

My current build script looks like:

cd $WORKSPACE/srcdir/minotaur

# Minotaur is hard-coded to look in IPOPT_INC_DIR/coin, but the path we have
# from Ipopt_jll is {$includedir}/coin-or.
cp ${includedir}/coin-or/* ${includedir}/coin
# Minotaur assumes the ASL library is at amplsolver.a
cmd="s/amplsolver.a/libasl.${dlext}/g"
sed  -i $cmd CMakeLists.txt

mkdir -p build
cd build
cmake \
    -DCMAKE_INSTALL_PREFIX=${prefix} \
    -DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
    -DCMAKE_BUILD_TYPE=Release \
    -DASL_INC_DIR:PATH=${includedir} \
    -DASL_LIB_DIR:PATH=${libdir} \
    -DCLP_INC_DIR:PATH=${includedir} \
    -DCLP_LIB_DIR:PATH=${libdir} \
    -DOSI_INC_DIR:PATH=${includedir} \
    -DOSI_LIB_DIR:PATH=${libdir} \
    -DOSICLP:BOOL=ON \
    -DIPOPT_INC_DIR:PATH=${includedir} \
    -DIPOPT_LIB_DIR:PATH=${libdir} \
    -DBUILD_SHARED_LIBS:BOOL=1 \
    -DUSE_OpenMP::BOOL=OFF \
    ..
make -j${nproc}
make install

But I get errors like

12:41:50] [ 40%] Building CXX object src/CMakeFiles/minotaur.dir/base/OAHandler.cpp.o
[12:41:50] cd /workspace/srcdir/minotaur/build/src && /opt/bin/aarch64-apple-darwin20-libgfortran5-cxx11/aarch64-apple-darwin20-clang++ --sysroot=/opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root  -DCOIN_BIG_INDEX=0 -DDEBUG=0 -DMNTROSICLP=1 -DMNTROSIGRB=0 -DNDEBUG -DSPEW=0 -DUSE_IPOPT -DUSE_MINOTAUR_AMPL_INTERFACE -DUSE_OPENMP=0 -DUSE_OSILP -Dminotaur_EXPORTS -I/workspace/srcdir/minotaur/build/src/base -I/workspace/srcdir/minotaur/src/base -I/workspace/srcdir/minotaur/src/interfaces  -Wall -pedantic -Wmissing-include-dirs -Wunused -Wextra -Wundef -Wshadow -Wredundant-decls -Woverloaded-virtual -O3 -O3 -DNDEBUG -fPIC   -o CMakeFiles/minotaur.dir/base/OAHandler.cpp.o -c /workspace/srcdir/minotaur/src/base/OAHandler.cpp
[12:41:50] /workspace/srcdir/minotaur/src/base/OAHandler.cpp:16:2: error: "Cannot compile parallel algorithms: turn USE_OpenMP flag ON."
[12:41:50] #error "Cannot compile parallel algorithms: turn USE_OpenMP flag ON."
[12:41:50]  ^
[12:41:50] [ 41%] Building CXX object src/CMakeFiles/minotaur.dir/base/Objective.cpp.o
[12:41:50] cd /workspace/srcdir/minotaur/build/src && /opt/bin/aarch64-apple-darwin20-libgfortran5-cxx11/aarch64-apple-darwin20-clang++ --sysroot=/opt/aarch64-apple-darwin20/aarch64-apple-darwin20/sys-root  -DCOIN_BIG_INDEX=0 -DDEBUG=0 -DMNTROSICLP=1 -DMNTROSIGRB=0 -DNDEBUG -DSPEW=0 -DUSE_IPOPT -DUSE_MINOTAUR_AMPL_INTERFACE -DUSE_OPENMP=0 -DUSE_OSILP -Dminotaur_EXPORTS -I/workspace/srcdir/minotaur/build/src/base -I/workspace/srcdir/minotaur/src/base -I/workspace/srcdir/minotaur/src/interfaces  -Wall -pedantic -Wmissing-include-dirs -Wunused -Wextra -Wundef -Wshadow -Wredundant-decls -Woverloaded-virtual -O3 -O3 -DNDEBUG -fPIC   -o CMakeFiles/minotaur.dir/base/Objective.cpp.o -c /workspace/srcdir/minotaur/src/base/Objective.cpp
[12:41:51] /workspace/srcdir/minotaur/src/base/OAHandler.cpp:122:37: error: use of undeclared identifier 'omp_get_thread_num'
[12:41:51]         sstm << "_OACutRoot_Th_" << omp_get_thread_num() << "_" << stats_->cuts;
[12:41:51]                                     ^
[12:41:51] /workspace/srcdir/minotaur/src/base/OAHandler.cpp:144:35: error: use of undeclared identifier 'omp_get_thread_num'
[12:41:51]       sstm << "_OAObjRoot_Th_" << omp_get_thread_num() << "_" << stats_->cuts;
[12:41:51]                                   ^
[12:41:51] /workspace/srcdir/minotaur/src/base/OAHandler.cpp:267:37: error: use of undeclared identifier 'omp_get_thread_num'
[12:41:51]             sstm << "_OACut_Th_" << omp_get_thread_num() << "_" << stats_->cuts;
[12:41:51]                                     ^
[12:41:51] /workspace/srcdir/minotaur/src/base/OAHandler.cpp:303:40: error: use of undeclared identifier 'omp_get_thread_num'
[12:41:51]             sstm << "_OAObjCut_Th_" << omp_get_thread_num() << "_" << stats_->cuts;
[12:41:51]                                        ^
[12:41:51] /workspace/srcdir/minotaur/src/base/OAHandler.cpp:592:33: error: use of undeclared identifier 'omp_get_thread_num'
[12:41:51]         sstm << "_OACut_Th_" << omp_get_thread_num() << "_" << stats_->cuts;
[12:41:51]                                 ^
[12:41:51] [ 41%] Building CXX object src/CMakeFiles/minotaur.dir/base/Operations.cpp.o

Is it possible to compile without OpenMP? I have -DUSE_OpenMP::BOOL=OFF and -DUSE_OPENMP=0