Skip to content
This repository has been archived by the owner on Jul 16, 2024. It is now read-only.

[TUTORIAL] Code a problem in Cpp, use it in Python

Jump to bottom
Dario Izzo edited this page Mar 24, 2015 · 1 revision

Introduction

Using the C++ API (PaGMO) to code and optimisation problem is recommended for speed. But it is also nice to be able to use the python syntax to script the problem solution. In this tutorial we will learn to code a problem in C++ to then expose it into a python module my_module. At the end, we will be able to write:

from PyGMO import *
import my_module
prob = my_module.my_problem()
algo = algorithm.jde()
pop = population(prob,20)
pop = algo.evolve(pop)

We will start showing how to define new problems using the C++ API. This is done by deriving from the problem::base class. The procedure is similar to how new problems are defined in Python (PyGMO) except that now we will have to manually compile the source files. Once they have been compiled, they can either be used in other C++ code or they can be exposed in Python (as wrappers). Familiarity with Python extensions is not required. This tutorial will provide all the necessary steps. If however the reader is interested in learning more about exposing C/C++ functions in Python, documentation can be found here https://docs.python.org/2.7/extending/extending.html#extending-python-with-c-or-c

Furthermore, this tutorial assumes you are already familiar with defining new PyGMO problems in Python.

Part 1 - Creating new PaGMO Problems

First step: subclass pagmo::problem::base

Create a new header file called "prob_A.h" and add

#ifndef PAGMO_PROBLEM_PROB_A_H
#define PAGMO_PROBLEM_PROB_A_H

#include <string>

#include <pagmo/src/config.h>
#include <pagmo/src/serialization.h>
#include <pagmo/src/types.h>
#include <pagmo/src/problem/base.h>

namespace pagmo{ namespace problem {

/// A random problem
/**
 * @author Dario Izzo ([email protected])
 */
class __PAGMO_VISIBLE prob_A: public base
{
    public:
        prob_A(unsigned int dim=5, double data = 0.5);

        base_ptr clone() const;
        std::string get_name() const;

        std::string a_method() const;
        void set_member(const double);
        const double& get_member() const;
    protected:
        void objfun_impl(fitness_vector &, const decision_vector &) const;
        std::string human_readable_extra() const;

    private:
        friend class boost::serialization::access;
        template <class Archive>
        void serialize(Archive &ar, const unsigned int)
        {
            ar & boost::serialization::base_object<base>(*this);
            ar & m_member;
        }
        double m_member;
};

}} // namespaces

BOOST_CLASS_EXPORT_KEY(pagmo::problem::prob_A)
#endif // PAGMO_PROBLEM_PROB_A_H

The header file is useful as it defines an interface to our problem (prob_A). It can then be included in other C++ projects to provide a way for them to access the newly defined problem.

Please Note: it is very important to have the serialize method. See https://github.com/esa/pagmo/wiki/Developing-guide#howto-program-new-problems-into-pagmo-c the section about serialization.

Second step: define the actual methods

Create a new source file called prob_A.cpp:

#include <string>
#include <vector>
#include <numeric>
#include <cmath>

#include "prob_A.h"

namespace pagmo { namespace problem {


/// Constructor
/**
 * Constructs the problem
 *
 * @param[in] seq std::vector of kep_toolbox::planet_ptr containing the encounter sequence for the trajectoty (including the initial planet)
 * @param[in] t0_l kep_toolbox::epoch representing the lower bound for the launch epoch
 */
prob_A::prob_A(unsigned int dim, double data): base(dim), m_member(data)
{
    // Construct here the problem (bounds etc.)
}


/// Clone method.
base_ptr prob_A::clone() const
{
    return base_ptr(new prob_A(*this));
}

/// Implementation of the objective function.
void prob_A::objfun_impl(fitness_vector &f, const decision_vector &x) const
{
    f[0] = 0.0;
    for (decision_vector::size_type i = 0; i < x.size(); ++i) {
        f[0] += x[i];
    }
}

/// Gets the data member
/**
 * @return[out] data member
 */
const double& prob_A::get_member() const {
    return m_member;
}

/// Sets the data member
/**
 *
 * @param[in] value new value
 */
void prob_A::set_member(double value) {
    m_member = value;
}

std::string prob_A::a_method() const {
    return "I do null";
}


std::string prob_A::get_name() const {
    return "Prob A";
}


/// Extra human readable info for the problem.
/**
 * Will return a formatted string containing the values vector, the weights vectors and the max weight. It is concatenated
 * with the base::problem human_readable
 */
std::string prob_A::human_readable_extra() const
{
    std::ostringstream oss;
    oss << "\n\tData member value: " << m_member;
    return oss.str();
}

}} //namespaces

BOOST_CLASS_EXPORT_IMPLEMENT(pagmo::problem::prob_A)

Third step: create main function

We will instantiate the newly created problem and solve it using jDE:

#include<pagmo/src/population.h>
#include<pagmo/src/algorithm/jde.h>
#include<iomanip>

#include"prob_A.h"

int main() {
    pagmo::problem::prob_A prob(5);
    pagmo::population pop(prob,50);
    pagmo::algorithm::jde algo(500);
    algo.set_screen_output(true);
    algo.evolve(pop);
    std::cout << "Done .. the champion is ... " <<std::endl;
    std::cout << pop.champion().x << std::endl;
    return 0;
}

Finally, we compile our code:

g++ main.cpp prob_A.cpp -o main -std=c++11 -pthread -DNDEBUG -lpagmo -lboost_thread -lboost_serialization -lboost_system

Note that this assumes you have pagmo and boost libraries already installed. A guide to compiling and instaling PaGMO can be found here https://github.com/esa/pagmo/wiki/Guide-to-Compilation

Hurrah! Now you can simply type ./main to run the program.

Side note: if g++ complains about not being able to find pagmo/src/population.h then make sure you have installed PaGMO correctly. You can try adding the -I and -L flags to g++ to manually specify header and library search paths. Furthermore, you might then have to use LD_LIBRARY_PATH when running ./main to help it find libpagmo.so.

Part 2 - Creating Python wrappers

Sometimes you might want to interface with your problems from Python. To do that, we need to recompile prob_A.cpp to obtain Position Independent Code (https://en.wikipedia.org/wiki/Position-independent_code). We will then need to create a C++ Python wrapper for it. Finally we will link the two forming a shared library. This will behave as an ordinary Python module which we can then import.

First step: Create PIC object from prob_A.cpp

g++ -c -fPIC prob_A.cpp  -std=c++11 -pthread -DNDEBUG

Second step: Create Boost Python module

Create a new file called "my_pygmo_probs.cpp". This will contain a description on how to execute our prob_A from Python. The preferred way of doing this is using the Boost library (http://www.boost.org/doc/libs/1_57_0/libs/python/doc/tutorial/doc/html/python/exposing.html), this is not very different though from using CPython.

#include <Python.h>
#include <boost/python/copy_const_reference.hpp>
#include <boost/python/module.hpp>

#include <pagmo/PyGMO/utils.h>

#include "prob_A.h"
    
using namespace boost::python;
using namespace pagmo;
    
// Wrapper to expose problems.
template <class Problem>
static inline class_<Problem,bases<problem::base> > problem_wrapper(const char *name, const char *descr)
{   
    class_<Problem,bases<problem::base> > retval(name,descr,init<const Problem &>());
    retval.def(init<>());
    retval.def("__copy__", &Py_copy_from_ctor<Problem>);
    retval.def("__deepcopy__", &Py_deepcopy_from_ctor<Problem>);
    retval.def_pickle(generic_pickle_suite<Problem>());
    retval.def("cpp_loads", &py_cpp_loads<Problem>);
    retval.def("cpp_dumps", &py_cpp_dumps<Problem>);
    return retval;
}

BOOST_PYTHON_MODULE(my_pygmo_probs) {
    common_module_init();

    problem_wrapper<problem::prob_A>("prob_A", "A first added problem (does nothing)")
        .def(init< optional<unsigned int, double> >())
        .def("a_method", &problem::prob_A::a_method)
        .add_property("an_attribute",make_function(&problem::prob_A::get_member, return_value_policy<copy_const_reference>()), &problem::prob_A::set_member,"getter and setters exposed as attribute");
}

Compile it with:

g++ -c -fPIC my_pygmo_probs.cpp -I/usr/include/python2.7  -std=c++11 -pthread -DNDEBUG

Third step: create the shared library

g++ -shared -o my_pygmo_probs.so my_pygmo_probs.o prob_A.o  -std=c++11 -pthread -DNDEBUG -lboost_python-mt -lpython2.7 -lboost_serialization-mt -lpagmo

Side note: if ld complains that it cannot find -lboost_python-mt, then try using -lboost_python-2.7-mt instead:

g++ -shared -o my_pygmo_probs.so my_pygmo_probs.o prob_A.o  -std=c++11 -pthread -DNDEBUG -lboost_python-2.7-mt -lpython2.7 -lboost_serialization-mt -lpagmo

Fourth step: Importing it in Python

Fire up the Python interpreter in the current directory (where you have compiled my_pygmo_probs.so). Type in the following:

import PyGMO
import my_pygmo_probs

Congratulations! Now you can access prob_A via my_pygmo_probs.prob_A(). Try it out:

my_pygmo_probs.prob_A()
my_pygmo_probs.prob_A().a_method()

Note the difference: my_pygmo_probs.prob_A is the class while my_pygmo_probs.prob_A() is the actual instance. Normally you will want to call methods from the instance and not the class.

Remember to import PyGMO first!

If you want to import my_pygmo_probs from anywhere, you will probably need to either copy it in a system-wide folder (one of the folders in sys.path), or change the PYTHONPATH variable. More information on both methods can be found here: https://docs.python.org/2/tutorial/modules.html#the-module-search-path

This concludes the PaGMO C++ Tutorial. You should now be able to define new problems using the PaGMO C++ API, compile them, use them in other C++ projects or import them into Python and use them in other Python projects.

Clone this wiki locally