Skip to content
This repository has been archived by the owner on Jul 16, 2024. It is now read-only.

GSoC 2015 BBOB

Jump to bottom
Krzysztof Nowak edited this page May 15, 2015 · 5 revisions

Project Description

Performance evaluation metrics for evolutionary algorithms are crucial component for parameter tuning, testing new algorithm and problems, as well as characterizing and explaining the properties and behavior of algorithms by statistical runtime comparisons. One such performance evaluation testbed is the Real-Parameter Black-Box Optimization Benchmark (BBOB). The aim of this project is adding a new module to pagmo::util containing several performance evaluation methods from BBOB 2015 experimental setup (see [1] and [2]). The completed module will allow the users to run vast benchmarking campaigns, and analyze the performance of algorithms and initial parameters over a large sets of problems. Student is expected to get familiar with the performance evaluation methods of evolutionary algorithms (with the help of the mentor)

[1] Real-Parameter Black-Box Optimization Benchmarking: Experimental Setup. PDF with explanation and source code

[2] Nikolaus Hansen and Anne Auger. "Performance evaluation of an advanced local search evolutionary algorithm". In In Proceedings of the IEEE Congress on Evolutionary Computation. IEEE Press, 2005 ONLINE

Implementation Details:

The student will extend the pagmo::util module with the pagmo::util::bbob module (as well as interface it to PyGMO's PyGMO.util.bbob). Where possible, the existing reference code should be reused and wrapped as C++ classes. Second part of the project will be dedicated to enriching PyGMO with results visualization (using matplotlib) and writing interface for a more extensive experimental setups (e.g. large benchmarking campaigns over many problems and algorithms).

An example for individual algorithm comparison:

from PyGMO import *
from PyGMO.util import bbob

ert = bbob.compute_ert(algorithm.jde(variant=2, gen=20), problem.rosenbrock(10), n_runs=1000)  # Compute the expected running time data
bbob.plot_cumulative_distribution(ert)  # Presents the results on a cumulative distribution plot

Required programming skills: Python , C++

Skills that help: Knowledge of statistical testing, matplotlib and data presentation

Mentors: Mentors#Krzysztof, Mentors#Dario

Timeline

This is just a rough sketch, it will change and get more detailed over time.

  1. ( - 25.05) Wrapping bbob benchmark problems as pagmo::problem
  2. (25.05 - 30.06) BBOB package should be wrapped by this time (problems and metrics) and interfaced to Python and documented (tutorials, sphinx documentation). Probably a merge with the master.
  3. (30.06 - 10.08) Implementing genetic algorithms from the proposal: BOA, BAT, Cuckoo, Tabu search (order to be determined). All the algorithms should have been implemented and documented by this time.
  4. (10.08 - 17.08) Extra week for Examples on how to benchmark PaGMO algorithms using BBOB.
  5. (18.08 - 21.08) Final polishing and merging of codebase.

Algorithms

BAT Algorithm and its Multi-objective variant (own implementation):

Matlab implementation by the author available here:

Multi-objective variant would be done using decomposition.

Cuckoo Search and its Multi-Objective (own implementation):

Matlab implementation by the author available:

Multi-objective variant would be done using decomposition (TODO: paper reference ?)

Tabu Search (to be wrapped):

Bayesian Optimization Algorithm (wrapped):

Clone this wiki locally