NeatSim

This is the README file accompanying the code and data used in the following paper, which I'll assume you have read.

Jonathan Merlevede, Rinde van Lon and Tom Holvoet, “Neuroevolution of a Multi-Agent System for the Dynamic Pickup and Delivery Problem”. In: International Joint Workshop on Optimisation in Multi-Agent Systems and Distributed Constraint Reasoning (co-located with AAMAS), Paris, France – May 2014.

Before reading on, you should know that this code was written in the context of this paper. It was never really meant to be used by anyone but the authors while coding: documentation is scarce, the code not always elegant. Nevertheless, I would often have preferred having ugly source code over having none, so here it is (and then there is also reproducibility...).

To any researcher who considers building on my work, I would advise to start with a fresh pull of SharpNEAT and maybe look at the code here for reference. To a large extent this also applies to my other code. The Java part currently uses an outdated version of RinSim; newer versions of RinSim are not perfectly backwards compatible with this version.

Overview

As described in the paper, the software consists of a scenario generator and two programs that communicate over sockets using Thrift.

• The first program is the C# part, based on SharpNeatV2, which implements the NEAT algorithm.
• The second program is written in Java and is called upon by NEAT to determine the fitness of individuals during evolution. Determining fitness involves carrying out a simulation in RinSim).

The Java program can also be used for evaluating and visualising heuristics that have already been evolved.

Concretely, this archive consists of the following elements.

• Generator: the scenario generator
• NeatSim: the Java part of the application
• SharpNeatV2: the C# part of the application
• thrift: the thrift service file

Quickstart

It is advised to run the SharpNeat part of the application on Windows, but it should also be possible to run SharpNeat using Mono.

Scenario generation

The scenario generator is implemented in Octave/Matlab. The scenarios in the paper were generated using Octave, and are included in this ZIP file. To generate scenarios, run generateScenarios.m. This file is mostly a collection of parameters - adapt the file to your needs (number of scenarios, etc.). In the paper we used the parameters currently in the file.

Evolution

In order to evolve a new heuristic, the two parts of our software have to be up and running.

• Start the NeatSim server - the main class is called Main. Make sure that NeatSim is configured properly - you will also have to download the required dependencies using Maven (see neatsim.properties and the description of NeatSim below).
• Start the SharpNEATv2 GUI (SharpNeatV2\bin\SharpNeatGUI.exe). Select the 'NeatSim - Simulation' experiment, the desired evolutionary properties and then press the start button.

In order to do any serious evolution, you will probably want to use distributed evaluation using JPPF. See the 'NeatSim' section below for (a little) more information.

Evaluation

Existing genomes (as stored in XML files by SharpNEAT) can be read, evaluated and visualized by the NeatSim code (which uses RinSim). For an example of reading an XML file, take a look at the test class TestNeuralNetworkReader. For an example of how to start a simulation, take a look at the test class TestDeterminism.

Code

The code is organized in three folders: NeatSim, SharpNeatV2 and Thrift.

Scenario generator

• Scenarios are actually generated by createScenarios.
• reverseA was used to reverse-engineer the activity matrix used by Gendreau.
• existing contains the scenario files used by Gendreau.
• Most of other files are either junk, helper files or were created for verifying that the generator and existing files by Gendreau are similar (they mostly draw plots).

NeatSim

This folder contains the Java portion of the code, which can perform simulations to assign fitness values to heuristics (individuals). Most settings can be found in neatsim.properties, which is fairly well documented. The following points are probably the most important to know. Dependencies are managed by Apache Maven. Because some of the software parts that we used were not part of a (stable) Maven release, the project directory includes a Maven project repository.

• NeatSim can run in two modes: evaluation mode and server mode. In evaluation mode, NeatSim reads genomes in the form of XML files from a location on the disk specifqied by a directory, prefix and suffix in the config file and evaluates them. In server mode, NeatSim listens for incoming evaluation requests on a socket using Thrift.
• NeatSim supports three evaluation modes. It can evaluate genomes locally in a single thread, locally in multiple threads or it can distribute simulation tasks over a grid using the JPPF grid computing framework. This requires a working JPPF network and correct settings in both neatsim.properties and the JPPF configuration file in config/jppf.properties. Using grid computing is often necessary, because simulations take approximately one second and evolution generally requires a large number (in the order of a million) number of fitness evaluations.

When trying to understand the code, the GendreauEvaluator is probably a good class to start in. It can also be useful to look at the translation of the configuration file into code.

Note: One important thing to note when using JPPF is that the server and the client (NeatSim) have to use the same version. To see or change which version NeatSim uses, look in the pom.xml file.

SharpNeatV2

This folder contains the modified version of [SharpNEAT v2] that I used for evolving the vehicle heuristics. I really only added code for an evaluator that sends the genomes of an entire generation to Java for evaluation (using Thrift), but had to change some parts of SharpNeat that might make it hard to easily upgrade to newer versions of SharpNEAT (e.g. the possibility to evaluate an entire generation instead of just a single genome, but also changing the visibility of some properties of the neural networks in order to read them into the Thrift data transfer objects). I also added my experiment to the SharpNEATV2 configuration file SharpNeatDomains.experiments.xml. Most of my code is in the SharpNeat\NeatSim application.

The hostname and the port of the Java Thrift 'evaluation server' are currently hardcoded to localhost and 7913 in the ProtocolManager.

Thrift

This folder contains the Thrift service file that I used to generate the Java and C# interface and code for interprocess communication. It also includes the generated code and the (Windows) binary Thrift file that I used for generation. Using a newer version of Thrift or Thrift for Mac from the Thrift website should not be a problem.

-- Jonathan Merlevede