This document builds up on our “Get Started” guide and our user guide, and introduces lincs’ Python API.
This API is more flexible, albeit more complex, than the command-line interface you’ve been using so far.
The learning set is pre-processed in the LearningData so that learning algorithms don’t have to manipulate the different type of criterion values. In the LearningData, we keep only the ranks of the performances of each alternative in the learning set. The learning set is also destructured into a few arrays. Here are the attributes that describe this pre-processed learning set:
+
The number of distinct values actually seen for each criterion (including the min and max values for numerical criteria):
+
list(learning_data.values_counts)# Indexed by [criterion_index]
+
We see that the learning data knows 1002 values for the real-valued criterion. This is usual as it’s rare that two floating point values are exactly equal, so the 1000 alternatives have distinct values, and the min and max are two more values. The learning data contains 101 values for the integer-valued criterion, meaning that the alternatives in the learning set do actually cover the whole set of possible values. And similarly, 6 values for the enumerated criterion.
+
For each criterion, the ranks of the performance of each alternative:
+
[list(v)[:10]+['...']forvinlearning_data.performance_ranks]# Indexed by [criterion_index][alternative_index]
+
This lets heuristic strategies operate in parallel on models and still produce deterministic results. URBGs are callable to get the next pseudo-random integer:
WPB learning is iterative, and the iteration_index is stored in the learning data. It starts at zero and tells you the current iteration:
+
learning_data.iteration_index
+
+
+
0
+
+
+
The remaining attributes are modified at each iteration, and start uninitialized. For this presentation, we’ll first run one iteration of WPB so that their values make sense.
Its model_indexes contains indexes of models in increasing order of accuracy.
+
list(learning_data.model_indexes)
+
+
+
[1, 5, 4, 8, 3, 6, 2, 0, 7]
+
+
+
Its accuracies attribute holds the accuracy of each model. They are stored as the count of correctly-classified alternatives, between 0 and alternatives_count included:
+
list(learning_data.accuracies)# Indexed by model_index
+
+
+
[728, 593, 725, 671, 637, 609, 676, 833, 640]
+
+
+
If you iterate the accuracies in the order of model_indexes, they are sorted:
With this better understanding of LearningData, let’s write our own Observer strategy. It’s arguably the simplest to starts with, because it’s not expected to change the LearningData.
+
To start as simple as possible, lets reproduce the behavior of the --...-verbose flag on the command line, by creating an observer that just prints the best accuracy at each step.
+
Observer strategies must define two methods to be called by the learning algorithm: after_iteration, to be called at the end of each iteration, after the breeding of models is done, and before_return, to be called just before the final model is returned.
Now let’s do something slightly more complicated: our goal for IntermediatesObserver is to keep track of the best model so far at different times during the learning. Specifically, we want to keep the models at iterations 1, 2, 4, 8, etc..