Skip to content

0.17.0
Compare
Choose a tag to compare
@douglasrizzo douglasrizzo released this 27 Jun 14:31
· 50 commits to master since this release
0.17.0
02a11ec

Estimation overhaul

This version presents a major overhaul in the ability estimation module. I decided to revisit the theory behind finding extrema of unimodal functions without the use of derivatives (which is the case of the log-likelihood used to estimate the most likely abilities for examinees during a CAT, given their response patterns) and bumped into this amazing playlist by @osveliz.

New estimator implementations

From that playlist, I have implemented the following methods for finding the maxima of the log-likelihood function:

New estimators from SciPy

I also found out that all minimization methods implemented in scipy.optimize.minimize_scalar work really well in catsim, so I have also made those available in the estimation module.

Better results

These changes have introduced a small but noticeable speedup in simulation times, since many of the new methods tend to evaluate the log-likelihood function fewer times than the older estimators present in catsim. In particular, all of the new methods are more theoretically sound than the HillClimbingEstimator as well as 5 to 10 times faster than the DifferentialEvolutionEstimator.

Breaking changes

Due to that, I have decided to retire the DifferentialEvolutionEstimator and joined all estimators (including the HillClimbingEstimator) into a new NumericalSearchEstimator class.

Instantiating the new estimators

For the old hill-clibing estimation method:

estimator = NumericalSearchEstimator("hillclimbing")

For the four new estimators implemented into catsim::

estimator = NumericalSearchEstimator("ternary")
estimator = NumericalSearchEstimator("dichotomous")
estimator = NumericalSearchEstimator("fibonacci")
estimator = NumericalSearchEstimator("golden2")

For the three estimators from scipy.optimize.minimize_scalar():

estimator = NumericalSearchEstimator("brent")
estimator = NumericalSearchEstimator("bounded")
estimator = NumericalSearchEstimator("golden")

Closing remarks

If everything goes right with the new GitHub Actions I've implemented, the documentation will be updated shortly. The examples in the README as well as the documentation website will have all information regarding the new estimators. After the documentation update, you find a comparison among the new estimation methods here.

Assets 2
Loading