Strange score distribution in the paper.

[erow]

This is Figure 7 MIG distribution on dSprites. The MIG score should be non-negative, and the scores are low. I analyze the pre-trained models and get the following results.

I don't know why they are so different, and the other metrics are different too.

[Near32]

Hello,
The MIG computation function (and the Modularity one too) makes use of scikit-learn's mutual_info_score function (see here).
I suspect scikit-learn's mutual_info_score function having changed throughout the different versions is the issue here, and this is likely to be the root cause of discrepancies between the results you computed in February 2021 (I assume you were using scikit-learn version 0.24.0) and the original results of the paper (published on ArXiv around Nov. 2018, when the latest scikit-learn version was 0.20.0).

I am experiencing a lot of discrepancies with the Modularity score on my end (with a code adapted to another framework though).