Testing

[schmiph2]

Hi jfsantos

Thank you for porting the SRMRToolbox to Python. I compared the results from your implementation with the results of the Matlab implementation. Both using the same configuration. The differences between the values are rather high. I tested for example the file 'speech_bab_0dB.wav' (16kHz sampling rate ) of the supporting material of Loizou's speech enhancement book (see Link to Book). I got a value of 1.8493 using the Matlab implementation and a value of 1.86596 using the Python implementation. So the difference is much larger then the tolerances used in your test script. If i use the normalized version, the difference is even larger ( 1.1696 vs. 1.4679).
Have you observed similar behaviour? Am I using the toolbox incorrectly?

[jfsantos]

Sorry for taking half a year to reply. I finished grad school almost two years ago and do not use this package much in my new job.

I just ran a comparison and the differences indeed are very high with the fast implementation. I would hypothesize the differences in the slow implementation will not make a big difference, but the fast implementation depends on the Gammatone package, which was updated independently and I unfortunately did not tag a release so we're installing whatever is on master in that repo. I also did not tag Numpy and Scipy, but I assume their implementations are numerically stable and should not cause such a large difference.

If you want to use this package, I would recommend comparing the results from the MATLAB implementation to this implementation on several audio files, and making sure the correlation between the results is still very high (>=0.99). The absolute values do not matter much, but as long as the values are correlated, this implementation will still be useful (especially if you cannot/do not want to use MATLAB).