Status Report

[DonaldTsang]

@aizvorski since the repo has been out of date for 3 years, is it possible to pass the code to @siavelis and/or @OhmGeek ?

[slhck]

I'd like to add here (coming from slhck/ffmpeg-quality-metrics#8) that moving this to a separate organization would be most beneficial for all involved, since then there could be more exposure, creating PRs would be easier, and so on.

In principle we could also fork this over to the VQEG organization, and we could link to it from the software/tools website as well (https://vqeg.github.io/software-tools/).

[aizvorski]

@DonaldTsang This is possible, however - from my experience with open source - I would like to see someone actually do a better job before taking the lead. Contribute some code; review other people's contributions; have a fork which is consistently better than / ahead of the original; etc.

It looks like @OhmGeek has done some cool stuff with this repo, just hasn't bumped it upstream (and hence I wasn't aware of it).

[aizvorski]

@slhck VQEG would be a nice home for this; it certainly is very close in terms of topic.

[slhck]

It would be nice indeed if the fork at https://github.com/OhmGeek/video-quality was merged back upstream. Could you, Alex, perhaps do that? The things @OhmGeek has done are certainly useful (e.g. packaging, Python 3 compatibility).

My 2¢: rather than trying to implement all the metrics, first focus on providing a usable package that covers the most important ones. Usability improves when you have:

• a ready-to-install Python package without huge dependencies or compilation
• interfaces for decoding images/videos that are not raw YUV
• a sane output format (JSON, CSV, …) to facilitate piping to other tools
• …

At least that's what I was aiming for with https://github.com/slhck/ffmpeg-quality-metrics/ — only problem being that it's limited to what ffmpeg can do.

But maybe @DonaldTsang could elaborate on what the research context or goal is?

[DonaldTsang]

@slhck the main goal of having a centralized repo is to make sure that people have enough ways of trying and testing out both established measuring methods of videos, and more nuanced ways of doing such tasks according (roughly) to more recent papers, in order to aid in the standardization of video quality evaluations in other high profile or high usage repos (in a very close to heart case https://github.com/hydrusnetwork/hydrus ).
The major issues are:

1. badly documented or non-standardized algorithms
2. documented algorithms with no actual code (e.g. C, C++, Go or Python)

• C, C++ or Go would be used for low-level implementations
• Python would be used for high-level implementations or wrappers

3. lack of community dialog about the development of new methods