Magnifying movements in video

[alexanderfrey]

Hi,

thanks for your great work. I came here since you wrote that your software might be helpful in detecting subtle changes in video in the same way Eulerian-Magnification does. Can you please elaborate on that or show how to do that ?
brycedrennan/eulerian-magnification#16

I loaded a video and played a bit with the controls but seem to be a bit lost with the overall gui.

Thanks for your help
Alexander

[mark-orion]

Hi Alexander. Very busy here, apologies for the late response.
PyFSPro has a built-in function that calculates a 3D vector out of the image. The calculation is done by calculating the difference between the mean of all pixels in the upper half / lower half and left half / right half of the (output) image. Z is based on the mean of all pixels in the output.
The easiest way to get hold of the data is to activate logging mode to write the output to a file or STDOUT by using the command line options below. Now click on "Data Display" what will make a HUD like symbol popping up on the screen. "Data Output (1)" will enable the data flow. With "Datamode" you can switch between the standard averaging mode described above (AVG) and another mode that uses cumulative Z-scores. "Zero Output(2)" allows re-centering of the display in case the HUD marker wanders off the screen. The upper of the two sliders in the top-left corner adjusts a multiplicator (a bit like gain) what can be useful to amplify changes.
I have not tried it, but if you want to do things similar to the eulerian-magnification, then you could use one of the chroma channels (Cr, Cb - selectable under the "BW" dropdown in the bottom left). This would translate colour into values. Using the method described above and piping the output into some kind of filter or analyzing the logfile later on might allow extracting signals like the heartbeat from subtle changes in the color of a human face etc.

Happy experimenting!

Here are the command lines for logging:
./PyFSPro.py --log testlog
./PyFSPro.py --log STDOUT this will log directly to STDOUT.
The log format is:
Unixtime, Cleartext time, x, y, z, joystick x, joystick y