(feature) Ops estimation per node/per net.

[enricoros]

Would be great to understand how many ops are required per each node and overall in the network (maybe under the assumption of an acyclical graph).

This can actually be a teaching tool for people to improve their understanding of Convolutions for example, and useful to "weigh" a particular network.

This doesn't need to clutter the UI, it can totally reside in the Op/Net details panel.

[lutzroeder]

What do you mean by ops? Can you provide some examples? This would require #71 to be addressed first. The tool doesn't have a dependency on the Python runtime and frameworks.

[enricoros]

I was thinking of something along the lines of https://arxiv.org/abs/1605.07678, where the paper ranks neural nets based on the Accuracy (vertical axis, bubble chart) and G-Operations.

The GOPs number indicates how many operations (usually multiplications+additions) are spent on a particular layer.
For FC layers, for example, you have M*N operations where every activation (output) is a combination of all the inputs, so for an input of shape [1, 3, 3, 50] and an output of [1, 1, 1, 100] you would have 45,000 operations plus 100 biases, if present.
CONV layers are more conservative as in for every point in the output space you have a linear combination of 'close' points in the input space, and the formula for OPs is different.

This tool provides a reference calculation of OPs for different layers: https://github.com/apaszke/torch-opCounter/tree/master/src (see profiler.lua).

Thanks for considering these feature requests 👍

[lutzroeder]

#204