How to understand the gradient backward propagation of perceptual loss?

[bolimath]

In train.lua, Line 199, local grad_out_percep = percep_crit:backward(out, target)

this line will call for the function crit:updateGradInput(input, target) in file fast_neural_style/PerceptualCriterion.lua,

however, according to the definition of the function
function crit:updateGradInput(input, target)
self.gradInput = self.net:updateGradInput(input, self.grad_net_output)
return self.gradInput
end

and self.grad_net_output is defined as zero at Line 137,
-- Set up a tensor of zeros to pass as gradient to net in backward pass
self.grad_net_output:resizeAs(output):zero()

In my opinion, if the grad_net_output is set as zero, the return of function updateGradInput will always be zero, which implies the perceptual gradient will not be involved in the backward of the network.

Can someone be kindly to help me understand it?

[htoyryla]

I guess this is explained by the perceptual losses being evaluated, not at the network output, but in content and style loss layers which have been inserted inside the network at selected layers. So we pass a zero gradient from the output backwards, and the real gradients will be added by the content and style loss layers.

At least this is what happens in the original, iterative neural-style, and I have the impression that fast-neural-style uses the same approach for perceptual loss evaluation. In other words, you should look at the content and style loss layers to see what they do to the gradient.