Tensor contiguity and view

[rayed-therap]

context_vec = context_vec.contiguous().view(b, num_tokens, self.d_out)

I'm a little confused by the use of contiguous() and view(). torch.tensor.contiguous returns a contiguous in memory tensor and torch.tensor.view returns a new tensor with the same data as the self tensor but of a different shape. Seems like we're allocating new tensors in memory when maybe we can just use reshape which will return a view if possible; otherwise, copy the the elements into a new tensor. The view docs even mention using reshape in case of confusion.

https://pytorch.org/docs/stable/generated/torch.Tensor.view.html

[rasbt]

Good point, I was previously thinking about that but then I used .view() everywhere else in the book so I used that here for consistency. Maybe .reshape would be a bit more elegant here ... something to think about. Thanks for highlighting that!