self play is exceptionally slow on GPU

[dberma15]

I'm using the code as is on Othello in tensoeflow and I'm finding it to be an order of magnitude slower on GPU compared to cpu. Is there a reason for this?

[nullne]

GPU isn't used when self play

[zjusbo]

The time might be wasted when copying the data from CPU memory to GPU and then copy it back.

The NN forward uses GPU, other operations use CPU in the selfplay.