Question: GPU support.

[sehHeiden]

I tried the solar panel Segmentation model from the model zoo.

But the execution was ruther slow it took 80+ minutes for a digital Orthophoto tile (20 cm resultion, 5000 points in each direction).

Inference was done on the CPU, question can I do the inference with Deepness on GPU, too?

Or is it feasible to do?

CPU: Ryzen 7 5800X
GPU: NVidea 1660Ti

[przemyslaw-aszkowski]

Hi
the underneath engine in Deepness is ONNX, which supports GPU. But it doesn't work out of the box, you would need to setup your PC - see instructions for onnxruntime with GPU.
Cheers
Przemek

[sehHeiden]

Thanks a lot. That gave me a lot of headaches. As Pycharm did work, but onnx neither here or this plugin, nor for onnxruntime.

I just, updated cudnn it worked for DEEPNESS. Than removed the CPU option for onnxruntime and the gpu was taken. That was easy. ;)

What makes me wonder is the timing I run the model with DEEPNESS and onnxruntime.

1. DEEPNESS:
   1. CPU: 80 min
   2. GPU: 61 s
2. ONNX RUNTIME (with Python 3.12 but without the erosion, class probability and vectorization...)
   1. CPU: 107 s
   2. GPU: 27 s

What I want to say is, theplugin is much slower on CPU, that what I tried in code.

[sehHeiden]

A symbol/message would be nice, whether GPU, etc. was detected (before running).

[przemyslaw-aszkowski]

Hi,
good idea to show the message whether CPU/GPU detected!
Also quite interesting findings with slow CPU. I do not know the reason, an investigation will be needed.

[sehHeiden]

As the plugin used the Gpu lately I couldn't test, why the CPU was so slow.

I read about speedups of about 40 X for using the GPU. That would be the correct order of magnitude. Or even better.
Perhaps I also set the wrong resolution like 5 cm per pixel compared to the original 10 or 20 cm per pixel.

What I noticed that when I select the raster with a 10 cm resolution, than loading the default parameter does not lead to selecting 10 cm as parameter for the net. For 20 cm it did (often?).