sherpa-onnx 能否使用 cuda-12.4.0 、 cudnn-8.9.x 实现 GPU 运算

[iAInNet]

使用 GPU 运行模型，执行命令如下

./sherpa-onnx/build/bin/sherpa-onnx-offline
--tokens=./models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17/tokens.txt
--sense-voice-model=./models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17/model.onnx
--num-threads=2
--debug=1
--provider=cuda
./models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17/test_wavs/zh.wav
./models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17/test_wavs/en.wav

报错如下：

Creating recognizer ...
terminate called after throwing an instance of 'Ort::Exception'
what(): /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1209 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory

cuda 和 cudnn 安装方式参考文档是： https://k2-fsa.github.io/k2/installation/cuda-cudnn.html#cuda-12-4 。

which nvcc

/home/voice11/repositories/cuda_env/cuda-12.4.0/bin/nvcc

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Feb_27_16:19:38_PST_2024
Cuda compilation tools, release 12.4, V12.4.99
Build cuda_12.4.r12.4/compiler.33961263_0

因为报错的代码来自 onnxruntime_src， 我先去 onnxruntime github 中查找了相关问题：

• FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory microsoft/onnxruntime#21684
• Incompatible libs between libonnxruntime_providers_cuda.so version 1.18 and CUDA 12.4 microsoft/onnxruntime#20944

看描述，cuda12.x 中的库文件是 libcublasLt.so.12 , 但是 libonnxruntime_providers_cuda.so 要找的是 libcublasLt.so.11

查看 ldd /home/voice11/repositories/sherpa-onnx/build/_deps/onnxruntime-src/lib/libonnxruntime_providers_cuda.so ，的确有不少库文件丢失

linux-vdso.so.1 (0x00007ffc3e1b4000)
libcublasLt.so.11 => not found
libcublas.so.11 => not found
libcudnn.so.8 => /home/voice11/repositories/cuda_env/cuda-12.4.0/lib/libcudnn.so.8 (0x00007b6d5de00000)
libcurand.so.10 => /home/voice11/repositories/cuda_env/cuda-12.4.0/lib64/libcurand.so.10 (0x00007b6d57600000)
libcufft.so.10 => not found
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007b6d7b525000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007b6d7b520000)
libcudart.so.11.0 => not found
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007b6d57200000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007b6d7b439000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007b6d7b417000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007b6d56e00000)
/lib64/ld-linux-x86-64.so.2 (0x00007b6d7b53c000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007b6d7b412000)

再查看 /home/voice11/repositories/sherpa-onnx/build/_deps/onnxruntime-src/lib/libonnxruntime.so

linux-vdso.so.1 (0x00007ffc1e2de000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x000074a40d2ca000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x000074a40d2c5000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x000074a40d2a9000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x000074a40d2a4000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x000074a40b800000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000074a40d1bb000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000074a40d19b000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000074a40b400000)
/lib64/ld-linux-x86-64.so.2 (0x000074a40d2df000)

并没有使用 libonnxruntime_providers_cuda.so，所以并不清楚 libonnxruntime_providers_cuda.so 是怎么跑出来的，有一个相关代码改动是： #1210 。

至此，排查就不知道如何进行下去了。

[csukuangfj]

我们支持的是 onnxruntime 1.17.1, 请使用 cuda 11.8

[csukuangfj]

如果你不想使用 onnxruntime 1.17.1, 请自行修改 sherpa-onnx 的代码，替换 onnxruntime 的版本，然后参考 onnxruntime 的文档，选择合适的 cuda 版本.