如何使用cuda/gpu版本的libtorch

[jalaxy33]

如题。系统为 windows11，使用默认 toolchain，系统已安装 cuda-toolkit 并添加环境路径。

xmake.lua 配置文件如下：

add_rules("mode.debug", "mode.release")

set_runtimes("MD")
add_requires("libtorch", {configs = {shared=true, cuda=true}})

set_languages("c++20")

target("main")
    set_kind("binary")
    add_packages("libtorch")
    add_files("src/*.cpp")
    set_default(true)

当前项目只有一个用来测试的 main.cpp：

#include <iostream>
#include <torch/torch.h>

int main(int argc, char **argv) {
    std::cout << "hello world!" << std::endl;
    std::cout << torch::cuda::is_available() << std::endl;
    return 0;
}

编译运行成功，终端输出：

hello world!
0

问题描述：
第二行输出 0，说明程序未能检测到 GPU，只能使用 CPU 版本的 libtorch，无法使用 cuda 对张量运算进行加速。

需求：
使用 cuda/gpu 版本的 libtorch。

[xq114]

需要排查一下，暂时可以先下官方的gpu预编译包，然后 add_requires("cmake::Torch", {configs = {envs = {CMAKE_PREFIX_PATH = "path/to/libtorch"}}}) 引入

[jalaxy33]

按您说的方式，我下载了gpu预载包，解压后添加：

add_requires("cmake::Torch", {alias = "libtorch", configs = {envs = {CMAKE_PREFIX_PATH = "D:/3rdLibs/packages/libtorch/libtorch-win-shared-with-deps-2.4.1+cu124/libtorch"}}})

结果编译失败，错误日志显示：

cannot import module: package.manager.cmake.install_package, not found!

libtorch/ 的目录结构如下：

├─bin
├─cmake
├─include
├─lib
├─share
└─test
  build-hash
  build-version

---

我查了一下网上的解决方案，说想要使用 gpu 版的 libtorch 需要添加两行链接器选项：

/INCLUDE:?searchsorted_cuda@native@at@@YA?AVTensor@2@AEBV32@0_N1@Z
/INCLUDE:?warp_size@cuda@at@@YAHXZ

网上的方案给的都是在visual studio中设置的例子，我尝试在xmake.lua中通过 add_ldflags 或 add_culdflags 添加，编译通过但是使用的仍是 cpu。参考链接如下：

• https://zhuanlan.zhihu.com/p/371802785
• libtorch 1.8.0 precompiled has no CUDA backend linked (Adding "-INCLUDE:?warp_size@cuda@at@@YAHXZ" no longer helps) pytorch/pytorch#54131
• https://learn.microsoft.com/zh-cn/cpp/build/reference/include-force-symbol-references?view=msvc-170

---

我目前的 xmake.lua 如下：

add_rules("mode.debug", "mode.release")

add_requires("libtorch", {configs={cuda=true, blas='eigen'}})
set_languages("c++17")

target("main")
    set_kind("binary")
    add_packages("libtorch")
    add_files("src/*.cpp")
    add_ldflags("/INCLUDE:?searchsorted_cuda@native@at@@YA?AVTensor@2@AEBV32@0_N1@Z", {force = true})   -- 尝试添加
    add_ldflags("/INCLUDE:?warp_size@cuda@at@@YAHXZ", {force = true})   -- 尝试添加
    set_default(true)

测试代码 src/main.cpp 如下：

#include <iostream>
#include <torch/script.h>
#include <torch/torch.h>

int main(int argc, char **argv) {
    std::cout << "hello world!" << std::endl;

    if (torch::cuda::is_available()) {
        std::cout << "Using cuda" << std::endl;
    } else {
        std::cout << "Using cpu" << std::endl;
    }
    return 0;
}

编译运行结果：

hello world!
Using cpu

我刚开始用 xmake，很多东西不太清楚，折腾好几天了，麻烦您帮忙看看，谢谢！

[xq114]

用你的代码试了下

set_project("toytrain")
set_languages("cxxlatest")

add_rules("mode.debug", "mode.release")
set_runtimes("MD")

add_requires("cmake::Torch", {system = true, configs = {envs = {CMAKE_PREFIX_PATH = "C:\\Users\\xq114\\Downloads\\libtorch"},
                                                        presets = {CMAKE_BUILD_TYPE = "Release"}}})

target("toytrain")
    set_kind("binary")
    add_packages("cmake::Torch")
	add_files("src/main.cpp")
    add_ldflags("/INCLUDE:?warp_size@cuda@at@@YAHXZ", {force = true})

编译

> xmake -v
checking for cl.exe ... C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe
checking for the c++ compiler (cxx) ... cl.exe
checking for C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe ... ok
checking for flags (-O2) ... ok
checking for flags (cl_external_includedir) ... ok
checking for flags (-DNDEBUG) ... ok
[ 50%]: compiling.release src\main.cpp
"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX64\\x64\\cl.exe" -c -nologo -O2 -std:c++latest -MD -DWIN32 -D_WINDOWS -DUSE_DISTRIBUTED -DUSE_C10D_GLOO /EHsc -external:W0 -external:IC:\Users\xq114\Downloads\libtorch\include -external:W0 -external:IC:\Users\xq114\Downloads\libtorch\include\torch\csrc\api\include -external:W0 "-external:IC:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.5\\include" -external:W0 "-external:IC:\\Program Files\\NVIDIA Corporation\\NvToolsExt\\include" -DNDEBUG -Fobuild\.objs\toytrain\windows\x64\release\src\main.cpp.obj src\main.cpp
checking for flags (cl_sourceDependencies) ... ok
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include\optional(82): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include\xutility(389): warning C4267: 'initializing': conversion from 'size_t' to '_Ty', possible loss of data
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include\xutility(403): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include\utility(247): warning C4267: 'initializing': conversion from 'size_t' to '_Ty1', possible loss of data
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include\xutility(390): warning C4267: 'initializing': conversion from 'size_t' to '_Ty', possible loss of data
[ 75%]: linking.release toytrain.exe
"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX64\\x64\\link.exe" -nologo -dynamicbase -nxcompat -machine:x64 -libpath:C:\Users\xq114\Downloads\libtorch\lib "-libpath:C:\\Program Files\\NVIDIA Corporation\\NvToolsExt\\lib\\x64" "-libpath:C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.5\\lib\\x64" /opt:ref /opt:icf torch.lib kineto.lib cudart_static.lib caffe2_nvrtc.lib torch_cuda.lib torch_cpu.lib c10_cuda.lib c10.lib cudart.lib nvToolsExt64_1.lib /INCLUDE:?warp_size@cuda@at@@YAHXZ -out:build\windows\x64\release\toytrain.exe build\.objs\toytrain\windows\x64\release\src\main.cpp.obj
checking for cl.exe ... C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe
checking for the c compiler (cc) ... cl.exe
[100%]: build ok, spent 16.735s

可以运行

> xmake run
hello world!
Using cuda

[xq114]

引入不了预编译包查一下xmake版本是不是最新，xmake clean -a 清除缓存重试

[jalaxy33]

ok了，感谢！