Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support R windows_amd64_rtools build #60

Open
natbprice opened this issue May 19, 2024 · 3 comments
Open

Support R windows_amd64_rtools build #60

natbprice opened this issue May 19, 2024 · 3 comments

Comments

@natbprice
Copy link

It appears that the azure extension does not currently work with R on Windows because it requires a special build (GCC 12/MinGW-w64). See duckdb/duckdb#10204 and duckdb/duckdb-r#100.

There is a workflow for this build, but it appears that it is being excluded in https://github.com/duckdb/duckdb_azure/blob/main/.github/workflows/MainDistributionPipeline.yml. I assume this is because it does not work as-is, but it is a bit beyond my understanding to determine what might be required to fix this issue.

@natbprice
Copy link
Author

I forked the repo and tested the build after removing exclusion: https://github.com/natbprice/duckdb_azure/actions/runs/9177755461/job/25236890148

mkdir -p build/release && \
cmake  -DEXTENSION_STATIC_BUILD=1 -DDUCKDB_EXTENSION_CONFIGS='/d/a/duckdb_azure/duckdb_azure/extension_config.cmake'  -DOSX_BUILD_ARCH=  -DVCPKG_MANIFEST_DIR='/d/a/duckdb_azure/duckdb_azure/' -DVCPKG_BUILD=1 -DCMAKE_TOOLCHAIN_FILE='D:\a\duckdb_azure\duckdb_azure/vcpkg/scripts/buildsystems/vcpkg.cmake' -DVCPKG_TARGET_TRIPLET='x64-mingw-static' -DDUCKDB_EXPLICIT_PLATFORM='windows_amd64_rtools'  -DCMAKE_BUILD_TYPE=Release -S ./duckdb/ -B build/release && \
cmake --build build/release --config Release
-- Building for: Unix Makefiles
-- Found Python3: C:/hostedtoolcache/windows/Python/3.11.9/x64/python3.exe (found version "3.11.9") found components: Interpreter 
-- Running vcpkg install
Detecting compiler hash for triplet x64-windows...
Detecting compiler hash for triplet x64-mingw-static...
The following packages will be built and installed:
  * azure-core-cpp[core,curl,http,winhttp]:[email protected]
    azure-identity-cpp:[email protected]
    azure-storage-blobs-cpp:[email protected]
  * azure-storage-common-cpp:[email protected]
    azure-storage-files-datalake-cpp:[email protected]
  * curl[core,non-http,schannel,ssl,sspi]:[email protected]#1
  * vcpkg-cmake:x64-windows@2023-05-04
  * vcpkg-cmake-config:x64-windows@2022-02-06#1
  * wil:x64-mingw-static@2023-10-28
  * zlib:[email protected]
Additional packages (*) will be modified to complete this operation.
A suitable version of 7zip was not found (required v23.1.0) Downloading portable 7zip 23.1.0...
Downloading 7zip...
https://www.7-zip.org/a/7z2301-extra.7z->D:\a\duckdb_azure\duckdb_azure\vcpkg\downloads\7z2301-extra.7z
Downloading https://www.7-zip.org/a/7z2301-extra.7z
Extracting 7zip...
Restored 2 package(s) from GitHub Actions Cache in 944 ms. Use --debug to see more details.
Installing 1/10 vcpkg-cmake-config:x64-windows@2022-02-06#1...
Elapsed time to handle vcpkg-cmake-config:x64-windows: 2.[31](https://github.com/natbprice/duckdb_azure/actions/runs/9177755461/job/25236890148#step:10:32) ms
vcpkg-cmake-config:x64-windows package ABI: ba4cffff032243d4c0473a0eeabddf5f75327cd59c0bc3c2e8a4c2b76e9dc811
Installing 2/10 vcpkg-cmake:x64-windows@2023-05-04...
Elapsed time to handle vcpkg-cmake:x64-windows: 2.28 ms
vcpkg-cmake:x64-windows package ABI: b5f858c3bb197b1a07f510b5333562c0c98fe176119806206e3a4bc256ac2d03
Installing 3/10 wil:x64-mingw-static@2023-10-28...
Building wil:x64-mingw-static@2023-10-28...
warning: -- Using community triplet x64-mingw-static. This triplet configuration is not guaranteed to succeed.
-- [COMMUNITY] Loading triplet configuration from: D:\a\duckdb_azure\duckdb_azure\vcpkg\triplets\community\x64-mingw-static.cmake
-- Downloading https://github.com/Microsoft/wil/archive/v1.0.231028.1.tar.gz -> Microsoft-wil-v1.0.231028.1.tar.gz...
-- Extracting source D:/a/duckdb_azure/duckdb_azure/vcpkg/downloads/Microsoft-wil-v1.0.231028.1.tar.gz
-- Using source at D:/a/duckdb_azure/duckdb_azure/vcpkg/buildtrees/wil/src/0.231028.1-2698fce114.clean
-- Configuring x64-mingw-static
-- Building x64-mingw-static-rel
-- Up-to-date: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/wil_x64-mingw-static/share/wil/natvis
-- Installing: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/wil_x64-mingw-static/share/wil/natvis/wil.natvis
-- Installing: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/wil_x64-mingw-static/share/wil/copyright
-- Performing post-build validation
Stored binaries in 1 destinations in 334 ms.
Elapsed time to handle wil:x64-mingw-static: 5.1 s
wil:x64-mingw-static package ABI: be2d8cd484e0927fc22fd6cd1ea86c2d28278c2455d08c5f1543cb294e90c05b
Installing 4/10 zlib:[email protected]...
Building zlib:[email protected]...
warning: -- Using community triplet x64-mingw-static. This triplet configuration is not guaranteed to succeed.
-- [COMMUNITY] Loading triplet configuration from: D:\a\duckdb_azure\duckdb_azure\vcpkg\triplets\community\x64-mingw-static.cmake
-- Downloading https://github.com/madler/zlib/archive/v1.3.tar.gz -> madler-zlib-v1.3.tar.gz...
-- Extracting source D:/a/duckdb_azure/duckdb_azure/vcpkg/downloads/madler-zlib-v1.3.tar.gz
-- Applying patch 0001-Prevent-invalid-inclusions-when-HAVE_-is-set-to-0.patch
-- Applying patch 0002-skip-building-examples.patch
-- Applying patch 0003-build-static-or-shared-not-both.patch
-- Applying patch 0004-android-and-mingw-fixes.patch
-- Using source at D:/a/duckdb_azure/duckdb_azure/vcpkg/buildtrees/zlib/src/v1.3-8825ee792f.clean
-- Configuring x64-mingw-static
-- Building x64-mingw-static-dbg
-- Building x64-mingw-static-rel
-- Installing: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/zlib_x64-mingw-static/share/zlib/vcpkg-cmake-wrapper.cmake
-- Fixing pkgconfig file: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/zlib_x64-mingw-static/lib/pkgconfig/zlib.pc
-- Downloading https://mirror.msys2.org/mingw/mingw64/mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst;https://repo.msys2.org/mingw/mingw64/mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst;https://mirror.yandex.ru/mirrors/msys2/mingw/mingw64/mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst;https://mirrors.tuna.tsinghua.edu.cn/msys2/mingw/mingw64/mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst;https://mirrors.ustc.edu.cn/msys2/mingw/mingw64/mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst;https://mirror.selfnet.de/msys2/mingw/mingw64/mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst -> mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst...
-- Downloading https://mirror.msys2.org/msys/x86_64/msys2-runtime-3.4.9-3-x86_64.pkg.tar.zst;https://repo.msys2.org/msys/x86_64/msys2-runtime-3.4.9-3-x86_64.pkg.tar.zst;https://mirror.yandex.ru/mirrors/msys2/msys/x86_64/msys2-runtime-3.4.9-3-x86_64.pkg.tar.zst;https://mirrors.tuna.tsinghua.edu.cn/msys2/msys/x86_64/msys2-runtime-3.4.9-3-x86_64.pkg.tar.zst;https://mirrors.ustc.edu.cn/msys2/msys/x86_64/msys2-runtime-3.4.9-3-x86_64.pkg.tar.zst;https://mirror.selfnet.de/msys2/msys/x86_64/msys2-runtime-3.4.9-3-x86_64.pkg.tar.zst -> msys2-msys2-runtime-3.4.9-3-x86_64.pkg.tar.zst...
-- Using msys root at D:/a/duckdb_azure/duckdb_azure/vcpkg/downloads/tools/msys2/023cdb3ca06f77f2
-- Fixing pkgconfig file: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/zlib_x64-mingw-static/debug/lib/pkgconfig/zlib.pc
-- Installing: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/zlib_x64-mingw-static/share/zlib/copyright
-- Performing post-build validation
Stored binaries in 1 destinations in 291 ms.
Elapsed time to handle zlib:x64-mingw-static: 15 s
zlib:x64-mingw-static package ABI: b056b95a19ea9feb83d0070b702bd6f630096bddf7b557ebc27f89559b61e346
Installing 5/10 curl[core,non-http,schannel,ssl,sspi]:[email protected]#1...
Building curl[core,non-http,schannel,ssl,sspi]:[email protected]#1...
warning: -- Using community triplet x64-mingw-static. This triplet configuration is not guaranteed to succeed.
-- [COMMUNITY] Loading triplet configuration from: D:\a\duckdb_azure\duckdb_azure\vcpkg\triplets\community\x64-mingw-static.cmake
-- Downloading https://github.com/curl/curl/archive/curl-8_5_0.tar.gz -> curl-curl-curl-8_5_0.tar.gz...
-- Extracting source D:/a/duckdb_azure/duckdb_azure/vcpkg/downloads/curl-curl-curl-8_5_0.tar.gz
-- Applying patch 0002_fix_uwp.patch
-- Applying patch 0005_remove_imp_suffix.patch
-- Applying patch 0012-fix-dependency-idn2.patch
-- Applying patch 0020-fix-pc-file.patch
-- Applying patch 0022-deduplicate-libs.patch
-- Applying patch mbedtls-ws2_[32](https://github.com/natbprice/duckdb_azure/actions/runs/9177755461/job/25236890148#step:10:33).patch
-- Applying patch export-components.patch
-- Applying patch dependencies.patch
-- Applying patch cmake-config.patch
-- Applying patch cmake-config-ios.patch
-- Using source at D:/a/duckdb_azure/duckdb_azure/vcpkg/buildtrees/curl/src/curl-8_5_0-7491aadb0f.clean
-- Configuring x64-mingw-static
-- Building x64-mingw-static-dbg
-- Building x64-mingw-static-rel
-- Fixing pkgconfig file: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/curl_x64-mingw-static/lib/pkgconfig/libcurl.pc
-- Using cached mingw-w64-x86_64-pkgconf-1~2.1.0-1-any.pkg.tar.zst.
-- Using cached msys2-msys2-runtime-3.4.9-3-x86_64.pkg.tar.zst.
-- Using msys root at D:/a/duckdb_azure/duckdb_azure/vcpkg/downloads/tools/msys2/023cdb3ca06f77f2
-- Fixing pkgconfig file: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/curl_x64-mingw-static/debug/lib/pkgconfig/libcurl.pc
-- Installing: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/curl_x64-mingw-static/share/curl/vcpkg-cmake-wrapper.cmake
-- Installing: D:/a/duckdb_azure/duckdb_azure/vcpkg/packages/curl_x64-mingw-static/share/curl/usage
-- Performing post-build validation
Stored binaries in 1 destinations in 1.1 s.
Elapsed time to handle curl:x64-mingw-static: 1.9 min
curl:x64-mingw-static package ABI: fa82f0987e8da32724109f1802dc0864d289b58a52a02649fa1020a47be894e2
Installing 6/10 azure-core-cpp[core,curl,http,winhttp]:[email protected]...
Building azure-core-cpp[core,curl,http,winhttp]:[email protected]...
warning: -- Using community triplet x64-mingw-static. This triplet configuration is not guaranteed to succeed.
-- [COMMUNITY] Loading triplet configuration from: D:\a\duckdb_azure\duckdb_azure\vcpkg\triplets\community\x64-mingw-static.cmake
-- Downloading https://github.com/Azure/azure-sdk-for-cpp/archive/azure-core_1.11.0.tar.gz -> Azure-azure-sdk-for-cpp-azure-core_1.11.0.tar.gz...
-- Extracting source D:/a/duckdb_azure/duckdb_azure/vcpkg/downloads/Azure-azure-sdk-for-cpp-azure-core_1.11.0.tar.gz
-- Using source at D:/a/duckdb_azure/duckdb_azure/vcpkg/buildtrees/azure-core-cpp/src/ore_1.11.0-4959a5d799.clean
-- Configuring x64-mingw-static
-- Building x64-mingw-static-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:1[34](https://github.com/natbprice/duckdb_azure/actions/runs/9177755461/job/25236890148#step:10:35) (message):
Error:     Command failed: "C:/Program Files/CMake/bin/cmake.exe" --build . --config Debug --target install -- -v -j5
    Working Directory: D:/a/duckdb_azure/duckdb_azure/vcpkg/buildtrees/azure-core-cpp/x64-mingw-static-dbg
    See logs for more information:
      D:\a\duckdb_azure\duckdb_azure\vcpkg\buildtrees\azure-core-cpp\install-x64-mingw-static-dbg-out.log

Call Stack (most recent call first):
  D:/a/duckdb_azure/duckdb_azure/build/release/vcpkg_installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_build.cmake:74 (vcpkg_execute_build_process)
  D:/a/duckdb_azure/duckdb_azure/build/release/vcpkg_installed/x64-windows/share/vcpkg-cmake/vcpkg_cmake_install.cmake:16 (vcpkg_cmake_build)
  ports/azure-core-cpp/portfile.cmake:[37](https://github.com/natbprice/duckdb_azure/actions/runs/9177755461/job/25236890148#step:10:38) (vcpkg_cmake_install)
  scripts/ports.cmake:170 (include)


error: building azure-core-cpp:x64-mingw-static failed with: BUILD_FAILED
Elapsed time to handle azure-core-cpp:x64-mingw-static: 21 s
Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
  https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+azure-core-cpp
You can submit a new issue at:
  https://github.com/microsoft/vcpkg/issues/new?title=[azure-core-cpp]+Build+error+on+x64-mingw-static&body=Copy+issue+body+from+D%3A%2Fa%2Fduckdb_azure%2Fduckdb_azure%2Fbuild%2Frelease%2Fvcpkg_installed%2Fvcpkg%2Fissue_body.md
You can also submit an issue by running (GitHub CLI must be installed):
  gh issue create -R microsoft/vcpkg --title "[azure-core-cpp] Build failure on x64-mingw-static" --body-file D:/a/duckdb_azure/duckdb_azure/build/release/vcpkg_installed/vcpkg/issue_body.md
-- Running vcpkg install - failed
CMake Error at D:/a/duckdb_azure/duckdb_azure/vcpkg/scripts/buildsystems/vcpkg.cmake:899 (message):
Error:   vcpkg install failed.  See logs for more information:
  D:/a/duckdb_azure/duckdb_azure/build/release/vcpkg-manifest-install.log
Call Stack (most recent call first):
  C:/rtools[42](https://github.com/natbprice/duckdb_azure/actions/runs/9177755461/job/25236890148#step:10:43)/x86_64-w64-mingw32.static.posix/share/cmake-3.24.1/Modules/CMakeDetermineSystem.cmake:124 (include)
  CMakeLists.txt:23 (project)


CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
make: *** [extension-ci-tools/makefiles/duckdb_extension.Makefile:56: release] Error 1
Error: Process completed with exit code 1.

@abfleishman
Copy link

Any updates on this issue or next steps for addressing it?

@eitsupi
Copy link

eitsupi commented Oct 7, 2024

This may be an upstream problem (azure cpp or mingw) that is unlikely to be resolved in the short term. microsoft/vcpkg#39554
Why not use WSL2?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants