CPP deployment using AppVeyor

Introduction

AppVeyor is a continuous intergration and deployment service for Windows developers. However, there is no good manual how to deploy a build of C++ project like MIPT-MIPS.

There are three major gamers in C++ development on Windows:

• Microsoft Visual Studio
• Minimalistic GNU on Windows (MinGW)
• Clang

Our appveyor.yml supports build by Visual Studio and MinGW. In this manual I'll explain how it works, line-by-line.

Build with Visual Studio

Visual Studio support is tricky for our non-commercial project, as usually there are no licenses for the latest Visual Studio version. Thus, sometimes you have to work in a 'blind' mode. Below I'll try to cover the most tricky corner cases I faced.

Set up

First of all, you have to select MSVC image. We are using the latest one:

version: "{branch}-ci-{build}"
image: Visual Studio 2017

There are some differences between versions of Visual Studio, I'll explain below.

Then, you add Visual Studio environment to the matrix. Please note the path to Boost linkable libraries: these paths are not mentioned on AppVeyor manuals!

environment:
  matrix:
  - build: msvc
    platform: x86
    BOOST_ROOT: C:\Libraries\boost_1_64_0
    BOOST_LIBRARYDIR: C:\Libraries\boost_1_64_0\lib32-msvc-14.1
    LIBELF_INCLUDE: C:\Projects\mipt-mips\libelf\lib
  - build: msvc
    platform: x64
    BOOST_ROOT: C:\Libraries\boost_1_64_0
    BOOST_LIBRARYDIR: C:\Libraries\boost_1_64_0\lib64-msvc-14.1
    LIBELF_INCLUDE: C:\Projects\mipt-mips\libelf\lib

Before build

After matrices are set, we may starting environment preparation:

# set variables for MSVC
- cmd: if "%build%"=="msvc"  call "C:\Program Files (x86)\Microsoft Visual tudio\2017\Community\Common7\Tools\VsDevCmd.bat" -arch=%PLATFORM%
- cmd: set INCLUDE=%LIBELF_INCLUDE%;%BOOST_ROOT%;%INCLUDE%
- cmd: set PATH=%LIBELF_INCLUDE%;%PATH%
- cmd: set LIB=%BOOST_LIBRARYDIR%;%LIB%

INCLUDE, PATH, and LIB are MSVC environment variables which usually do not interfere with MinGW, so we may set them unconditionally. As one may guess, they have following purposes:

• INCLUDE is a set of paths where MSVC will search for headers
• LIB is a set of paths where MSVC will search for static libraries (e.g. Boost)
• PATH should contain path to all *.dlls we may use in runtime (for MIPT-MIPS it is libelf.dll)

Please note that MSVC 2015 uses different command to set up instead of VsDevCmd.bat:

# set variables for MSVC
- cmd: if "%build%"=="msvc"  call "%VS140COMNTOOLS%\..\..\VC\vcvarsall.bat" %PLATFORM%

Build

For MIPT-MIPS we use a simple CMD script to do all the testing. It can be simply called in build_script section:

build_script:
- cmd: if "%build%"=="msvc"  cd C:\Projects\mipt-mips\simulator && call build.cmd

AppVeyor provides test_script scenario as well, but we do not actually need it and do almost nothing here:

test_script:
- cmd: cd C:\projects\MIPT-MIPS\simulator
- cmd: dir mipt-mips.exe

Build with MinGW

In short, MinGW is a GCC that uses Windows API. Because of that, there may be some differences between system calls. The most widespread error occurs because Windows distinguish 'binary' and 'non-binary' file opens. In 'non-binary' file mode, Windows will automatically translate \n symbol to \r\n, which is acceptable for text files but not for binaries. To prevent doing that, do the similar change in your C/C++ code:

-    FILE* file_descr = fopen( elf_file_name.c_str(), "r");
+    FILE* file_descr = fopen( elf_file_name.c_str(), "rb");

You don't have to check any macroes, as b character will be ignored by Linux.

Set up

MinGW is installed to the all of AppVeyor images, so we can stick with 2017:

version: "{branch}-ci-{build}"
image: Visual Studio 2017

Then, set up the matrix. We are going to use Msys2 environment provided by AppVeyor.

environment:
  matrix:
  - build: g++
    platform: x86
    MINGW_ROOT: C:\msys64\mingw32
    BOOST_ROOT: C:\msys64\mingw32\include\boost
    LIBELF_INCLUDE: C:\msys64\mingw32\include\libelf
    MINGW_ARCH: i686
    MSYSTEM: MINGW32
  - build: g++
    platform: x64
    MINGW_ROOT: C:\msys64\mingw64
    BOOST_ROOT: C:\msys64\mingw64\include\boost
    LIBELF_INCLUDE: C:\msys64\mingw64\include\libelf
    MINGW_ARCH: x86_64
    MSYSTEM: MINGW64

Before build

By default, MinGW is not included to Windows' PATH variable. Ok, it is not a problem for us:

- cmd: set PATH=%MINGW_ROOT%;%MINGW_ROOT%\bin;C:\msys64\usr\bin\;"C:\Program Files\LLVM\bin";%PATH%

Also, we need to download libraries. They are fetched by Msys2's package manager. Don't forget to use right target architecture name for downloading.

- cmd: bash -lc "pacman --needed --noconfirm -S mingw-w64-%MINGW_ARCH%-libelf"
- cmd: bash -lc "pacman --needed --noconfirm -S mingw-w64-%MINGW_ARCH%-boost"

As in Linux, MinGW uses CPATH, C_INCLUDE_PATH, and CPLUS_INCLUDE_PATH variables to search for includes and libraries. It is worth to mention that environment variables are shared by Msys2 and Windows CMD, so you can set them in CMD and use in bash later.

- cmd: set CPATH=%LIBELF_INCLUDE%;%CPATH%
- cmd: set CPLUS_INCLUDE_PATH=%LIBELF_INCLUDE%;%BOOST_ROOT%;%CPLUS_INCLUDE_PATH%

Build

Build is run under the Msys2 bash. Note that I had to add -mt suffix to boost libraries as Pacman doesn't fetch signle-threaded versions.

build_script:
- cmd: if not "%build%"=="msvc" bash -lc "cd /c/projects/mipt-mips/simulator/; make BOOST_POSTFIX=-mt mipt-mips test"

We use same dummy scenario for 'testing':

test_script:
- cmd: cd C:\projects\MIPT-MIPS\simulator
- cmd: dir mipt-mips.exe

Clang

WIP on Clang support in low priority as have clang builds in Linux and OSX on Travis. Additionally, supporting Clang for Windows is tricky as it may behave either MSVC-like compiler or GCC-like compiler