Feedback from Windows integration for a CMake + Qt + C + Python + Perl project

Hi!

Thanks for making an early preview available of Windows on Travis. I’ve been trying to add Windows support to our existing .travis.yml configuration which already supports Linux and macOS. Below I’ll share some issues I encountered during integration along with workarounds and suggestions for improvement.

A few characteristics of the project:

  • Already supports Linux (Xenial) and macOS on Travis.
  • Uses CMake as build system.
  • MSVC is preferred (assumed?) on Windows.
  • The build process requires Python and Perl for code generation and/or tests.
  • Depends on Qt5 for the GUI part.

Outline of this post:

  • Windows VM not booting
  • Caveats with backslashes and absolute paths in env and cache
  • Python is not installed by default, refreshenv not working
  • msbuild is not available in PATH
  • Qt 5 is not installed
  • Timeout while storing build cache
  • Conclusion

Windows VM not booting

Our existing .travis.yml contained language: minimal, this prevented the machine from booting.

Workaround: set language: c, language: shell or (presumably) any other supported language.

Suggestion: fix this bug, boot VMs even for unsupported languages (and do not install extra packages in that case).

Caveats with backslashes and absolute paths in env and cache

This environment and cache configuration does not work as intended:

- env:
  - WIRESHARK_BASE_DIR: C:\wireshark-libs
- cache:
  directories:
    - $WIRESHARK_BASE_DIR
    - C:\ProgramData\chocolatey\bin

The backslashes are probably treated as escape character by Bash:

Setting environment variables from .travis.yml
$ export WIRESHARK_BASE_DIR=C:\wireshark-libs
...
adding C:wireshark-libs to cache
creating directory C:wireshark-libs
adding C:ProgramDatachocolateybin to cache
creating directory C:ProgramDatachocolateybin
...
C:\Users\travis\build\Lekensteyn\wireshark\wireshark-libs: No such file or directory
C:\Users\travis\build\Lekensteyn\wireshark\wireshark-libs: unknown file type
C:\Users\travis\build\Lekensteyn\wireshark\ProgramDatachocolateybin: No such file or directory
C:\Users\travis\build\Lekensteyn\wireshark\ProgramDatachocolateybin: unknown file type

Another configuration example:

- cache:
  directories:
    - C:\\blah
    - C:\\\\tree
    - C:\\ProgramData\\chocolatey\\bin
    - /c/ProgramData/chocolatey/lib

and its effect:

adding Clah to cache
creating directory Clah
adding C:\tree to cache
creating directory C:\tree
adding C:\ProgramDataadding /c/ProgramData/chocolatey/lib to cache

While Clah was displayed, ls -l /c did reveal the presence of a new blah directory. Another issue was found when outputting cached dirs (see Travis CI - Test and Deploy with Confidence, \t is somehow converted to a tab and \b to a backspace):

changes detected (content changed, file is created, or file is deleted):
C:\Users	raviuild\Lekensteyn	ravis-windows-test\mydir\out.txt

Workaround:

  • Use forward slashes (C:/ProgramData/chocolatey/bin) or MSYS-style paths (/c/ProgramData/chocolatey/bin). Using two backslashes also works, but due to a casher bug it is displayed in a wrong way.
  • When absolute paths are in use (e.g. C:\Qt\5.12.0), manually move the the Qt directory contents to the root folder in the before_install phase. Additionally, to avoid reuploading an unmodified cache, update a casher checksum file. Example, given two directories C:/wireshark-libs and C:/Qt:
for path in wireshark-libs Qt; do if [ -d "$path" ]; then echo "Restoring C:/$path"; mv "$path/"* "C:/$path"; fi; done
if [ -d Qt ]; then md5deep64 -o f -r "C:/wireshark-libs" "C:/Qt" | sort > ~/.casher/md5sums_before; fi

Suggestions:

Python is not installed by default, refreshenv not working

We need Python 3 (and pip3), so we installed it through Chocolatey (cinst -y python3). This did not update PATH though.

The refreshenv (RefreshEnv.cmd) command did not have any effect:

before_install:
  - |
    if [ "$TRAVIS_OS_NAME" == "windows" ]; then
      cinst -y python
      echo "old PATH=$PATH"
      refreshenv
      echo "new PATH=$PATH"
    fi
  - echo "now PATH=$PATH"
  - pip3 install pytest pytest-xdist

In all cases, it showed PATH=/bin:/usr/bin:/c/tools/ruby25/bin:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/ProgramData/GooGet:/c/Program Files/Google/Compute Engine/metadata_scripts:/c/Program Files (x86)/Google/Cloud SDK/google-cloud-sdk/bin:/c/Program Files/Google/Compute Engine/sysprep:/c/Program Files/Docker:/c/ProgramData/chocolatey/bin:/c/Program Files/CMake/bin:/c/Program Files/Git/cmd:/c/Program Files/LLVM/bin:/c/Program Files/dotnet:/c/Users/travis/AppData/Local/Microsoft/WindowsApps:/c/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/bin and pip3 is not found.

Workaround: manually query the updated path and update PATH:

# refreshenv does not seem to work in bash, so reload it manually.
# Entries in the Machine PATH might contain trailing slashes, drop those.
# Apply Process paths before Machine to ensure /bin appears before others.
export PATH="$(powershell -Command '("Process", "Machine" | % {
  [Environment]::GetEnvironmentVariable("PATH", $_) -Split ";" -Replace "\\$", ""
} | Select -Unique | % { cygpath $_ }) -Join ":"')"
echo "new PATH=$PATH"

Redirecting stdin to /dev/null was locally necessary or else Powershell would not exit (this does not seem to be the case with Travis though). The above snippet is similar to refreshcmd, but additionally preserves additional paths for git-bash (/bin, etc.). Unlike refreshcmd, it will append new paths rather than prepend (otherwise find would execute FIND.exe instead of GNU find).

Suggestions:

  • Install Python 3 by default.
  • Make refreshenv work (ideally, make it optional after installing Chocolatey packages).
  • Perhaps provide an option for native PowerShell commands. AppVeyor allows for keys such as:
    • - ps: powershell command for Windows only
    • - sh: bash command for Linux/macOS only
    • - command for both Windows/Linux (no prefix). Perhaps bash (Linux/macOS) or git-bash (Windows)? AppVeyor uses bash/powershell depending on platform.

msbuild is not available in PATH

The command msbuild /p:Configuration=RelWithDebInfo /m *.sln failed due to msbuild not being in the default PATH. Currently the MSVC version (2015, 2017) is also not configurable, so that might be related.

Workaround: use cmake --build . --config RelWithDebInfo --parallel for a similar effect, but rely on CMake to locate the toolset path.

Qt 5 is not installed

Qt 5 is quite a big dependency, it took 7-8 minutes to download and install Qt 5.12 msvc2017_64 (604M installed size). (AppVeyor has multiple Qt versions installed by default.)

Workaround: manually install Qt and automate the installation process:

curl -vLO http://download.qt.io/official_releases/online_installers/qt-unified-windows-x86-online.exe
./qt-unified-windows-x86-online.exe --verbose --script tools/qt-installer-windows.qs

The automation script for non-interactive installation can currently be found at: https://github.com/wireshark/wireshark/blob/master/tools/qt-installer-windows.qs

Note: adding --verbose will result in an extra 14k lines in the log file. If everything is working, you can remove this. Alternatively buffer the output and display it in case of errors:

curl -vLo ~/qt-unified-windows-x86-online.exe http://download.qt.io/official_releases/online_installers/qt-unified-windows-x86-online.exe
if ! ~/qt-unified-windows-x86-online.exe --verbose --script tools/qt-installer-windows.qs > ~/qt-installer-output.txt; then
  cat ~/qt-installer-output.txt; exit 1
fi

Suggestion: Install Qt 5 by default (for example, 5.12 LTS built for MSVC 2017, 32-bit and 64-bit).

Related feature request:

Timeout while storing build cache

The documentation says:

Failure to upload the cache does not mark the job as failed.

but if the cache takes a long time to store, a timeout could occur after 10 minutes:

store build cache
Running casher in bash
changes detected, packing new archive


No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received

The build has been terminated

This issue occurred while trying to store some large directories (C:/Qt, C:/ProgramData/chocolatey/bin, C:/ProgramData/chocolatey/lib and a project-specific download cache):

  • C:/Qt/5.12.0/msvc2017_64 - 604MiB
  • C:/Qt/5.12.0/msvc2017 - 516MiB
  • C:/ProgramData/chocolatey/bin - 4MiB
  • C:/ProgramData/chocolatey/lib - 564MiB

However, even after removing all these directories and cleaning up the download cache in the before_cache phase, it would still hang. That turned out to be related to the PATH change, somehow casher did not like that Windows paths occur before Bash paths (e.g. /c/Windows/system32 before /bin).

Suggestion: report the estimated cache size (and abort if it is too large), periodically report progress. Make casher depend less on bash utilities, and certainly not break on a more conventional PATH for Windows.

Conclusion

The initial integration experience was not quite smooth. The VM would not start at all (due to language: minimal) without any feedback. The configuration (cache directories and execution environment) behaved rather unexpected with regard to backslashes. Qt is a large dependency, trying to optimize the installation time for this using the Travis cache was painful and revealed a couple of bugs (absolute paths not being handled correctly and silently hanging on an unexpected PATH value).

A major pain is that choco install does not update PATH and currently requires an ugly hack. This should probably be fixed before Windows support leaves the alpha phase.

The Bash environment is both a strength and weakness. Users of Travis are likely already familiar with Bash (loops, if, etc.) and can use tools like GNU find, rm -rf, etc. On the other hand, PowerShell is more common on Windows (and also used by AppVeyor). Time will tell if the Bash environment brings further incompatibilities.

After getting the integration done, the actual jobs ran quite well. One killer feature over AppVeyor is the ability to run parallel jobs in a single build as well as multiple builds. I’m looking forward to make use of this feature.

It would also be nice to have Windows listed at https://www.traviscistatus.com/ (backlog, popularity).

The final required .travis.yml changes can be seen at:

4 Likes

+1 on installing Python 3 by default.

+1 for refreshenv

See RefreshEnv for bash for a hack to get refreshenv working.

I suggest you use choco package vswhere to locate the VS build tools available on the box.
It has a -legacy mode for pre-2017 tools. Can it be used to find earlier than VS 2017 · Issue #187 · microsoft/vswhere · GitHub
It is definately on the box, as I had to remove it for my builds, in order to confirm that my project was buildable without VS compilers, so I have

    - rm -rf
        "/c/Program Files (x86)/MSBuild"
        /c/Windows/Microsoft.NET/Framework/v4.0.30319/msbuild.exe

That is probably where you will find msbuild.exe.

1 Like

To avoid the timeout while storing cache (and in general speeding up your build) you’ll need to disable Windows Defender. (Ideally Travis should include this configuration in the VM image.)