Using Build Matrix for multiple Ubuntu distributions

Travis CI Discussions & Feedback
,
1

Hi,
I’m trying to test some scripts on multiple Ubuntu distributions such as Xenial, Bionic, Focal, etc. There are 5 tests to be run on each distribution, and currently using Xenial & Bionic, so 5*2=10 jobs. After adding another distribution, it would increase to 15 jobs.

Currently, each of the 10 jobs is being specified separately, so 10 entries, and I would need to add 5. That didn’t look very clean to me (even though would need to do that once) and making any change would require changing it in multiple jobs. Therefore I’m trying to use a Build Matrix to reduce all this. This is my current travis.yml

language: python
python: 3.7
os: linux

# Build Matrix for Travis
dist:
  - xenial
  - bionic
  - focal

# Scripts to be tested
env:
  - CIINSTALL=yes SCRIPT=12   # BasicSetup + GenSoftware
  - CIINSTALL=yes SCRIPT=3    # ML-Basic
  - CIINSTALL=yes SCRIPT=4    # Build-OpenCV
  - CIINSTALL=yes SCRIPT=14   # BasicSetup + OpenCV (Conda OpenCV)
  - CIINSTALL=yes SCRIPT=5    # BuildML

script:
  # BasicSetup + GenSoftware
  - if [[ $SCRIPT -eq 12 ]]; then
      ./1-BasicSetUp.sh;
      export PATH=/opt/anaconda3/bin:$PATH;
      hash -r;
      ./2-GenSoftware.sh;
    fi

  # ML-Basic
  - if [[ $SCRIPT -eq 3 ]]; then
      ./3-ML-Basic.sh;
    fi

  # Build-OpenCV
  - if [[ $SCRIPT -eq 4 ]]; then
      ./Build-OpenCV.sh;
      ffmpeg -version;
      gdb --ex='set confirm on' -ex=run -ex=bt -ex=quit --args python3 extras/opencv_test.py;
    fi

  # BasicSetup + OpenCV (Conda OpenCV)
  - if [[ $SCRIPT -eq 14 ]]; then
      ./1-BasicSetUp.sh;
      export PATH=/opt/anaconda3/bin:$PATH;
      hash -r;
      ./Build-OpenCV.sh;
      ffmpeg -version;
      gdb --ex='set confirm on' -ex=run -ex=bt -ex=quit --args python extras/opencv_test.py;
    fi

  # BuildML
  - if [[ $SCRIPT -eq 5 ]]; then
      ./Build-ML.sh;
    fi


before_install:
    - sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) main universe restricted multiverse"
    - sudo apt-get update -qq
    - sudo apt-get install -y software-properties-common
    - sudo apt-get install -qq build-essential curl g++ python3-dev python3-setuptools gdb
    - echo 'America/Los_Angeles' | sudo tee /etc/timezone
    - sudo dpkg-reconfigure --frontend noninteractive tzdata

What I was hoping for - 3 dist * 5 env = 15 jobs
However, I’m getting only 5 jobs - For the xenial distribution only (https://travis-ci.com/github/rajat2004/Ubuntu-Setup-Scripts/builds/173785509)
My commit - https://github.com/rajat2004/Ubuntu-Setup-Scripts/commit/cf7e9c9af2db11f1056e1284d51cbeb1490da10f (Earlier use of 10 jobs an be seen in this)

Is there any way to use multiple distributions in a build matrix? I searched but didn’t find any such example.

I’m not very experienced in using Travis, and this is my first time posting here, so please let me know if any more details are required, etc.

Thanks!

2

I am also experiencing the same difficulty when trying to build with multiple distributions. If anyone has an answer to this question, it would be much appreciated. I posted the same question on SO before seeing this. One piece of insight that I can provide is that it doesn’t seem possible to specify dist as a list. See the reference for dist, here, and compare it to, say, the reference for os, here.

If any Travis CI staff happen to read this comment, may I ask why this is the case? And would it be possible to update Travis to use the build matrix for multiple distributions?

3

dist: is not an axis. So you need to create jobs with different dist: manually:

jobs:
  include:
   - dist: xenial
   - dist: bionic
   - dist: focal

See https://travis-ci.org/github/native-api/test_travis/builds/555132125/config for an example.

4

@native-api Thanks for the info! I tried it out, however it’s creating 7 jobs, 5 for xenial (which is not present in config right now) and 1 each for bionic & focal. The matrix is being expanded on xenial only.
Travis build - https://travis-ci.com/github/rajat2004/Ubuntu-Setup-Scripts/builds/178315503
Am I doing something wrong here?

Thanks!

5

Since it’s not an axis, matrix expansion only uses the value specified at global level (you have nothing specified and xenial is currently the default). You’ll need to specify all jobs for bionic and focal by hand under jobs: include:.

You can create a feature request to change that, or perhaps create a pull request against https://github.com/travis-ci/travis-yml (I believe they use it for matrix expansion logic now, at least with build config validation enabled).

6

@native-api Yup, currently doing that only - https://github.com/rsnk96/Ubuntu-Setup-Scripts/blob/master/.travis.yml
I’ve opened an issue - https://github.com/travis-ci/travis-yml/issues/187
Haven’t contributed to Travis before, or actually even coded in Ruby, so let’s see. But would love to contribute, should be fun!

1 Like
7

2 posts were split to a new topic: Specify multiple compilers