macOS build system behavior + documentation confusion leading to all build failures

Currently, we are using the following .travis.yml configuration:

language: objective-c
osx_image: xcode12.5
install:
  - gem install bundler
  - bundle install
script:
  - bundle exec ./travis_controller.sh --verbose

By using this configuration, we seem to notice that Travis implements behavior where it will automatically pick the latest macOS version to be used for the build system, despite documentation setting a different expectation.

We notice this, because we are encountering the following realworld scenario:

  1. New macOS version becomes available
  2. Travis auto-selects newest macOS version for build system (per behavior described above)
  3. Suddenly, all builds start to fail with the following error, because rubies are not available on the new, edge OS version:
Required ruby-2.6.5 is not installed - installing.
curl: (22) The requested URL returned error: 404 Not Found
Searching for binary rubies, this might take some time.
Requested binary installation but no rubies are available to download, consider skipping --binary flag.
Gemset '' does not exist, 'rvm ruby-2.6.5 do rvm gemset create ' first, or append '--create'.
The command "rvm use $(< .ruby-version) --install --binary --fuzzy" failed and exited with 2 during .

To add some context from the user perspective: I built the provided .travis.yml file using the Travis documentation from here:

To me, I interpret that table to mean that specifying an osx_image value of xcode12.5 will result in macOS version 11.3 being selected for the build environment.

However, according to real world experience, it appears macOS version 11.4 started to get picked for the build system starting 3-4 days ago (and lead to the second-order problem of rubies suddenly not being available and all builds failing).

Is there some way this could all be clarified? Or perhaps I’m misinterpreting something?

Hey,

We have replied to the thread from the ticket that you have opened to suppor@travis-ci.com. We will be following up from there.

Thanks.