Travis-CI here-string automated input = broken

Travis CI Discussions & Feedback
1

Hi,

I’m trying to use Travis-CI for a Bash project, but some insanely stupid issues are ruining everything. The project is at https://travis-ci.org/github/l3laze/sind.

Running on Travis is eating here-string input meant to automate testing, meaning I need to ignore at least half of the tests or let it fail on them…even though it runs with 100% coverage locally.

I’ve tried to setup Docker to run locally, but I’m running Ubuntu 18 on top of Android 8 via UserLAnd, and the official Docker installation instructions fail + unofficial workarounds for that failure also fail. Lol. This is my “I’m really impressed by this highly-recommended junk” face: =|

Thank you,

Tom

2

You didn’t mention where exactly it’s wrong so I assume it’s somewhere in your Bash scripts.

I suggest you trace them to see what’s happening and where exactly it goes wrong.

https://wiki.bash-hackers.org/scripting/debuggingtips#making_xtrace_more_useful

Problem diagnostic is always done via the same procedure:

  1. Find the exact place in the code where the problem manifests itself
  2. Examine the program’s state at that moment to find out which of the involved values is wrong
  3. Trace the wrong value back to its origin

Bash’s tracing with the inproved PS4 as per the link allows to do all three: it shows lines and commands with expansions complete and it shows all variable assignments.

3

This was easily solved with effectively no changes by switching to GitHub for CI. It seems to be an issue with Travis at it’s core, but I have no interest in Ruby and so am not about to try to dig into the cause of this.

I tried tracing both by running as bash -x and with set -x ... set +x wrapping the problematic code, with and without PS4 set for better tracing. Also tried with set -euo pipefail to see if that would force it to stop with an error. I even went so far as to run part of the problematic code from the tests directly in .travis.yml. Nothing helped, lol.

Part of the effect of this was that it ignored/mis-reported coverage for parts that had to have been run. For example the install script was running twice to test installing from the local repo (for development) and from GitHub Releases. Coverage reports said it was run only once though despite the fact it worked fine locally, and even after I customized the test for Travis.

On the other hand, GitHub CI not only ran the code, but all those tests Travis had to have ignored as it couldn’t run them “just work”, AND I don’t need any custom tests for CI there.

I’ve been using Travis as a CI for many of my projects for almost 4 years now. Countless hours have gone to attempting to get it working properly rather than development of my projects. After the above problem and an issue last night trying to setup CI/CD for kcov and finding I have to install at least Ruby + travis-cli to do so, I’m all done. It was fine while it was all I could use, but I’d prefer to concentrate on my projects rather than finding workarounds for problems with someone else’s.

4

I’m sorry to hear that you’ve had so much problem working with Travis CI.

In general, in .travis.yml, we discourage use of exit, as you do here:

This terminates the build immediately, without any subsequent cleanup.

5

That exit is not the problem as the issue occurred long before it was added. That was only used to prevent it from running further while I was trying to fix it as the extra output from running both with and without coverage made it more difficult to read the results.