There are plenty of other posts discussing this; I tried to summarize them in the past here: Multiarch builds (lxd) always time out on failure instead of exit
The only thing that worked for me in all cases was to add || travis_terminate $? to all script lines (and to make sure there was no set -e in my .travis.yml anywhere). See here for the result: https://github.com/gap-system/gap/blob/master/.travis.yml#L184
Even without that, there are so many issues with arm64/ppc64le/s390x support in Travis, and absolutely no support (these issues have been reported many times now over several months, with ZERO reaction by the Travis team), that I am seriously considering to drop this again (as I have to constantly restart builds for these architectures which suffer from various temporary failures, e.g. fluke "quota exceeded error). Instead, I’ll look into using GitHub Actions with qemu – I just discovered https://github.com/marketplace/actions/run-on-architecture and will give a try.