We have a Python project with 13 jobs. Generally these jobs take <10 minutes (often around 5-6) each, as seen here: https://travis-ci.com/github/nipy/nipype/builds/190839373
About 2 days ago, our jobs suddenly started failing with “The job exceeded the maximum time limit for jobs, and has been terminated.” after 50 minutes: https://travis-ci.com/github/nipy/nipype/builds/191039325
Note that these were cron jobs, so no code changes occurred between the two jobs.
I assumed this was an individual test timing out, possibly due to a recently release dependency, but I was unable to find a candidate. I also restarted the failing job today and watched it on and off, and it was simply going extremely slowly.
Has there been a change in resource allocations? Is it possible to get some advice on things to try to get our build times back into a reasonable range?