Builds stuck in Received state

[faxu]

We’re in the process of preparing a release and since yesterday afternoon, our builds in one repo have been stuck in the “Received” state, not even advancing to queued. https://travis-ci.org/github/onnx/wheel-builder/pull_requests

We saw there is a notice on traviscistatus.com that notes “Experiencing Issues with GitHub API Requests” and “under maintenance.” If this the likely cause of our issue, and is there an ETA for resolution?

Our release timeline may be impacted by this, so we’d like to get as much info as possible to troubleshoot. Thank you!

[hugovk]

This is affecting many people as Travis is moving infra from .org to .com.

You may be best migrating over to .com, especially if it’s delaying your release, but note build history may or may not be migrated too:

Builds hang in queued state

Unfortunately, this might happen as we move infrastructure over from .org to .com - if we see a job stuck in a queue for more than 16 hours, we will review this on a case by case basis and possibly cancel the job to ensure resources are not being reduced across the platform. You may want to consider migrating across to .com sooner rather than later, or consider building at a quieter time of the day if remaining on .org for a while longer.

Will builds' logs be migrated upon .org -> .com migration?

It’s still the case that history is not fully migrated. The old builds are shown, but not their logs. Is there a target for when it will be migrated? When ready, will already-migrated repos have the history back-filled, or will only new migrations have history migrated?

[askhade]

What is the expected timeline for this? Is this temporary (couple of more hours) ? The FAQs suggest the migration is planned to finish by Dec 2020 so are we to expect such degraded behavior till then?

[hugovk]

I’ve no idea, but they talk of moving infra from .org to .com, so I get the impression it’s not going to improve much on .org, unless a lot of people migrate to .com and free more resources on .org? But then will more infra be moved to .org?

[SlySven]

At present it is not clear that moving to the .com will improve the build waits - the statistics shown (on the number of Linux jobs being run) on https://www.traviscistatus.com {note the “.com” in the URL!} make for depressing reading over the last week.

Obviously not moving won’t be good in the long term (beyond the end of 2020) as then the .org site will then only be usable to see how historic builds went.