GitHub status not posted on commits on repositories using legacy Service integration

https://www.traviscistatus.com/incidents/rx6fhs3wqcln

We are investigating the reports of GitHub commit status not being posted.

The issue only affects those repositories using the legacy Services integration (i.e., all .org repositories and some .com ones), and only under some circumstances. You can migrate your repositories to GitHub Apps on https://travis-ci.com/account/repositories.

Please subscribe to the incident above to get updates on this.

@BanzaiMan, I am observing this issue in a project using GitHub Apps and it started about 2 days ago:

Since I know very little how Travis integrates with Github, I cannot judge whether the problem is Github or how Travis updates Github.

Note: This project might have been using legacy services (I donā€™t know anymore), but was migrated from travis-ci.org to travis-ci.com 3 days ago.

[Update] Oops, I was not aware that the commit status API is deprecated: https://blog.travis-ci.com/2018-09-27-deprecating-github-commit-status-api-for-github-apps-managed-repositories

This is quite strange.


shows me a failure.

Thanks,
as written in my post above: I used a deprecated API and this broke when migrating the job to travis-ci.com.

Iā€™m sorry, but I donā€™t understand the issue, then. The status is clearly posted on GitHub. That GitHub API doesnā€™t return that status seems a GitHub issue to me. Did I miss something?

@BanzaiMan, no issue on Travis side, but a misunderstanding on my side.

I observed issues with Github/Travis 2-3 days ago and migrated a job from travis-ci.org to travis-ci.com because I thought that might fix it. Thus, the repo moved to Github Apps. But now I could no longer see commit status updates via the Github API (because these are no longer updated by Travis). This behavior is explained in https://blog.travis-ci.com/2018-09-27-deprecating-github-commit-status-api-for-github-apps-managed-repositories but I was not aware of it. I read the linked blog post after writing my post above.

So my migration created new problems that I had not anticipated and that looked similar as the problems we observed with Github/Travis in the last days. That was an unhappy coincidence.

I hope this is clear now. Sorry for the confusion.

1 Like

Hi @BanzaiMan, I think a repository I contribute to may still be affected by this issue. Iā€™m not an owner, so cannot dig very deep. Could you confirm? Two PRs never seem to update their status from ā€œPending ā€” The Travis CI build is in progressā€:

Thanks much!

@BanzaiMan The two links I shared in my previous comment are now merged, so here are some new ones:

We have a few more details to share on this subject.

In the last couple of weeks, we have observed an increased number of failures to post statuses on GitHub. The sources of the increase itself remains unclear, but the repeated problems prompted us to clean up some wasteful requests. This helped a little, but not entirely; at the height of the user activities, we still saw some errors.

We reached out to GitHub, and was granted some additional (but temporary) capacity so that our services can continue. This was yesterday (April 2nd).

We are continuing to reduce waste, so that the issue is resolved for good.

It is worth repeating that the issue applies only to Legacy integration. GitHub Apps should not exhibit this issue; if you have evidence to show otherwise, please open a separate issue.

2 Likes

@BanzaiMan do you have any notes / docs on migrating from legacy to github apps?

1 Like

@BanzaiMan Hi, was this ever fixed? I have a repo using the legacy services, and I see that https://www.traviscistatus.com/incidents/rx6fhs3wqcln was marked as resolved. I believe Iā€™m still seeing issues such as https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/2195 where the build runs in travis https://travis-ci.org/github/GoogleCloudPlatform/PerfKitBenchmarker/builds/673227140 but the commit status is not reported back to the commit / PR. Is the solution here to migrate from legacy to GitHub Apps? Thanks so much!

As I stated earlier, the incident concerns reaching the GitHub API rate limit. That issue is resolved, thanks to the (temporary) raise granted by GitHub.

There are other cases where we fail to post statuses for various reasons (chief among them is having tokens being out of sync). These may still occur, but these would have occurred even before the incident.

I encourage you to do so.

2 Likes

Hey, I was having this issue, and so I figured Iā€™d migrate my open source projects to .com. When I go to https://travis-ci.com/account/migrate and try to migrate my existing repositories, I see this error pop up in the console when I hit ā€˜migrateā€™. Anyone else getting this?

Iā€™m also experiencing this problem on several open source projects I maintain of status checks not being posted to GitHub even though the build was triggered and completed successfully.

Additionally, Iā€™m unable to access any of the settings for all repositories located in organizations I manage, so I canā€™t even really troubleshoot. My membership in these organizations only appears in the new travis-ci.com UI and not the travis-ci.org UI.

There is no incident for this even though it definitely appears to still be affecting folks, any help here?

Example PRs without a status but with a Travis build:


Iā€™m also seeing status updates not getting reported. I tried to migrate and hit a 500 error during migration as well.