Lets say in first stage we are building our code and pushing artifacts out. In second stage - we are deploying and running FVT.
Both core code and FVT are based on the same git repo and should be in lockstep, i.e. FVT tests should be compatible with the compiled code.
What we’re seeing is build stage checks out merge preview with the state of master at the beginning of the stage.
In the middle of that stage, master gets updated. Since travis always looks at the tip of master - the FVT stage now checks out the merge preview with the NEW head of master.
In the case where new head of master contains new FVT tests that are incompatible with the code compiled in build stage - we get a build failure.
This is a completely illegitimate failure, since its essentially a mismatch between compiled code and FVT code. What should be happening - the code tested across stages should be the same code, same merge preview, otherwise this introduces all sorts of instabilities into the build.
We can work around this behaviour with Git tags and all, but it seems like we’re creating functionality that should be in core product.
looks like you are committing build artifacts into the source control.
@native-api I am not sure where you got that from. We are definitely not committing build artifacts into source control.
What I am saying is currently different stages in the build can get different versions of the PR merge if:
PR branch changes.
Master changes.
That means different stages of the build are getting different source code versions which can cause major instability of the build.
In regards to workspaces - unfortunately our enterprise has not really caught up with the version that contains that feature But it would not really solve the above problem anyway. I mean we can pass the repo folder from the first stage to the rest of the stages to preserve its state - but it would be a clunky solution at best. In my mind Travis should be preserving the understanding of the pr state from the build start - and making sure that every job gets THAT version of the pr, not an updated one. Otherwise build stages/jobs may end up with incompatible artifacts.
AFAICS, the problem is that Travis fetches a “merge preview” commit autogenerated by Github, +refs/pull/<PR#>/merge.
Which is regenerated by Github whenever the PR branch or the base branch changes.
This is a legitimate bug – but I don’t know whether it’s possible to get this commit any other way that would be protected from this effect.
Since you are an enterprise customer, you can report this bug directly to Enterprise support at enterprise@travis-ci.com and refer to this topic.
Temporary workaround consist of realizing merge commit +refs/pull/<PR#>/merge as a temporary branch in the first stage of the build. Subsequent stages check out this branch, rather than +refs/pull/<PR#>/merge. After some period of time - this branch is cleaned up.
But it would not really solve the above problem anyway. I mean we can pass the repo folder from the first stage to the rest of the stages to preserve its state - but it would be a clunky solution at best. In my mind Travis should be preserving the understanding of the pr state from the build start - and making sure that every job gets THAT version of the pr, not an updated one. Otherwise build stages/jobs may end up with incompatible artifacts.