# A Rust job somehow uses wrong compiler version installed by another job **URL:** \ **Category:** Travis CI Discussions & Feedback\ **Tags:** caching, build-matrix\ **Created:** [June 22, 2020, 9:59am UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081 "2020-06-22T09:59:45Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![RalfJung](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/ralfjung/32/2036_2.png) [@RalfJung](https://travis-ci.community/u/RalfJung)\ **Post date:** [June 22, 2020, 9:59am UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/1 "2020-06-22T09:59:45Z")
Something is going very wrong with [this job](https://travis-ci.com/github/Gilnaa/memoffset/jobs/352063114): it says that the rustc version is `rustc 1.46.0-nightly (4fb54ed48 2020-06-14)`, which is 1 week outdated, but at the same time `rustup update` says ```auto nightly-x86_64-unknown-linux-gnu unchanged - rustc 1.46.0-nightly (a8cf39911 2020-06-21) ``` To me it looks like whatever caching travis is doing when I say `- rust: nightly` has gone seriously wrong. ---
**Author:** ![RalfJung](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/ralfjung/32/2036_2.png) [@RalfJung](https://travis-ci.community/u/RalfJung)\ **Post date:** [June 22, 2020, 10:10am UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/2 "2020-06-22T10:10:27Z")
Wow, even after adding ```auto # Make sure we really have the latest nightly. - rustup toolchain uninstall nightly - rustup toolchain install nightly - rustc -Vv ``` I still get a nightly from a week ago. How is that even possible? ---
**Author:** ![RalfJung](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/ralfjung/32/2036_2.png) [@RalfJung](https://travis-ci.community/u/RalfJung)\ **Post date:** [June 22, 2020, 10:16am UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/3 "2020-06-22T10:16:14Z")
Oh that is odd, I somehow have two toolchains: ```auto $ rustup show Default host: x86_64-unknown-linux-gnu rustup home: /home/travis/.rustup installed toolchains -------------------- nightly-2020-06-15-x86_64-unknown-linux-gnu (default) nightly-x86_64-unknown-linux-gnu active toolchain ---------------- nightly-2020-06-15-x86_64-unknown-linux-gnu (default) rustc 1.46.0-nightly (4fb54ed48 2020-06-14) ``` (these extra newlines are just how travis copy-and-paste works unfortunately…) ---
**Author:** ![RalfJung](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/ralfjung/32/2036_2.png) [@RalfJung](https://travis-ci.community/u/RalfJung)\ **Post date:** [June 22, 2020, 11:01am UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/4 "2020-06-22T11:01:52Z")
I ended up with this work-around: ```auto # Make sure we really have the latest nightly. # For some reason Travis puts us on a snapshot by default... - rustup default nightly - rustc -Vv - cargo -V ``` ---
**Author:** ![native-api](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/native-api/32/430_2.png) [@native-api](https://travis-ci.community/u/native-api)\ **Post date:** [June 22, 2020, 3:05pm UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/5 "2020-06-22T15:05:04Z")
> [@RalfJung](#): > > `nightly-2020-06-15-x86_64-unknown-linux-gnu (default)` It looks like this default was set by your script, `ci/miri.sh`, in an earlier build, [Travis CI - Test and Deploy with Confidence](https://travis-ci.com/github/Gilnaa/memoffset/jobs/351695488#L255). [Rustup installs versions into `~/.cargo/`](https://stackoverflow.com/questions/46739842/where-does-rustup-install-itself-to), so everything that you change there is cached between builds. As such, cleaning your cache and avoiding messing up Travis’ stock functionality without knowing what you are doing should fix the problem. ---
**Author:** ![RalfJung](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/ralfjung/32/2036_2.png) [@RalfJung](https://travis-ci.community/u/RalfJung)\ **Post date:** [June 22, 2020, 3:19pm UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/6 "2020-06-22T15:19:39Z")
> [@native-api](#): > > It looks like this default was set by your script, `ci/miri.sh` , in an earlier build, [Travis CI - Test and Deploy with Confidence](https://travis-ci.com/github/Gilnaa/memoffset/jobs/351695488#L255). That’s a different job though? It’s the Miri job, not the all-features job. Are caches shared across all jobs…? > [@native-api](#): > > without knowing what you are doing Travis [doesn’t provide a good way to _compute_ the nightly version](https://github.com/travis-ci/travis-ci/issues/10002) we need, so there’s no avoiding messing with `rustup` unfortunately. I know pretty well what I am doing with `rustup`, but I didn’t expect travis to confuse the caches of different jobs. ---
**Author:** ![native-api](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/native-api/32/430_2.png) [@native-api](https://travis-ci.community/u/native-api)\ **Post date:** [June 22, 2020, 3:35pm UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/7 "2020-06-22T15:35:05Z")
I didn’t check if that was the exact job that poisoned the cache or not (I’ll look into that now). This is just very strong evidence that you are doing something funny and as a result, are yourself responsible for diagnosing the resulting errors rather than (or at least before) reporting them as Travis bugs. A build log reports cache IDs that jobs save and restore so you are able to pull the thread back from a faulty job. If you don’t want Travis’ language installation logic for whatever reason, use [`language: generic`](https://docs.travis-ci.com/user/languages/minimal-and-generic/) and install all the necessary tooling yourself ([`generic` image doesn’t have Rustup](https://travis-ci.org/github/native-api/test_travis/builds/700931576) but you can install and cache it). `nightly` by definition means “rolling release” so it’s logical that Travis doesn’t allow to select a specific build. ---
**Author:** ![RalfJung](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/ralfjung/32/2036_2.png) [@RalfJung](https://travis-ci.community/u/RalfJung)\ **Post date:** [June 22, 2020, 3:39pm UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/8 "2020-06-22T15:39:03Z")
I don’t have any idea how to go about debugging this, but my theory for now is that indeed state somehow got mixed up between jobs, so I changed the other job to set the default back to `nightly` before the cache gets created. Thanks for pointing that out! > [@native-api](#): > > `nightly` by definition means “rolling release” so it’s logical that Travis doesn’t allow to select a specific build. There are various good reasons for not wanting to always test the _latest_ nightly, but some specific nightly determined e.g. by the contents of some file. > [@native-api](#): > > If you don’t want Travis’ language installation logic for whatever reason, use [`language: generic`](https://docs.travis-ci.com/user/languages/minimal-and-generic/) and install all the necessary tooling yourself ([`generic` image doesn’t have Rustup](https://travis-ci.org/github/native-api/test_travis/builds/700931576) but you can install and cache it). Yeah I did that elsewhere, also because Travis tends to cache a bit too much for Rust. But that always feels like a hack. ---
**Author:** ![native-api](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/native-api/32/430_2.png) [@native-api](https://travis-ci.community/u/native-api)\ **Post date:** [June 22, 2020, 4:31pm UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/9 "2020-06-22T16:31:36Z")
> [@RalfJung](#): > > That’s a different job though? It’s the Miri job, not the all-features job. > [@RalfJung](#): > > I don’t have any idea how to go about debugging this, but my theory for now is that indeed state somehow got mixed up between jobs > [@native-api](#): > > A build log reports cache IDs that jobs save and restore so you are able to pull the thread back from a faulty job. Here’s the thread pull: - Faulty job. Pull Request #44 [#164.8 failed](https://travis-ci.com/github/Gilnaa/memoffset/jobs/352063114), June 22, 2020 11:53:29, rustc 1.46.0-nightly (4fb54ed48 2020-06-14), [cache ID `PR.44/cache--linux-xenial-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855--cargo-nightly`](https://travis-ci.com/github/Gilnaa/memoffset/jobs/352063114#L168) – takes it from PR - 1st build in this PR. Pull Request #44 [#162.8 failed](https://travis-ci.com/github/Gilnaa/memoffset/jobs/352048464), June 22, 2020 11:11:51, rustc 1.46.0-nightly (4fb54ed48 2020-06-14), cache ID `master/cache--linux-xenial-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855--cargo-nightly.tgz` – takes it from `master` - Last `master` build before that. master Merge pull request [#43](https://github.com/Gilnaa/memoffset/issues/43) from RalfJung/raw\_ref, [#161.8 passed](https://travis-ci.com/github/Gilnaa/memoffset/jobs/351701982), June 20, 2020 16:40:24, **rustc 1.46.0-nightly (2d8bd9b74 2020-06-19)**, no cache, [saves cache ID `master/cache--linux-xenial-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855--cargo-nightly`](https://travis-ci.com/github/Gilnaa/memoffset/jobs/351701982#L261) - **So cache poisoning happens somewhere between 161.8 and 162.8.** Let’s take a closer look at build 161, especially at Miri’s handiwork. - Miri job from 161. master Merge pull request [#43](https://github.com/Gilnaa/memoffset/issues/43) from RalfJung/raw\_ref, [#161.1 failed](https://travis-ci.com/github/Gilnaa/memoffset/jobs/351701975), [sets default to `nightly-2020-06-15-x86_64-unknown-linux-gnu`](https://travis-ci.com/github/Gilnaa/memoffset/jobs/351701975#L241)… **[saves cache ID `master/cache--linux-xenial-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855--cargo-nightly`](https://travis-ci.com/github/Gilnaa/memoffset/jobs/351701975#L643)** * * * So **jobs 161.1 and 161.8 use the same cache ID!** That’s because their configuration only differs in job name and script. [Cache ID is computed from OS image, language version and environment variables as specified in `.yml`](https://docs.travis-ci.com/user/caching#caches-and-build-matrices) (see [What is the cache ID for a job generated from? How can staged jobs use the same cache?](https://travis-ci.community/t/what-is-the-cache-id-for-a-job-generated-from-how-can-staged-jobs-use-the-same-cache/884) for source code links). **Job name and script are not considered!** * * * **It looks like whoever was using `MIRI=` envvar in [Travis CI - Test and Deploy with Confidence](https://travis-ci.com/github/Gilnaa/memoffset/builds/172318113)** (huh, [that was actually you](https://github.com/Gilnaa/memoffset/commit/3da754e54ba6dffacf4a88d83be31e027bd28cf9)) **was doing that for a reason!** But they forgot to leave a comment in `yml` explaining their decision, letting that knowledge become lost! ---
**Author:** ![RalfJung](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/ralfjung/32/2036_2.png) [@RalfJung](https://travis-ci.community/u/RalfJung)\ **Post date:** [June 23, 2020, 6:48am UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/10 "2020-06-23T06:48:58Z")
> [@native-api](#): > > **It looks like whoever was using `MIRI=` envvar in [Travis CI - Test and Deploy with Confidence](https://travis-ci.com/github/Gilnaa/memoffset/builds/172318113)** (huh, [that was actually you](https://github.com/Gilnaa/memoffset/commit/3da754e54ba6dffacf4a88d83be31e027bd28cf9)) **was doing that for a reason!** But they forgot to leave a comment in `yml` explaining their decision, letting that knowledge become lost! I added it because the other jobs did, and I _think_ it was just a hack to get the name to display, but I might be wrong. Anyway the `MIRI` env var actually became meaningful for the Miri tool, so we had to stop using it. > [@native-api](#): > > That’s because their configuration only differs in job name and script. [Cache ID is computed from OS image, language version and environment variables as specified in `.yml`](https://docs.travis-ci.com/user/caching#caches-and-build-matrices) (see [What is the cache ID for a job generated from? How can staged jobs use the same cache?](https://travis-ci.community/t/what-is-the-cache-id-for-a-job-generated-from-how-can-staged-jobs-use-the-same-cache/884) for source code links). **Job name and script are not considered!** Thanks a lot! That is very helpful. So I’ll add some dummy env var then to ensure the cache is separate. ---
**Author:** ![native-api](https://sea1.discourse-cdn.com/flex015/user_avatar/travis-ci.community/native-api/32/430_2.png) [@native-api](https://travis-ci.community/u/native-api)\ **Post date:** [June 23, 2020, 3:10pm UTC](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/11 "2020-06-23T15:10:34Z")
I created a feature request to elimitate this source of hard-to-catch errors: > [@Use all parameters of a job to calculate cache ID; retire "shared caches"](https://travis-ci.community/t/use-all-parameters-of-a-job-to-calculate-cache-id-retire-shared-caches/9105): > > In [A Rust job somehow uses wrong compiler version installed by another job](https://travis-ci.community/t/a-rust-job-somehow-uses-wrong-compiler-version-installed-by-another-job/9081/9), it turned out that in this unassuming config: matrix: include: - name: miri rust: nightly os: linux script: - sh ci/miri.sh - name: all-features rust: nightly os: linux script: # `--lib` added to prevent doctests from being compiled. # This is due to `unstable_const` requiring extra `feature(...)` directives # which the doctests do not have. - cargo te… You may wish to upvote it if you are interested in seeing it implemented