Further down in the document , there is a more comprehensive explanation of how cache names are determined. I will not repeat what’s written there.
Does this mean that I cannot make a “shared” cache for multiple jobs?
That is correct. If two jobs differ in any of the factors mentioned, they do not share the cache. The reason here is that, when multiple jobs share the same cache name, the cache can corrupt due to simultaneous access and it can cause a lot of problems.
opened 09:20AM - 07 Apr 17 UTC
caching
important
Is it possible to use the same cache for all builds of a job?
We have a lot o… f dependencies and so we decided to create an extra job which build the dependencies and store it in the cache.
All other (following) builds of that job should use the dependencies in the cache.
opened 07:44AM - 02 Jun 17 UTC
closed 08:04PM - 13 Nov 18 UTC
feature-request
build stages
locked
important
We want to achieve a 2 stage CI:
* A `build` stage that compiles a program. 2 … builds: mac and linux.
* A `test` stage that use the compiled program. Multiple builds: 2 OSes * number of different language versions (e.g. `python: 2.7` and `python: 3.5`).
It is hard to pass the compiled program to the `test` stage. 2 approaches I thought of:
* use Travis's caching: Not feasible since the `language:` / `python:` keys are different between the `build` stage and the `test` stage. The cache wouldn't be shared according to https://docs.travis-ci.com/user/caching#Caches-and-build-matrices. Don't even consider what would happen when there are parallel builds of different commits, or when one restart a build of an older commit.
* use custom S3: It can work in our own repo, however, without exposing our S3 credentials, our forks wouldn't be able to use CI (unless they override the S3 credentials with their own).
It would be great if there is some ad-hoc feature for passing build artifacts across stages.
For reference, the GitLab CI artifacts is pretty perfect: https://docs.gitlab.com/ee/ci/yaml/#artifacts
Their `artifacts` from all previous stages are passed by default, but also can be fine-tuned with `dependencies`.
opened 07:51AM - 26 Aug 18 UTC
closed 01:24AM - 01 Dec 18 UTC
feature-request
caching
I'd like to be able to exclude some `env` vars from the travis cache-name digest… . I'm not asking for cache shared between jobs with different env -- although this feature would make that possible, it's up to the user to make sure the remaining non-excluded env vars result in a unique cache name.
For example when we're caching a `ccache` directory and nothing else, it only contains compiled object files. So it makes sense for the cache-name to depend on `CC`, `CFLAGS`, etc., but not other env vars that have no effect on what goes into the ccache (e.g. `LDFLAGS`). Sometimes we use extra env vars to enable/disable additional post-compilation steps in selected jobs, changing these also unnecessarily invalidates the cache.
I was thinking of adding a `cache_friendly_env` key, or `cache_friendly` under `env`, or some magic prefix to variable name that'd be stripped away before passing to bash, but I think the least intrusive (and less error-prone) way will be to simply list variables that the digest should ignore in cache config:
```yaml
cache:
directories: [ "$HOME/.ccache" ]
env_ignore: [ MAKEFLAGS, LDFLAGS ]
```
all aim to address this issue to varying degrees. We have ideas about how to strike a balance (useful caching strategy while keeping the likelihood of cache corruption to a minimum), but the implementation would still need to be discussed and the actual work prioritized.
1 Like