Continuing the discussion from Introducing Workspaces:
For workspaces and caches, we should accept relative directories to make life easier.
To test it, add:
cache:
branch: paths-relative
Note that use of environment variables (e.g., $TRAVIS_BUILD_DIR) in directories specifications will not work across OSes because those values will be different on different OSes.
Many caching shortcuts implicitly use $HOME, ~, and other variables. This includes
ccache for C and C++ (~/.ccache)elm for Elm ($HOME/.elm)npm for Node.js when Yarn is in use ($HOME/.npm)cocoapods for Objective-C/Swift when the podfile option is in use (this is probably not much of an issue, because it is only runs on the Mac)pip for Python ($TRAVIS_HOME/.cache/pip)packages for R ($R_LIBS_USER)cargo for Rust ($TRAVIS_HOME/.cargo and others)It seems to me accommodating these would be some work, and that, in general, we expect these files to be platform-dependent. Consequently, advise users that the cross-platform files should be cached using relative paths instead of these shortcuts.
Hey is there any update on this issue?
I’m trying to set up a cross-plaform build, where I have a build stage on multiple operating systems and a single deploy stage that uploads all build artifacts from all jobs. Basically like in the example: https://docs.travis-ci.com/user/using-workspaces/#multiple-workspaces-example
Do I have to use 3 workspaces here, one each for windows, linux, macos? How to I know where the loaded workspaces end up in the deploy stage? Example:
jobs:
- stage: build
os: windows
workspaces:
create:
name: build
paths: ./build
script: ./build_artifacts.sh
- stage: deploy
os: linux
worspaces:
use: build
deploy: ...
Can I access the windows build result in the linux deploy job? Where does it end up? It’s not placed in ./build unfortunately.
Can I access the windows build result in the linux deploy job?
In my experience at least for now workspaces are not well shareable between the different systems/OS.
Ok, I found where it puts them. 
If the workspace path on Windows is: /c/Users/travis/build/name/repo/workspace, on linux it will be put into ~/C:/Users/travis/build/name/repo/workspace.
Well. So a horrible workaround is possible 
Would be nice to know if there’s a roadmap for getting the workspaces out of beta though. Maybe anyone from the dev team on here can provide some info?
These cases can be worked around as I did experimentally here. I believe support for ~ would be a nice to have though, as I believe relative paths and home folder together cover the large majority of common cases for multiplatform caching across jobs.
I’m not so sure whether ~ support would be good. The stuff that programs save into ~ is usually OS- and even environment-specific.
Since casher runs tar with -P, you can save stuff from homedir with paths like ../.program anyway.
I’m not so sure whether ~ support would be good. The stuff that programs save into ~ is usually OS- and even environment-specific.
It would be very useful and would allow to significantly simplify the config/workflow and speed up things in some cases.
They are sometimes environment and OS specific, and sometimes not. Consider for instance Gradle and Maven, I currently have to resort to horrible kludges, when I could simply add ~/.m2 or ~/.gradle/caches to my workspace.
I frankly believe that willingly making it harder to cache stuff because in some systems stuff cached is os-specific is a bad choice. There is no principled reason for files in the local project folder to be portable across OSs/architectures and files in the user home to be not, in my opinion.
Workspaces are not for caching stuff, cache is. Workspaces are for transferring build artifacts between jobs. And build artifacts virtually always reside under the working directory.
The feature is still incomplete since transferring build artifacts between multi-OS jobs doesn’t work. Use case: I need to prepare some assets on Linux (which is impossible to do on Windows) and then share those assets to consumers-like Win/Mac/Linux jobs.
Except that I often (ab?)use them for caching, as I want my first job to fetch dependencies, and subsequent jobs to use them without fetching them over and over.
I do not want persistence across builds, I want it across jobs.
I believe it’d make sense to consider workspaces a way to share data across jobs, rather than a way to share build artifacts alone. Let users decide which kind of data is meaningful to share, rather than deciding for them and thus forcing the application of exotic workarounds.
See, for instance, this build. I got a ton of platforms to test on, I do not want to cache across builds, and yet it makes little sense to fetch the same java packages over and over. I am thus using workspaces to share ~/.m2, and the horrible kludge I needed to resort to was to copy such folder into the local working directory, and then in other jobs’ before_install phase move it again where it belongs.
It does not seem to me that I am doing anything unreasonable or fragile, and I achieve a consistent speedup (from 2h40m of machine time per build to 1h30m).
If supporting ~ would end up in a bloodbath implementation wise I’d fully understand to be reluctant at supporting it, but to me willingly not having such support just because it’s not the most common use case is a bit of a weak justification.
It is tricky. See Use relative paths on files by BanzaiMan · Pull Request #48 · travis-ci/casher · GitHub.
~ needs to be somehow expanded for use in tar, but in a way that’ll restore stuff to appropriate locations in different OSes.
It looks to me that we’ll either have to expand it to ../ (or calculate the relative path between workdir and $HOME on the fly in case it changes in the future), or expand it differently for cache and for workspaces (then the expansion will need to be done in travis-build instead).
It’s also unclear what to do with entries like $HOME/<stuff>.
Would a relative-to entry with support for env variables help? You could unpack the cache into an arbitrary folder, then move contents where such key suggests.
workspaces:
create:
- name: foo
paths:
- '.m2'
- '.gradle'
relative-to: $HOME
...
workspaces:
use:
- name: foo
relative-to: $HOME
What if there’s a combination of absolute and relative paths in the list?
They should be in two different workspaces to work correctly. If a workspace is marked as “relative-to” something, then it is relative to that folder.
It is also possible to immplement solely the relative version of workspaces, a workspace with absolute paths is simply relative-to: '/', which could be the default behavior, and mimick the current implementation (so I think there will be no retrocompatibility issues, but for the branch: paths-relative)
In any case, one workspace per “root”. How does it sound?
Due to this issue, my travis-ci build workflow is not efficient enough which might potentially cause early OSS/free tier minutes exhaustion (1000 minutes per month if I got it right, see new pricing model announce in https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing). The issue also holds me from using travis-ci for building Windows app package since app building jobs depend on Linux-specific preparing-like job (so I still have to use Appveyor for Windows build). So it would be nice if I could prepare the assets by running a single Linux job and then just share the respective workspace across all the multi-OS jobs.