How to skip jobs based on the files changed in a subdirectory?

#1

I am working at a company who use the monorepo pattern to store their code. Each sub project has its own test and deploy job so every push to this repo will blindly test and deploy all the projects. In order to reduce build times I would like to only build the projects within which files have changed. Sadly this does not seem to be possible using the Travis conditional syntax as that only works on prefixed meta data (such as the branch name etc).

Ideally I would be able to run a script, as early as possible in the job lifecycle, which performs a git diff inside each projects sub directory and skips the jobs as required. I have managed to get something working by copying the example in this repo - https://github.com/arnaskro/monorepo-travis.

A snippet of our slightly modified job definition looks like the following…

jobs:
  include:
  - stage: "Test"
    name: "common"
    before_install: if devops/check_for_changes.sh $TRAVIS_COMMIT_RANGE common; then echo "Testing"; else echo "NOT testing" && travis_terminate 0; fi
    install:
      - cd common
      - npm install
      - cd ..
    script:
      - cd common
      - npm test
    cache:
      npm: true
      directories:
        - common/node_modules

It seems before_install is the earliest lifecycle phase I can hook into but this only happens after the environment language has been installed (in our case node via nvm) and the build cache restored. This means the minimum time to determine the job is not necessary is ~1 minute. We don’t need the language or cache to determine whether the job is necessary or not.

Is there a way to abort the job earlier than the before_install phase?

Or is there a better way to achieve the same thing? I considered…

  • a preliminary stage that runs the git diff and stores a record of the subprojects to build or not as env vars for later stages to consume
  • using the Travis API to trigger child builds for each of the subprojects that need building. I am not sure if that would work with Github build reporting.

Any advice would be greatly appreciated!

#2

This capacity does not exist at the moment. The previous discussion is found in

#3

Ok no problem. Thanks BanzaiMan.

A second scenario would be to try running a preliminary stage to execute the git diff which records the jobs to run on the environment. Can I set an environment variable on one stage and have it available in a later stage?