Job references ignored - build parallelisation ignored

Travis CI Discussions & Feedback
1

Hello,

Since mid of january our build job references are ignored:

- &integration-testing
  name: 'Integration Testing'
  script:
    - bash ./test/e2e.sh
  addons:
    apt:
      packages:
        - libgconf-2-4
    chrome: stable
  before_install:
    - google-chrome-stable --headless --disable-gpu --remote-debugging-port=9222 http://localhost &
  before_cache:
    - rm -rf ~/.cache/Cypress/cy/production/browsers
- <<: *integration-testing
- <<: *integration-testing

Which means, that the integration-testing job should exist 3 times, since a specific time, this is not working anymore for us.

Working (3 integration): https://travis-ci.org/SAP/luigi/builds/638034713
Not working: https://travis-ci.org/SAP/luigi/builds/639864437

I just tried to run the exact same config on my fork and it seems this feature is not working anymore.

Anything you can do about it? Code duplication would help of course, but is not beautiful.

Best, Markus

2

We are now validating your configuration


and remove duplicates. Can you tell me why you are trying to run 3 identical jobs?

3

The job itself is identical, but not what the job does under the hood.
Maybe you should add a flag for jobs, that the duplicates are not being removed.

The three identical jobs are communicating with cypress dashboard and the dashboard tells the testrunner which tests it should run next and optimizes build time.

4

You can add a name attribute to these jobs to distinguish them.

1 Like
5

I’m a bit curious about the “optimization” you are after.

For example, https://travis-ci.org/SAP/luigi/builds/655439290 had one “Integration” test that ran in 5.5 minutes. Presumably running all of the jobs that the 3 split ones achieved.

https://travis-ci.org/SAP/luigi/builds/638034713 has 3 integration tests, ranging in time from 3 minutes to 5. If they start simultaneously, you win a little, but if they don’t (there is not such guarantee in a distributed system), you would be waiting for the last one to finish. I don’t know the internals of your tests; the last one may not have any work to do at all, by the time it starts. If this is the case, you will have wasted a whole lot of time by running 3 jobs.

6

Thanks, I also stumbled upon advanced yaml usage, and this is currently waiting to be built.

- name: 'Integration Testing 1'
  <<: *integration-testing

- name: 'Integration Testing 2'
  <<: *integration-testing

I understand to your point, since the bootup and cache-rehydration is taking around one minute, which is already quite some time. On the other hand Cypress Dashboard is clever enough and hands out jobs as long as there are one - if only one job is running, this runner is doing all the work. If another worker joins later - it gets also something to do until the e2e test queue is finished.