Running Docker containers in parallel with tests

Travis CI Discussions & Feedback
1

Hello,

I’m very new to this - so please excuse my noobie questions. I feel like I’ve misunderstood something along the way.

What I’m trying to achieve is run 2 docker containers (to replicate the current infrastructure setup) on a new docker network. One docker container will be mysql:8 and the other will be a custom image extending wordpress:php7.4-apache. I’ve managed to setup volumes on both containers and extracted live backup data so the running containers have the live data.

What I need to do now is run Selenium against it. The problem I have is that, because apache is running as a foreground process, it doesn’t seem to get past it.

This is my .travis.yml file

services:
  - docker
  
before_install:
  - sudo apt-get update
  - sudo apt-get -y install p7zip-full jq bsdtar gzip wget
  - echo "$DOCKER_TOKEN" | docker login -u "$DOCKER_USERNAME" --password-stdin
  - docker build -t mysql:8  -f Dockerfile .
  - docker build -t skipton/cododigital.co.uk:latest  -f Dockerfile .
  - curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
  - rm -rf awscli-bundle
  - 7z x awscli-bundle.zip
  - sudo ./awscli-bundle/install -b /usr/local/bin/aws -i /usr/local/aws
  - /usr/local/bin/aws configure set aws_access_key_id $AWS_ACCESS_TOKEN
  - /usr/local/bin/aws configure set aws_secret_access_key $AWS_SECRET
install:
  - docker network create travis-ci
  - d=`date +%Y-%m-%d`
  - results=`aws s3api list-objects --bucket backups-cododigital --query "reverse(sort_by(Contents,&LastModified))" --prefix "backup_$d"`
  - dbs=`echo $results | jq --raw-output '.[].Key | match("^((.*)-db.gz)$") | .string'`
  - db=`echo $dbs | cut -d' ' -f1`
  - echo "$db"
  - uploads=`echo $results | jq --raw-output '.[].Key | match("^((.*)-uploads.zip)$") | .string'`
  - upload=`echo $uploads | cut -d' ' -f1`
  - echo "$upload"
  - themes=`echo $results | jq --raw-output '.[].Key | match("^((.*)-themes.zip)$") | .string'`
  - theme=`echo $themes | cut -d' ' -f1`
  - echo "$theme"
  - plugins=`echo $results | jq --raw-output '.[].Key | match("^((.*)-plugins.zip)$") | .string'`
  - plugin=`echo $plugins | cut -d' ' -f1`
  - echo "$plugin"
  - mkdir ~/theme ~/upload ~/db ~/plugin
  - aws s3api get-object --bucket backups-cododigital --key $db ~/db/$db
  - aws s3api get-object --bucket backups-cododigital --key $upload ~/upload/$upload
  - aws s3api get-object --bucket backups-cododigital --key $theme ~/theme/$theme
  - aws s3api get-object --bucket backups-cododigital --key $plugin ~/plugin/$plugin
  - cd ~/theme
  - bsdtar --strip-components=1 -xvf $theme
  - rm -f $theme
  - cd ~/db
  - gunzip -k $db
  - rm -f $db
  - find . -type f -exec mv '{}' '{}'.sql \;
  - cd ~/upload
  - bsdtar --strip-components=1 -xvf $upload
  - rm -f $upload
  - cd ~/plugin
  - bsdtar --strip-components=1 -xvf $plugin
  - rm -f $plugin
  - cd ~
before_script:
  - docker run --name db -v ~/db:/docker-entrypoint-initdb.d -e MYSQL_USER=travis -e MYSQL_PASSWORD="test123!" -e MYSQL_ROOT_PASSWORD=testing1234! -e MYSQL_DATABASE=myapp_test --network travis-ci mysql:8
  - docker run --name app -p 80:80 -v ~/upload:/var/www/html/wp-content/uploads -v ~/plugin:/var/www/html/wp-content/plugins -v ~/theme:/var/www/html/wp-content/themes -e WORDPRESS_DB_HOST=db -e WORDPRESS_DB_USER=travis -e WORDPRESS_DB_PASSWORD="test123!" -e WORDPRESS_DB_NAME=myapp_test -e WORDPRESS_CONFIG_EXTRA="define('WP_CACHE', true);define( 'W3TC_CONFIG_DATABASE', true );" --network travis-ci skipton/cododigital.co.uk:latest /bin/sh -c "wget http://downloads.lambdatest.com/tunnel/linux/64bit/LT_Linux.zip && unzip ./LT_Linux.zip && ./LT -user $LAMBDATEST_EMAIL -key $LAMBDATEST_KEY; cd /var/www;"
script:
  -  php /var/www/tests/HomepageTest.php

The tail end of the log is:

$ docker run --name db -v ~/db:/docker-entrypoint-initdb.d -e MYSQL_USER=travis -e MYSQL_PASSWORD="test123!" -e MYSQL_ROOT_PASSWORD=testing1234! -e MYSQL_DATABASE=myapp_test --network travis-ci mysql:8

10529WordPress not found in /var/www/html - copying now...
10530WARNING: /var/www/html is not empty! (copying anyhow)
10531Complete! WordPress has been successfully copied to /var/www/html
10532AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.2. Set the 'ServerName' directive globally to suppress this message
10533AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.2. Set the 'ServerName' directive globally to suppress this message
10534[Wed Jan 29 13:56:03.136852 2020] [mpm_prefork:notice] [pid 7] AH00163: Apache/2.4.38 (Debian) PHP/7.4.2 configured -- resuming normal operations
10535[Wed Jan 29 13:56:03.136960 2020] [core:notice] [pid 7] AH00094: Command line: 'apache2 -D FOREGROUND'
10536
10537
10538No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
10539Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received
10540
10541The build has been terminated
2

docker run won’t return until the command run in the container is complete. So you need to run those commands in background – and probably save their PIDs ($!) to kill them when you are done.

See ProcessManagement - Greg’s Wiki for details.

3

Hi @native-api,

Thanks for that information - it clarifies what I thought.

What’s the best way of stopping Apache being started?

In my Dockerfile there is an ENTRYPOINT command to a bash script which doesn’t do anything but start Apache. Removing this could possibly make it get to the next steps but then on my next deployment Apache wouldn’t be started.

I feel like what I need to do is:

  • Create new Dockerfile called DockerfileTest
  • Create new entrypoint-test.sh removing the starting of Apache command
  • Modify DockerfileTest to run entrypoint-test.sh
  • Add a step to .travis.yml to start Apache as a background process

Does that look the best approach?

4

That would start Apache outside of the container.

AFAICS, something like this should work:

- docker run --cidfile=db.cid <db> & PIDS="$!"
- docker run --cidfile=app.cid <app> & PIDS="$PIDS $!"
<...>
- docker kill $(cat app.cid db.cid)
- wait $PIDS

If not, ask for a better way at a Docker forum.

1 Like