Unit test in R with testthat fails but Travis CI build is successful

I am currently working on a toy R package to see how it works with CI. I am trying to use unit tests with the testthat package. I use GitHub and Travis for CI integration.

For reproducing the results, here is my repository:
git clone https://github.com/Redcart/helda.git

  1. I have coded a simple unit test in a script called test-proc_freq.R that should works
  • When I run locally devtools::test(), the test is passing as expected.

  • When I run locally devtools::check(), all is fine.

  1. then I modify the test-proc_freq.R file adding some line that should fails the test
  • When I run locally devtools::test(), the tests fail as expected.

  • When I run locally devtools::check(), I get no messages of any failures. I have the same messages when the test was passing.

I am wondering what the check function of the package devtools is doing for the unit tests ? I have the impression that it just checks that the unit tests are properly coded and can be launched but not whether it has passed or failed.

When I push my code on GitHub, Travis CI is triggered thanks to the following .travis-ci.yml file:

# some stuff before

script:
 - R CMD build .
 - R CMD check --as-cran helda_0.1.tar.gz
 - Rscript -e "library(testthat); testthat::test_dir('tests/testthat')"

# some stuff after

However in the two situations described above, the build with Travis CI is successful.

Even adding Rscript -e "library(testthat); testthat::test_dir('tests/testthat')" has not helped:

How can I make Travis CI fail the build when the unit tests fail ?

According to https://www.rdocumentation.org/packages/testthat/versions/2.3.1/topics/test_dir, test_dir returns “A list of test results” – i.e. nothing that would cause the code passed to Rscript to terminate abnormally.

Imprint