Hello, our team members noticed today that builds configured with “matrix: / allow_failures” list no longer ignore (or specially mark up in build results) the build items while build issues should not fail the build.
https://travis-ci.org/github/42ity/fty-common/builds/658744371/config?utm_medium=notification&utm_source=github_status (line 206) shows the config used in the failing build https://travis-ci.org/github/42ity/fty-common/builds/658744371?utm_source=github_status&utm_medium=notification (item 344.10 in the list failed, and no items allowed a failure as Travis decided). A re-build of the PR did not help.
Earlier builds worked as expected, e.g. 343 : https://travis-ci.org/github/42ity/fty-common/builds/657780343?utm_medium=notification&utm_source=github_status
Curiously, some later builds were scheduled by cron but failed with “ERROR: This build could not be run. Reason: invalid_config” e.g. https://travis-ci.org/github/42ity/fty-common/builds/660656507?utm_medium=notification&utm_source=github_status but were parsed for results as having the expected block of allowed-failure scenarios.
At this time another build scheduled by cron is pending: https://travis-ci.org/github/42ity/fty-common/builds/663533613?utm_medium=notification&utm_source=github_status so I hope this was an intermittent hiccup or quickly fixed issue pushed to travis codebase 
Thanks,
Jim Klimov
Hey Jim, sorry about the issues you’re seeing.
I’m not sure what happened to your allowed_failure jobs there. I have tried to reproduce the issue, but it looks like whatever the issue was has been fixed in a deployment that has run in the meantime. I’ll do some more digging later.
However, it seems the issue has gone away on your recent cron build https://travis-ci.org/github/42ity/fty-common/builds/663533613, which should mean it’s gone on any push or pull request builds, too. Could you please let me know if it isn’t?
As for the mysterious error This build could not be run. Reason: invalid_config this has happened during Heroku’s outage yesterday. This condition practically never happens, except if the component configuring your build is entirely unavailable. I’ll look into improving that error message.
Two other things regarding your build config:
Shared custom keys used for YAML references are now supposed to be prefixed with an underscore, marking them as “private” (custom) keys. This is meant to prevent naming conflicts with future additions to the Travis CI build config format. While this is not likely to ever be a problem with keys like pkg_deps_prereqs_distro our recommendation would be to still prefix them in order to get rid or those warning messages.
Your jobs.allow_failures section references jobs that are listed in jobs.include only, as far as i can tell. While jobs coming out of the matrix expansion cannot have a name attribute, those listed in jobs.include can, and using such names makes the allowed_failures section more readable and less error prone (e.g. with future additions or changes to any of the listed keys). For example, for your last job that might look like this:
jobs:
include:
- name: clang format check
env: BUILD_TYPE=clang-format-check CLANG_FORMAT=clang-format-5.0
...
allow_failures:
- name: clang format check
Thank you for the suggestions, I will ask about modernizing the templates in tool that generates these recipes for us (zproject).
Restarting a PR’s build did not help - it again had all 10 scenarios in same priority block.
But closing the PR, waiting and re-opening seems to have done the trick, a new build was scheduled with allowed failures.
Thanks for the feedback, Jim!
Yes, that unfortunately is expected. A build or job “restart” does not go through the “configuring the build” process again, it just restarts the same build or job as is.