It’s not clear from your explanation what exactly you’re getting and what you are expecting. Could you clarify and/or provide a link to the build?
If the Critical Error dialog is disabled in Windows (either globally or via SetErrorMode), a crashing program will simply return a non-zero exit code.
From what I can currently see, this is exactly what is happening for you.
As a side note, running just bin/executable.exe should be sufficient since Travis build is run with Git Bash which does wait for a command to complete.