Been using the aarch64 builds for a couple of days in https://travis-ci.org/randombit/botan and it’s working very nicely, replaces our previous setup which relied on cross compilation+qemu which was much slower.
Can you please install the following packages in the default aarch64 image?
ccache
liblzma-dev
libbz2-dev
They are included in the base x86-64 images and especially ccache seems like it should be installed by default.