I have recently stumbled across #1419 while trying to make the Android emulator work decently. There you say that you can’t provide KVM due to infrastructural limits in Google Compute Engine.
Today I have stumbled across this ( https://cloudplatform.googleblog.com/2017/09/introducing-nested-virtualization-for.html ) and this ( About nested virtualization | Compute Engine Documentation | Google Cloud ). Now, please bear with me - I know nothing about Linux and KVM - but it looks like this feature could help? As far as I understand, you just enable this feature and then KVM will be provided…
If it’s true, please consider this! The non-accelerated ARM emulators are so slow and Google is not even releasing the system images anymore for new Android APIs (already absent for API 26 and 27). So the current workaround (use ARM emulators even if it takes 20 mins) won’t work forever.
Thanks.
I am sure @BanzaiMan is fully aware of this as I suspect he lives and breathes it :-), but this may be interesting for others. I’d love any input / corrections if my understanding is wrong
My understanding is that with the consolidation of the Linux architecture within Travis (which collapsed the two ‘sudo’ environments that would route to Google Compute Engine or AWS depending on sudo false or true respectively) I think that means everything goes to GCE. I know for sure language: android went to AWS regardless of sudo status prior, but now my android builds are appear to be running on GCE workers.
So since GCE enables the nested virtualization feature, and I think Travis just went all GCE where this is possible, maybe this can be enabled.
Travis would just have to re-configure GCE following this info https://cloud.google.com/compute/docs/instances/enable-nested-virtualization-vm-instances, and it would work. Even if it was just for ‘language: android’ (or even some limited ‘beta testing’ configuration) it might be worthwhile to lower Travis compute costs as the ARM emulators are stupendously slow while pegging the CPU.
If it is useful at all, I would be willing to test any possible implementation with my dev fork of the AnkiDroid project’s builds and provide feedback - they’re pretty involved Android+Travis emulator users - https://travis-ci.com/mikehardy/Anki-Android
I created a project on github to test android emulators on linux and mac here. For demo purposes it includes attempting to start the x86 emulators (which of course will not run). Could be a useful reference for emulator set-up times.
Was able to get several to start and even use one config reliably here on both linux and mac.
Even wrote article about the experience.
Enabling hardware acceleration on linux (and mac for that matter) would be a huge boost in speed for running android x86 emulators.
Enabling nested virtualization (i.e. KVM support inside the travis worker VM) would also be super useful for non-android projects.
For example for projects that deal with configuration management, provisioning or the system boot process.
Examples:
As-is, on Travis-CI. because nested virtualization is disabled, such CI code has to execute QEMU without KVM support which results in a massive waste of CPU cycles. In my local experiments, the dracut-sshd test suite runs 50 % slower in a VM without nested virtualization.
okay now after using search, finding nothing and creating a new issue, I found this … please support nested virtualization, the arm-android emulator really is a pain in the automated-tests
I tested x86 emulator without hardware acceleration on Travis CI.
My config:
sys-img-x86-android-21
- echo no | android create avd --force -n test -t android-21 --abi x86
- emulator -avd test -memory 1500 -no-accel -gpu swiftshader -skin 720x1280 -camera-back none -camera-front none -no-boot-anim -no-window &
And it’s works more faster then on ARM.
Maybe it will be heplful for someone.
The latest android emulator canary has support for running x86 images without acceleration, test project here: https://travis-ci.org/msand/test_emulators/builds/502382369
https://github.com/msand/test_emulators/blob/master/.travis.yml
Is there a project where you can show this working?
I will try adding it to https://github.com/mmcc007/test_emulators
If you want, it would be great if you could do a PR on
https://github.com/mmcc007/test_emulators
Hey @msand,
for the record, and as we discussed elsewhere, I will try adding it to https://github.com/mmcc007/test_emulators
If you want, it would be great if you could do a PR on
https://github.com/mmcc007/test_emulators
As discussed, adding a test of the emulator by running an android app on it, makes sense. Trying to make that happen.
Right now, no. I can not share the project, but I can share part of my config.
Just add this lines to your config and check, it’s works.
android:
components:
sys-img-x86-android-21
echo no | android create avd --force -n test -t android-21 --abi x86
emulator -avd test -memory 1500 -no-accel -gpu swiftshader -camera-back none -camera-front none -no-boot-anim -no-snapshot-save -no-snapstorage
Also you can use other android API level. I checked with API 24 and it’s works. I think that latest API versions would work, but I didn’t check it.
-no-accel - means that emulator will run without hardware acceleration
https://developer.android.com/studio/run/emulator-commandline
-gpu swiftshader - means that software acceleration turn on
https://developer.android.com/studio/run/emulator-acceleration
Please check and inform me about any problem.
+1 on this. I also have the need to test android images, and they take forever. Also, it would make it overall easier for travis ci since kvm + docker will allow your users to test whatever the heck they need to if they’re willing to write their own qemu startups for it. For instance, you wouldn’t actually have to directly support Windows/Android/Mac, since all of those could be emulated inside a docker image with kvm passthrough at much better performance than currently.
Apparently it has been tried and is possible at least in one attempt
It works, if you choose CI configs carefully at least
Here’s a travis config using android emulators with hardware acceleration (that is, KVM virtual machines with nested virtualization) functioning now. https://github.com/ankidroid/Anki-Android/blob/master/.travis.yml
A build running it https://travis-ci.org/ankidroid/Anki-Android/builds/606100691
All the knowledge it took to produce documented here https://github.com/ankidroid/Anki-Android/pull/5594
Looks like accelerated emulators are not starting on bionic on travis. I noticed several projects have this problem:
https://travis-ci.org/ankidroid/Anki-Android/builds/620477295
https://travis-ci.org/mmcc007/test_emulators/builds/620812178
Accelerated emulators are still starting on cirrus:
I would be surprised if Travis just turned off nested virtualization, I wonder if it is because the emulator just updated - I just received an update to the emulator locally anyway. My first test (since I really don’t want to incur the hassle of moving) will be to try pinning the emulator back a version
Pinned emulator is working.
A bug report about this from someone: https://issuetracker.google.com/issues/145622251 please star/comment there! The issue does not seem to be related to KVM, we’re failing even without it.
Speaking about the original ticket (2 years old…), I was shocked today at how easily I could achieve accelerated Android emulators on GitHub Actions. I just set this up in a few hours : https://github.com/natario1/CameraView/pull/694 . Emulators run on macOS with HAXM and test time dropped from 30 minutes to about 5. The script is also clean and readable.
I dunno, I read that example. Nice that works on github actions, and really nice to have an example. Very nice to have acceleration on macOS.
Over the years of working with Travis for instance the problem I have seen is slow bitrot in the implementations hiding the exact complexity the actions are hiding (java install, android SDK install, emulator wait, etc). And in my experience with Github actions doing react-native builds on Microsoft I’ve been bitten by bugs in the underlying action implementations as they are altered, breaking my build there in similar ways that android SDK and java SDK broke on Travis.
Readability wise it just looks different to me, not more or less readable. That’s a personal preference. I do like that they seem to at least have a nice built-in way to share and reference 3rd party modules of functionality in the actions vs the way we’re integrating the “Jabba” java SDK install now - Travis could look to add that and I’d be happy - but unless the community adopts that sharing well and it’s easy to patch / modify it etc it’ll have the same stability issues over time that Travis does, and the tradeoff between having chunks of implementation in your build definition that work because you control them (but have to also fix and improve etc) vs plugins that others control and break outside of your control.
So, Github actions is more of a sideways push for me unless someone tests and publishes how many runners they are allocated in parallel for FOSS projects and it’s way more than Travis maybe. More than anything I just want the emulator implementations themselves to stabilize.
everything is back to normal now with 29.3.0, just needed to change ‘emulator-headless’ to ‘emulator -no-window’ then wait for a bug to be fixed (in the 29.3.0 release)