I experienced the issue too, and the solution was like your one.
@Michal mentioned following comment. So if language: python
wouldn’t work on arm64, what did his comment mean?
Thanks for raising this one and providing workaround solution. Such feedback is much appreciated. Right now, python packages 3.6/3.7/3.8 should be already available for arm64 builds.