Python support on Windows

Environments Windows
1

Thank you for supporting Windows. As you know Python is a very popular language and is cross-platform. I would love to see Python supported in the Windows Build Environment.

16 Likes
2

For Travis’s support for Python on Windows to be worth the bother, it would need:

  • Correct version of Visual Studio installed for the Python version (https://wiki.python.org/moin/WindowsCompilers)
  • Visual Studio’s environment is already set up in the Git Bash session that pre_install and install scripts run in i.e., Travis takes care of running vcvarsall.bat or whatever it’s called. This would require translating the bat code to bash, and Travis would really earn their fees by taking care of that for me.
  • pip install source packages that require a compiler Just Works (e.g., small projects such as ecos or scs can be installed with pip instal ecos or pip install scs and nothing else)
  • No being forced to use non-standard package managers such as conda (yes I know conda is common in the scientific community, but it’s not in the larger Python community. Pip is bundled with Python. Pip is the standard.)

Additional wish list:

  • MS Office is available for Python code that interacts with Excel (e.g., via xlwings)
  • Chrome and Firefox are available for projects that run Selenium tests.
  • Common packages on Linux/macOS are preinstalled on Windows, since you can’t just apt-get or brew install things like, I don’t know, say, BLAS or LAPACK (maybe not great examples, but hopefully you see what I’m driving at).
3 Likes
3

This is something I’m definitely interested in seeing happen. Unlike @wkschwartz, I think even if Travis doesn’t provide a works out-the-box compiler setup, it’ll still be good enough for my use cases, where wheels and pure Python sdists are good enough.

For widespread usage, however, would require the compiler setup to be done properly.

4

First of all …

Thank you for your interest in bringing Python support to Windows! We very much appreciate your enthusiasm.

I will describe how Python support works on Linux right now. Do keep in mind that Windows is a new game, and it would not have to happen in the same way as it does in the other two OSes. (Though, obviously, similar tooling may reduce the amount of work it would take to make it happen.)

How PHP support works now on Linux

Much of what we would call “Windows support” on Travis CI is described in https://github.com/travis-ci/travis-build/blob/master/lib/travis/build/script/python.rb.

  1. It assumes that there is a usable virtualenv installed on the system.
  2. Look for the applicable activate file. If it’s not there, fetch the applicable Python archive from our S3 bucket and expand it. The archive contains the activate file.
  3. source .../activate

Creating pre-compiled Python archives

https://github.com/travis-ci/cpython-builder/ is responsible for compiling and creating new Python archives. It sets up (if necessary) a Python building environment, compiles Python using python-build, and uploads the archives to the right place so that travis-build can find it at the run time.

How should Python support on Travis look?

This is an open question. As I said earlier, I might be less work to make it look a lot like Linux. Maybe not. My understanding of the Windows ecosystem is very limited.

Option 1: Leverage current tooling

In this case, I reckon the first thing to do would be to create a base set of Python archives for use by Windows VMs. This would start with a language: sh job on Windows, use python-build to compile, archive and upload to the remote file storage. I do not know if python-build, written in bash, is compatible with “git bash”. If it’s not, it could be some work to make it so.

This would also mean that we ensure that the Windows image has a reasonable C/C++ toolchain that we can use.

Option 2: Use Windows-native tooling

It might be more intuitive for Windows developers to use the Windows-native tools. For example, https://www.python.org/downloads/windows/ has many archives that might be usable. Or, alternatively, Chocolatey archive(s) might be more in tune with what we need.

I hope all of this makes some sense, and helps you get started.

5

If you don’t need working compiler setup it’s quite easy to test Python code right now, see https://github.com/WeblateOrg/translation-finder/blob/master/.travis.yml#L20-L26

2 Likes
6

https://staging.travis-ci.com/ifduyue/test-travis/jobs/235583/config

Python.3.6.7-amd64.exe /quiet install will hang for more than 10 minutes and then build job will get terminated

7

I tried to duplicate this, but it either times out installing Python 3 or (if I skip that step, Python 3 is supposed to already be installed) can’t find Python. Any thoughts on what I’m doing different than you?

8

Sigh… it seems to be a known issue with secrets.

9

Hi,

I’d love Windows Python3 support in Travis too - Happy to test + help. Here would be my dream .travis.yml:
Today: https://github.com/pypa/bandersnatch/blob/master/.travis.yml
Diff: https://pastebin.com/zmNFKqnP

Is there any way third parties can help with making the support happen?

1 Like
10

Hi

Thanks for supporting Windows. 8)

If I may suggest : I would be happier with python3 + conda support.

Hope this helps all & I welcome any comments and/or suggestions.

KW

11

Python’s distutils/setuptools build system takes care of detecting a VS compiler and setting up build environment (up to and including running vcvars*.bat and extracting envvars from the result), so you don’t really need that.

Ditto.

12

BTW, the Windows of Travis-CI, if I am correct, is not supporting python yet either.

Let me think about using pip later (excuse me for not making changes immediately)

KW

13

Also, I noticed that the conda build is taking some time.

To save the resources, is it desirable to utilize cache?

How would I be able to check if the cache content is relevant before a build breaks?

Best Wishes,

KW

14

So I can use conda but is there any way to use a plain python 3.6 or 3.7 and install some packages with pip?

1 Like
15

@mgiaco See https://github.com/cclauss/itinerant-tester-windows

16

See https://github.com/ove/ove-install. As long as you don’t have secure environment variables, like I do, it should build perfectly well and do whatever you like.

17

FWIW the presence of python (and pip) would on its own be hugely valuable for many projects, so is worth doing even if it takes a bit longer to solve the harder problems around native building. A huge number of python projects don’t have any native/C++ component.

Also if it speeds things up, supporting even just one python version (e.g. python 3 not python 2) would be a great first step, and enough for many projects.

Using the pre-built binaries from https://www.python.org/downloads/windows/ or choco would be more idiomatic (and probably a lot simpler to implement) than trying to build python from source, since that’s what most end-users of python applications/modules will be doing, so is what people running tests from Travis would find most useful.

1 Like
18

Why can’t the following intuitive approach work and be fast??

language: python
python: 3.7
os:
  - linux
  - osx
  - windows
install: pip3 install --upgrade pip
script: python3 platform-info.py

The minimal .travis.yml file required to run Python 3.7 on Linux, macOS, and Windows is unnecessary complex, fragile, and slow. Would it at least be possible to have choco install python done on the Windows base image to save a 1 min 30 sec startup time?

https://docs.travis-ci.com/user/languages/python/#running-python-tests-on-multiple-operating-systems

1 Like
19

I haven’t been able to figure out how to install miniconda with the Travis CI windows environment, although I gather that it is possible based on other people’s comments. Could anyone point me towards the documentation on this?

20