Python development versions no longer include pip (due to virtualenv 20.x)

Languages Python
1

Nothing changed on our side between our last good build and our first bad build, but all Python development versions (3.5-dev, 3.6-dev, 3.7-dev, 3.8-dev and nightly) fail with the following error:

$ pip --version
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.8-dev/bin/pip", line 6, in <module>
    from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip'

Is it possible to get back the old behavior? Or should we install pip ourselves?

Thanks!

1 Like
2

Getting the same thing on our side. Yesterday the build was working and today its failing with that error. Nothing has changed.

3

+1 but with

$ python --version
Python 2.7.17

the issue started yesterday

  File "...build/ve/darwin/bin/pip", line 6, in <module>
    from pip._internal.cli.main import main
ImportError: No module named main
4

Thatā€™s unrelated. You are relying on undocumented internals of Pip, and they have changed.

1 Like
5

You can look at the recent build history at https://travis-ci.com/travis-ci/cpython-builder/builds. If you are interested in finding your Python version build log, you can use the CLI:

$ travis history --limit 100 --com -r travis-ci/cpython-builder | grep VERSION=3\.5-dev
#8019 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200211T103640Z
#8012 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200210T103548Z
#8005 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200209T103354Z
#7998 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200208T103311Z
#7985 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200206T103831Z
#7978 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200205T103443Z
#7971 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200204T103350Z
#7964 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200203T103121Z
#7957 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200202T102952Z
#7950 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200201T103009Z
#7943 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200131T102807Z
#7936 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200130T152259Z
#7929 passed:    default build branch=default; source=rake; env="VERSION=3.5-dev" 20200129T102653Z
$ travis open --com -r travis-ci/cpython-builder 8012

The build process does go through changes, of course, but for the most part they are pretty stable. If the *-dev version behaviors change, the changes are most likely due to the changes upstream.

1 Like
6

Identified the reason. Itā€™s a change in virtualenvā€˜s behavior in the latest version.
It now saves a couple of required files ā€“ including pip installation ā€“ outside the virtualenvā€™s subtree. Travisā€™ CPython compilation worker saves a premade virtualenv into a tarball to avoid recreating it each time ā€“ which now doesnā€™t include those files.

I donā€™t yet know if itā€™s possible to change that, or to somehow reliably figure out which external files to include and guarantee that the virtualenv will work on a different system.

1 Like
7

Yep, virtualenv 20 was released yesterday. Iā€™ve opened an issue on their tracker:

8

If there is a Python release before the issue is fixed, they will likely be affected by this issue.

9

Virtualenv have already made a fix and released it as 20.0.2!

Is there something that needs to be done on the Travis side to update? Thanks!

10

Within 24 hours the next build will kick off, then those will be transported to the place builds can use shortly thereafter.

1 Like
11

Used a more future-proof solution. There are also environment changes in Multiarch images that also required adapting the code to (thatā€™s why 3.9-dev builds were failing).

2 Likes
12

An automatic build has run successfully at 3:46 UTC.

However, OPā€™s last build run at 9:54 UTC is still failing due to a defective tarball.

Iā€™ve double-checked that tarballs created after the fix are okay.

AFAICS, automatic builds upload artifacts to s3.amazonaws.com, however, user builds download them from storage.googleapis.com instead.

Is there a large gap between automatic build time and synchronizing the two cloud storages or something?

13

Okay, now the storage.googleapis.com tarball has correct contents (timestamped 11:43 UTC, likely from the next automatic build).

Confirmed that the -dev builds now work.

1 Like
14

Thanks everyone for the help! :heart:

15

Thank you! Also confirmed working for 3.6-dev to 3.9-dev and nightly.