Preinstalled Python packages include at least four copies of setuptools

#1

There are four different versions of setuptools installed into the xenial Python 3.6 environment

40.8.0
40.6.2
40.5.0
40.4.3

Here I uninstall them one at a time: https://travis-ci.org/jayvdb/coala/jobs/567438424#L204

It would be nice if there was just one or two (often the last one is hard to remove)

Also pipenv 2018.11.26 is preinstalled into the xenial Python 3.6 environment, even when there isnt a Pipfile. I dont recall this being present in trusty images. I see it in https://github.com/travis-ci/cpython-builder/pull/27

This is a rather large change from the baseline python virtualenv, as pipenv 2018.11.26 depends on

required = [
    "pip>=18.0",
    "certifi",
    "setuptools>=36.2.1",
    "virtualenv-clone>=0.2.5",
    "virtualenv",
    'enum34; python_version<"3"',
    # LEAVE THIS HERE!!! we have vendored dependencies that require it
    'typing; python_version<"3.5"'
]

It would be good if pipenv was optional based on whether a Pipfile exists in the repo.

Also IMO it would be a good idea to pin the preinstalled pytest dependency to pytest<5 , at least for a while, as pytest 5 broke most non-core plugins, and decommissioned py27 and py34 support. If a user has a requirements.txt which includes unpinned pytest and a pytest plugin which isnt maintained by the core pytest devs , there is a very high chance that their build will start breaking.

1 Like
#2

Confirmed that there are extraneous setuptools*.dist-info directories in the pre-created 3.6.7 virtualenv:

/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/setuptools
/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/setuptools-40.4.3.dist-info
/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/setuptools-40.6.2.dist-info
/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/setuptools-40.5.0.dist-info
/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools-40.8.0.dist-info

I suspect that they were being created with pip install --ignore-installed instead of the intended pip install -U or otherwise some way that blindly appends to installed files.

#3

The Travis pip caching makes this rather unnecessary. Note that pipenv isn’t standard, and poetry, flit and others are rapidly taking over that space as simpler solutions suitable for most project. The relevant adopted pep intentionally doesnt choose any build tool as the default, but allows the necessary build tool to be determined from pyproject.toml. An explainer https://medium.com/@grassfedcode/pep-517-and-518-in-plain-english-47208ca8b7a6

@joepvd, do you recall why pipenv was added?

#4

You are trying to do something highly unusual – install an ancient version of pip which doesn’t even support 3.6 (same for setuptools; I couldn’t find if that pip is still officially supported by PyPI). It’s highly probable that it will conflict with requirements of lots of things, not just pipenv.

So for your workflow, it’s better to recreate the virtualenv from scratch:

#no cross-platform way as per https://stackoverflow.com/questions/54641473/get-path-to-original-python-executable-not-the-virtualenv#comment101178226_54641554
BASE_PYTHON="$(python -c 'import sys; print(sys.real_prefix)')/bin/python"
deactivate
"$BASE_PYTHON" -m virtualenv ~/virtualenv/custom
source ~/virtualenv/custom/bin/activate
Imprint