Python 3.4 - failing to uninstall pre-installed six (since a few days)

Hi!
We have some test using Python 3.4/3.5/… running on Travis-ci(.com). Early on, they perform the following during the installation part:

pip install -U tox

It works for all Python version, except Python 3.4 since a few days (1-2 maybe). Tox depends on six. On the Python 3.4 builds, it fails with:

Found existing installation: six 1.10.0
Uninstalling six-1.10.0:
Successfully uninstalled six-1.10.0
Rolling back uninstall of six
Exception:
Traceback (most recent call last):
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/ init .py”, line 2813, in _dep_map
return self.__dep_map
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/ init .py”, line 2624, in getattr
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
[…]
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 1577, in _get
with open(path, ‘rb’) as stream:
FileNotFoundError: [Errno 2] No such file or directory: ‘/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/six-1.10.0.dist-info/METADATA’

Full logs

Installing collected packages: six, setuptools, virtualenv, toml, scandir, pathlib2, zipp, importlib-metadata, pluggy, filelock, py, tox
Found existing installation: six 1.10.0
Uninstalling six-1.10.0:
Successfully uninstalled six-1.10.0
Rolling back uninstall of six
Exception:
Traceback (most recent call last):
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2813, in _dep_map
return self.__dep_map
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2624, in getattr
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2804, in _parsed_pkg_info
return self._pkg_info
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2624, in getattr
raise AttributeError(attr)
AttributeError: _pkg_info
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/basecommand.py”, line 215, in main
status = self.run(options, args)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/commands/install.py”, line 342, in run
prefix=options.prefix_path,
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/req/req_set.py”, line 784, in install
**kwargs
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/req/req_install.py”, line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/req/req_install.py”, line 1064, in move_wheel_files
isolated=self.isolated,
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/wheel.py”, line 247, in move_wheel_files
prefix=prefix,
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/locations.py”, line 140, in distutils_scheme
d = Distribution(dist_args)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/setuptools/dist.py”, line 321, in init
_Distribution.init(self, attrs)
File “/opt/python/3.4.6/lib/python3.4/distutils/dist.py”, line 280, in init
self.finalize_options()
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/setuptools/dist.py”, line 389, in finalize_options
ep.require(installer=self.fetch_build_egg)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2324, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 862, in resolve
new_requirements = dist.requires(req.extras)[::-1]
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2568, in requires
dm = self._dep_map
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2815, in _dep_map
self.__dep_map = self._compute_dependencies()
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2824, in _compute_dependencies
for req in self._parsed_pkg_info.get_all(‘Requires-Dist’) or :
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2806, in _parsed_pkg_info
metadata = self.get_metadata(self.PKG_INFO)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 1468, in get_metadata
value = self._get(self._fn(self.egg_info, name))
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 1577, in _get
with open(path, ‘rb’) as stream:
FileNotFoundError: [Errno 2] No such file or directory: ‘/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/six-1.10.0.dist-info/METADATA’

Here’s a link to the build: https://travis-ci.com/secdev/scapy/jobs/204212531
(and the issue https://github.com/secdev/scapy/issues/2049)

I tried updating pip to the latest version before running it in https://github.com/secdev/scapy/pull/2050. Didn’t fix the issue

There hasn’t been any update in tox/six recently, therefore I assume it’s a Travis update?
I know Travis isn’t responsible for how we install the dependencies, but it seems that six is installed by default, and the default install might be broken at the moment.

Any help would be greatly appreciated :smile:

I checked that the METADATA file is present at the appropriate path in the py3.4 archive downloaded in the failing build.
You can check for its presence during the build, of course.

This is the core symptom:

Rolling back uninstall of six

This is not normal. This happens if an exception occurs during uninstall. Why it happens and if that’s the exception you see, I do not know.

Comparing the failing build log and an earlier successful build log, I see that dependencies are downloaded in a different order. Maybe that matters.

1 Like

Thanks for the analysis, I got to similar conclusions.

It’s unclear why installing six would fail. I’ll try to investigate. The most striking thing is of course that this didn’t happen earlier. I checked out setuptools, pip, tox, six, pkg_resources (bundled with setuptools), and none seem to have been updated anytime recently :confused:

It would appear that forcing a pip and setuptools update fixes the issue.
Edit: It doesn’t. The issue is random. We ended up using --skip-installed on pip to bypass the issue.

FWIW use your own virtualenv rather than messing with the global Python install…