So here’s a weird and strangely intermittent issue for which I have no discernible pattern. I maintain an open-source Python library (called SQLAthanor, with source here and Travis page here) and run all of my unit tests through Travis CI. My build matrix is fairly broad, encompassing:
- Python 2.7 - 3.7 (3.7 on
xenial, previous versions ontrusty) - Multiple versions of SQLAlchemy
- Multiple versions of PyYAML
- Multiple versions of simplejson
and all of the combinations of those items. Recently - as in the last few weeks - I have observed a pattern where my Travis jobs are erroring out during their environment setup phase. Specifically, when the various environmental requirements are being installed, and most frequently with tox and codecov. What typically happens is that I get a build error like the following in my build log:
$ python --version
Python 3.4.6
$ pip --version
pip 9.0.1 from /home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages (python 3.4)
6.83s$ pip install -U tox codecov
Collecting tox
Installing collected packages: six, attrs, pyparsing, packaging, scandir, pathlib2, more-itertools, zipp, importlib-metadata, pluggy, py, filelock, virtualenv, toml, tox, coverage, idna, urllib3, certifi, chardet, requests, codecov
Found existing installation: six 1.10.0
Uninstalling six-1.10.0:
Successfully uninstalled six-1.10.0
Rolling back uninstall of six
Exception:
Traceback (most recent call last):
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2813, in _dep_map
return self.__dep_map
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2624, in getattr
raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2804, in _parsed_pkg_info
return self._pkg_info
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2624, in getattr
raise AttributeError(attr)
AttributeError: _pkg_info
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/basecommand.py”, line 215, in main
status = self.run(options, args)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/commands/install.py”, line 342, in run
prefix=options.prefix_path,
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/req/req_set.py”, line 784, in install
**kwargs
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/req/req_install.py”, line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/req/req_install.py”, line 1064, in move_wheel_files
isolated=self.isolated,
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/wheel.py”, line 247, in move_wheel_files
prefix=prefix,
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pip/locations.py”, line 140, in distutils_scheme
d = Distribution(dist_args)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/setuptools/dist.py”, line 321, in init
_Distribution.init(self, attrs)
File “/opt/python/3.4.6/lib/python3.4/distutils/dist.py”, line 280, in init
self.finalize_options()
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/setuptools/dist.py”, line 389, in finalize_options
ep.require(installer=self.fetch_build_egg)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2324, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 862, in resolve
new_requirements = dist.requires(req.extras)[::-1]
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2568, in requires
dm = self._dep_map
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2815, in _dep_map
self.__dep_map = self._compute_dependencies()
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2824, in _compute_dependencies
for req in self._parsed_pkg_info.get_all(‘Requires-Dist’) or :
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 2806, in _parsed_pkg_info
metadata = self.get_metadata(self.PKG_INFO)
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 1468, in get_metadata
value = self._get(self._fn(self.egg_info, name))
File “/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/pkg_resources/init.py”, line 1577, in _get
with open(path, ‘rb’) as stream:
FileNotFoundError: [Errno 2] No such file or directory: ‘/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/six-1.10.0.dist-info/METADATA’
The command “pip install -U tox codecov” failed and exited with 2 during .
What’s strange about this is that the problem is intermittent (affects about 2 - 5 out of 131 jobs), and when I restart a job that errored out it usually completes successfully after the restart. Sometimes it will error out again (sometimes three or four times in a row!), but if I keep restarting it eventually completes with no problem. Here’s a link to a job that I specifically did not restart so that the full log would be available: Travis CI - Test and Deploy with Confidence
The problem seems to be most common on my Python 3.4 jobs, but I’ve seen it occur sometimes (very rarely) on Python 3.5 and 3.6 as well.
Does anyone have any suggestions?