Can't deploy to PyPI anymore: pkg_resources.ContextualVersionConflict: importlib-metadata 0.18

Deployment
,
1

Hi!
I am having problems deploying new package versions to PyPI:
https://travis-ci.com/github/spinlud/py-linkedin-jobs-scraper/jobs/430957329

In the build I can see the following error (expanding Deploying application):

Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/bin/twine", line 8, in <module>
    sys.exit(main())
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/twine/__main__.py", line 28, in main
    result = cli.dispatch(sys.argv[1:])
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/twine/cli.py", line 80, in dispatch
    main = registered_commands[args.command].load()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2446, in load
    self.require(*args, **kwargs)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2469, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pkg_resources/__init__.py", line 775, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (importlib-metadata 0.18 (/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages), Requirement.parse('importlib-metadata>=1; python_version < "3.8"'), {'keyring'})
dist/linkedin-jobs-scraper-1.2.2.tar.gz already exists, no checkout
linkedin_jobs_scraper.egg-info/PKG-INFO already exists, no checkout
linkedin_jobs_scraper.egg-info/SOURCES.txt already exists, no checkout
linkedin_jobs_scraper.egg-info/dependency_links.txt already exists, no checkout
linkedin_jobs_scraper.egg-info/requires.txt already exists, no checkout
linkedin_jobs_scraper.egg-info/top_level.txt already exists, no checkout
error: could not restore untracked files from stash
The stash entry is kept in case you need it again.
PyPI upload failed.
failed to deploy

This line in particular states there is a version conflict:

pkg_resources.ContextualVersionConflict: (importlib-metadata 0.18 (/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages), Requirement.parse('importlib-metadata>=1; python_version < "3.8"'), {'keyring'})

but I am not sure how to solve this. I have no problems on building and uploading the package from local, this issue happens only with the Travis pipeline.

This is the .travis.yml:

sudo: required

services:
  - docker

language: python
python:
  - '3.6'

install:
  - pip install -r requirements.txt

jobs:
  include:
    - stage: test
      script: ./tests/run_tests.sh
    - stage: deploy
      script:
        - python setup.py install_egg_info sdist bdist_wheel
      deploy:
        provider: pypi
        username: '__token__'
        password: $PYPI_TOKEN
        on:
          branch: master

stages:
  - test
  - deploy

Anyone has any idea on how to fix this issue?

4 Likes
2

Same issue here, came out of the blue.

3

Does it work using a new Python version than 3.6?

4

Try re-setting the deployment token. Perhaps it was not correctly migrated from .org to .com.

5

I ran into the same issue starting today (last deploy on 4 Nov was successful). Tried using both python 3.6 and 3.7.

pkg_resources.ContextualVersionConflict: (importlib-metadata 0.18 (/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages), Requirement.parse('importlib-metadata>=1; python_version < "3.8"'), {'keyring'})
6

The release of keyring 21.5.0 appears to have regressed travis-ci deploy functionality.

I have worked around the issue temporarily by adding the following line to my deploy stage:

install: pip install keyring==21.4.0

1 Like
7

In that diff, we can see the new keyring now requires importlib_metadata >= 1:

-	importlib_metadata; python_version < "3.8"	
+	importlib_metadata >= 1; python_version < "3.8"

And from one of the above failing logs, the ā€œInstalling deploy dependenciesā€ section says:

keyring 21.5.0 requires importlib-metadata>=1; python_version < "3.8", but you'll have importlib-metadata 0.18 which is incompatible.

So looks like one of the earlier dependencies is requiring importlib-metadata 0.18, and causing the problem. I donā€™t know which one it is, but itā€™d be good to find out which one requirers that and loosen their pinning.

pip install pipdeptree && pipdeptree --reverse --packages importlib-metadata could help.

9

@spinlud et al, please test this fix by adding the following to the deploy: entry:

edge:
   source: 'native-api/dpl'
   branch: 'pip_fix_upgrade_deps'
2 Likes
10

Updated with the following and it worked!

sudo: required

services:
  - docker

language: python
python:
  - '3.6'

install:
  - pip install -r requirements.txt

jobs:
  include:
    - stage: test
      script: ./tests/run_tests.sh
    - stage: deploy
      script:
        - python setup.py install_egg_info sdist bdist_wheel
      deploy:
        provider: pypi
        username: '__token__'
        password: $PYPI_TOKEN
        on:
          branch: master
        edge:
          source: 'native-api/dpl'
          branch: 'pip_fix_upgrade_deps'

stages:
  - test
  - deploy
1 Like
11

@native-api Thank you, this works!

15

Once https://github.com/travis-ci/dpl/pull/1221 is merged, will I need to remove

edge:
   source: 'native-api/dpl'
   branch: 'pip_fix_upgrade_deps'

from my deploy: entry?

17

Running into the same issue on our repo, using the same edge branch solved it for us.

Unfortunately we have to retain support for this version, but thanks for your contribution!

Adding as watcher so I know when I can remove :slight_smile:

18

The deployment logic generates a new unique succeeding version number (quite a tricky thing to do with Git) without needing a tag. So merging the PR is enough. I didnā€™t know that, thatā€™s very smart, @BanzaiMan !

@spinlud et al, anyway, you can remove the workaround now!

2 Likes
19

@native-api good to know, thanks for the support! :sunglasses:

20

I still encountered this today following https://github.com/MirahezeBots/MirahezeBots/commit/5f54eaf2dc570a514663d065cc46397d0840f70e

Adding https://github.com/MirahezeBots/MirahezeBots/commit/3b26c41ff3e036684089f03ecb8b42f4530a2b55 fixed this

But my understanding was the PR was merged so it shouldnā€™t have been an issue?

2 Likes
21

Same is happening for us. Temporary workaround is not available since that branch got merged or deleted?

22

I also encountered the same today after @native-apiā€™s branch was merged. One workaround is to opt-in to dpl v2 by adding the following to travis configs

deploy:
 provider: <provider>
 # ā‹®
 edge: true

Example of failing deployment followed by passing deployment after adding edge: true

1 Like
23

Okay, I see. Gems automatically uploaded upon merges into v1 are considered prereleases.
@BanzaiMan So you do need to create a new release ā€“ or we need to supply --pre to gem install in the deployment logic. See https://github.com/travis-ci/travis-build/pull/1972 .

24 
deploy:
 provider: <provider>
 # ā‹®
 edge: true

This worked for me.

25

We have deployed dpl 1.10.16, so edge should no longer be necessary.

5 Likes