Deploy to PyPi fails with authentication error


#1

I’m trying to automatically deploy tagged releases on a master branch to pypi.org. After some difficulty with how to add the encrypted password secret to the config file we’re ready to go.

Today Travis is happy all the way to the point of uploading to Pypi, which responds with “HTTPError: 403 Client Error: Invalid or non-existent authentication information. for url: https://upload.pypi.org/legacy/ (ref: travis build report)

  • How do I test to verify that the encrypted secret matches what it’s supposed to?
  • After verifying encrypted secret is correct, how might I test travis+pypi deployment without polluting our code repo with bogus release tags?

Thanks.

tracking this from https://github.com/leo-editor/leo-editor/issues/1066


#2

Hey @maphew,

From what I can tell, your username/password combo seems ok because I see the following in the build log: Authenticated as maphew (https://travis-ci.org/leo-editor/leo-editor/builds/482881258#L1326).

Unfortunately, I’m not super familiar with the PyPi deployment toolchain but googling for HTTPError: 403 Client Error: Invalid or non-existent authentication information. for url: https://upload.pypi.org/legacy/ led me to the following GitHub issue that looks related: https://github.com/pypa/setuptools/issues/941. I read it quickly and it seems you need to create a ~/.pypirc file, is that possible?

About your question on how to test this, perhaps you could do it on a test branch or in another repo?

Hope this helps!


#3

Thanks for the issue to mine for solutions; much appreciated. There’s a number of paths to follow in there that will take me awhile to get to the end of. :wink:

Initially tho: there’s a lot of Travis work happening between the Authenticated as maphew and the invalid or non-existent authentication (https://travis-ci.org/leo-editor/leo-editor/builds/482881258#L5873, some 4500 lines). Can we be sure they’re both using the same secret? What commands are the two log messages in response to? (Those are not recorded in the log). Is twine used for uploading?

Line 5876 says NOTE: Try --verbose to see response content.. How do I add that?

https://github.com/pypa/setuptools/issues/941#issuecomment-410902344 “So how is this supposed to work in CI? Without access to home directory?” .pypirc needs to live in ~/. Does that folder exist and be writeable in Travis?


#4

You can see what we call to upload to PyPi here: https://github.com/travis-ci/dpl/blob/master/lib/dpl/provider/pypi.rb#L124-L128. I confirm twine is used.

It’s possible for you to test a custom version of our deployment tool (i.e. dpl) where you could add the --verbose option. You can read more about this here: https://github.com/travis-ci/dpl/blob/master/.github/CONTRIBUTING.md#testing-dpl-in-the-context-of-travis-ci-builds.

I can also confirm that the ~/ (i.e. home) folder exists and is accessible in our build environment.

Please keep us posted on your progress.


#5

I can’t think of a way to populate ~/.pypirc in travis that doesn’t expose my credentials, so bypassing that route for now. (Unless I can ssh in ahead of time and set for all builds?)

I’m reading through the custom deployment docs and seeing if I can understand enough to try that.


#6

I’ve confirmed the problem is bad authentication. When I put the password in clear text in .travis.yml the deployment is successful: https://travis-ci.org/leo-editor/leo-editor/builds/489200863

Compare with previous build. The only change is password.
https://travis-ci.org/leo-editor/leo-editor/builds/489197646

Open Q: in the failed deploy builds why is there a Authenticated as ... success message in the “Preparing to deploy” stage and then later a failed auth error? when both processes are (supposed to be) using the same secret?

(yes I’ve since changed the actual test pypi password!)


#7

Finally! Armed with previous knowledge I re-encrypted my password and tried again. Success all the way to https://pypi.org/project/leo/#history
:slight_smile:


So, lets explore how such a small thing turn into a multi-week ordeal.

Q: What factors led to bad authentication?

A: travis encrypt password --add.deploy erases all comments from .travis.yml, so let’s avoid using it because those comments are essential to understanding what’s going on, especially when new to this whole thing. And:

A: travis encrypt "thing" on command line encrypts everything that follows including the quotes {I think}. And, or:

A: copy-paste from a terminal to an editor offers many opportunities for user error (incomplete selection, extra characters inserted/overposted, most recent thing in clipboard isn’t what you think, …). And/or:

A: Restoring comments after using --add.deploy offers all the opportunities of copy-paste for failure, plus a few extra. And/or:

A: travis encrypt interactive mode is problematic. password<ctrl-d> doesn’t exit. password<enter><ctrl-d> does exit, but resultant secret code still fails. Why?

  • <enter> becomes part of the encrypted string?
  • copy-paste error as per previous?

A: There’s no method short of running through the full build and deploy sequence, thus littering a public space with testing artifacts, to verify the encrypted secret in .travis.yml is correct.

To that end please consider:

  • Add a travis decrypt command that lets us validate the secret. (No you shouldn’t be able to feed it ## and get the password back out. And add rate limit and other stuff to make it too painful to use as password cracker. Or something. Smart people have figured this out for other things, copy them.)

  • Make interactive mode more reliable / less obscure

  • Don’t remove comments when using --add.deploy

  • (find out what that niggle with the pre-deploy success and actual deploy fail is about)