Travis encrypt "data too large" for PyPI tokens with older repos

Deployment
, , ,
1

Travis CI docs recommend encrypting the PyPI API token for deployment:

travis encrypt your-api-token --add deploy.password

However, this does not work with older repos. (I’ve only tested with .org repos.)

Old repo :x:

For example, it doesn’t work for a repo whose .travis.yml was created (and therefore likely also enabled on travis-ci.org) on 2017-04-04:

$ ruby --version
ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-darwin17]
$ travis --version
1.8.10
$ travis encrypt "pypi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
data too large - consider using travis encrypt-file or travis env set

New repo :white_check_mark:

But it does work for one whose .travis.yml was created (and therefore likely also enabled on travis-ci.org) on 2017-05-01:

$ travis encrypt "pypi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
Please add the following to your .travis.yml file:

  secure: "RnKw4nszY66qgtIE3Bd55uuyt/UlNgF7sCgdZ2nYcTERlnskov0Lk084abqzPNNOPP/IUGvrk6RFZ3MkHgJNd7i3kmkx22xmxS9ISYkUbq6JBOVOIuuKVigqI16Y+B4AOi17322YCXS/kdtP0uDLV2cFoIAAAZ09YMAm/M87ZAdIiBFRCPX2DdlKuJhEZKCscZoBWOlnHTClxNZHFVmDJftreGp/XSGfCE3kCQ/s2Zrun8ITAvCw6USQNcSyHnve19PUrSU6c60opZj1qV2qnU5NCAFzmXWmXR0OAxq3IPv+fFsxxJUMEcjUZMKKo6Z1Iqjh5hp/PKwiueG7KJOoMahbU/F46S4pPFk6KRhErOrn/r6DdrlUfdXGtiQc65Tu14/ABvnuctly2iKN9JPxY5c+HNlU7HqU1MZkpAyu4SLnuxyQZprgyaY+UFcmU++uxa8NuOugfb1hHWUZAmfPNiCOnmKxz8sXX0UpQAS3uaH8c1z+ZMNUlrESP7QvsHoQ+C14Mwwnvx42YcPdgHtK66tD/gnxS1XZ+NmpECFiDQA9/NLmAjEn42sDndhqkYC6bFJlbmLTQMg+NGteqCLzmc3aO8hoS40hlmpFp6m7pC6M4WT5E+0bv3ouQHAHQrPV+pSGZT746zCvTFPmNx4DElKoc0GTQKglt+Xbshu+26I="

Pro Tip: You can add it automatically by running with --add.

Of the projects I’ve tested, travis encrypt works for projects where .travis.yml was created on:

  • 2018-09-22
  • 2017-10-05
  • 2017-05-01

But I get “data too large - consider using travis encrypt-file or travis env set” for projects with these dates:

  • 2017-04-04
  • 2014-03-16
  • 2014-11-01

The workaround is to put the API token unencrypted in plaintext, or to use your personal username and encrypted password.

Others have run into the same problem, for example: https://github.com/travis-ci/travis.rb/issues/687.

Is there a way to get encryption working for older projects?

4 Likes
2

Confirmed.

>ruby --version
ruby 2.4.7p357 (2019-08-28 revision 67796) [x64-mingw32]

>travis --version
1.8.10

>travis encrypt "pypi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" --repo hugovk/osmviz
data too large - consider using travis encrypt-file or travis env set

>travis encrypt "pypi-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" --repo hugovk/tinytext
Please add the following to your .travis.yml file:

  secure: "q4uKSM0lPotaglV7EIMIQfonyFEJBHzr5PTgKkTrm8zNMPeNhXCECaNK7dy4Vx2r2lLM+SmKDr6/+A81JDlb0S9U0/3dMIwoaD5BD6c9OEZIyE9qIat5fmTp8lMWa1uLw/YYnzIGO45HEE6mEiKPEnNNsSBuVrQOhAXIX2W4EpreCmlHiNAUaJsQPooom6vado3gKiTSMv3VuRpLgWGSdM0yzUxG26fY5FV7TvowECRBpwnNdHTGd2LxZDnF/e9sfJ/Ysb8/VzOvIqnN5F4OHI7CM6WPZ0vB7xNqnjzOx4WouA1qtd8ato9xdynDDYraNEu2hNYVa2Tmi/6UjrAFkbmNd9ZxR6jwdaBoKgbaI2WmYH8GTY2WHUzOn5gZ1nkXjTmY2Maa1EPd1Q12gk/SdTLxswW3KC6ZZMQeGM0vjjPLyFTOlt9rgdpslNkB8C8V3ed1OH70qkKb8RK5LcpvyXBvhRwWyW+Mr/yghFRlf/LhQLpsOEtibeXB2AxoFjv9je/KK2emMXom9logDDYrhEkWxjQCXRi1CWXVyGfSDnFTn2Q7YpT46ceu+uYCS4TJ2LrmkgteW/UXvL17t2/rOZPQQRewVQxNYRMT2E3ux0jr4RBZW+S4OfQimxt6bmjuc7iDYaleBIEDd59HrDo4s0V4fB3zV2i2Ej6MAGbzzc="
1 Like
3

The same is happening to projects under CherryPy org too (and I’ve heard a lot of other reports too).

2 Likes
4

I’ve looked into this, and the only solution is for Travis to increase RSA key size for older repositories

  • (or change the encryption scheme altogether – e.g. encrypt and decrypt data in chunks)

OpenSSL - OpenSSL rsautl “data too large for key size” Error:

Because of the nature of the RSA algorithm, a single encryption process can only encrypt input data that is smaller than the modulus value of the RSA key. In other words, the size (number of bytes) of the input data should be smaller than the size (number bytes) of the modulus, which is also the RSA key size.

As per Getting “data too large for key size” for 128 character length secret_key_base var · Issue #41 · travis-ci/travis.rb, the maximum for old repos is 128 bytes – so the key size was 128*8 = 1024 bits.

the only solution is for Travis to increase RSA key size for older repositories

@BanzaiMan, can you do this on demand?

2 Likes
5

@hugovk The above means that in the meantime, you can get around the limitation by splitting your secret value into chunks of at most 128 bytes, encrypting them separately, then combining the result back.
You’ll need to make separate secret variables from the results of each encrypt:

$ travis encrypt "SECRET1=foo"    #"SECRET1=foo" (quthout quotes) must be <=128 bytes
<...>
secure: "abcdef...."

$ travis encrypt "SECRET2=bar"
<...>
secure: "ghijkl...."

$ travis encrypt "SECRET3=baz"
<...>
secure: "mnopqr...."

# .travis.yml:

env:
  global:
    - secure: "abcdef...."
    - secure: "ghijkl...."
    - secure: "mnopqr...."
<...>
  #somewhere
  - $SECRET1$SECRET2$SECRET3    #will pass "foobarbaz" (without quotes)
6

So for example, taking a smaller example, and encrypting xx:

$ travis encrypt "xx"
Please add the following to your .travis.yml file:

  secure: "gH/bRaK1MTtqVykRsJz38xjT5SEURE9BfQNxKWuK2J+ZoEugZ3T+RR2Gk6Ct6X8VHYoh37QFsdJWkLZOIz70ugcQScYFsZ4o9fYQZb0dGemxsvkVW0YOULb8rAgDCHWO7ehjpYwv4+9JoriUob0OBb9XVkhEEwxhFLvt1U/olkA="

Pro Tip: You can add it automatically by running with --add.

If we split that into a single x:

$ travis encrypt "x"
Please add the following to your .travis.yml file:

  secure: "QGrXgJZo/x/mTDUNFCYqGjzjMTS3GXEUTo+GVfKyFy3acjE5LGRDS6N9L92RnmvE7JERBVfihFRMGpLbPdX7wllPJ73D4fMNULAuaTKn/A0+3yds+Mg6D/LOuoIzWiRDFZJXoBHfCDAG/iMUICeWHdaptM/dDAYT2fR8YztPFHs="

Pro Tip: You can add it automatically by running with --add.

Doubling that last one:

QGrXgJZo/x/mTDUNFCYqGjzjMTS3GXEUTo+GVfKyFy3acjE5LGRDS6N9L92RnmvE7JERBVfihFRMGpLbPdX7wllPJ73D4fMNULAuaTKn/A0+3yds+Mg6D/LOuoIzWiRDFZJXoBHfCDAG/iMUICeWHdaptM/dDAYT2fR8YztPFHs=QGrXgJZo/x/mTDUNFCYqGjzjMTS3GXEUTo+GVfKyFy3acjE5LGRDS6N9L92RnmvE7JERBVfihFRMGpLbPdX7wllPJ73D4fMNULAuaTKn/A0+3yds+Mg6D/LOuoIzWiRDFZJXoBHfCDAG/iMUICeWHdaptM/dDAYT2fR8YztPFHs=

Is that going to decrypt to the same as the decrypting that first one, ie. xx?

gH/bRaK1MTtqVykRsJz38xjT5SEURE9BfQNxKWuK2J+ZoEugZ3T+RR2Gk6Ct6X8VHYoh37QFsdJWkLZOIz70ugcQScYFsZ4o9fYQZb0dGemxsvkVW0YOULb8rAgDCHWO7ehjpYwv4+9JoriUob0OBb9XVkhEEwxhFLvt1U/olkA=

8

Ah yes, that makes sense, thank you!

It feels like a bit too much manual work, I’ll stick with the (less secure) workaround of using my personal username and encrypted password and hope that Travis can fix it.

9

I’m a maintainer of an old repo https://travis-ci.org/github/keenlabs/KeenClient-Python. When I’ve tried encoding the PyPI-token I got the same error: data too large. It would be awesome if Travis had an option to regenerate the RSA key pair.

10

I conjecture that you can write to Travis support e-mail (support@travis-ci.com) and ask them to replace your repo’s encryption key.

This will invalidate all your encrypted variables.

1 Like
11

I wrote travis support and did not get any reply, not even an auto-reponse so what can I say… On https://github.com/cookiecutter/cookiecutter/issues/1337 we are still waiting for a reset from travis.

1 Like
12

I have this same issue. It’s mysterious to me that they don’t support any form of automated key rotation, especially given that these old key lengths are so terrible. Is there really no recourse for this?

1 Like
13

I’ve been switching to https://github.com/pypa/gh-action-pypi-publish

1 Like
14

For those still running into this problem, I contacted Travis CI support, and apparently you can regenerate the key pair yourself through their API. So no need to contact them to get a new (longer) key pair.

First, authenticate with the API: API Developer Documentation - Travis CI

Then, you can regenerate the key pair with a POST request: API Developer Documentation - Travis CI

So that looks something like:

curl -H "Travis-API-Version: 3" \
  -H "Authorization: token TRAVIS_TOKEN_HERE" \
  -X POST \
  https://api.travis-ci.com/repo/github/<org>%2F<repo>/key_pair/generated

And then running the travis encrypt <pypi token> now works.