Uploading multiple files using CURL

Travis CI Discussions & Feedback
,
1

I ultimately want automate my deployment process, using crons and by uploading the built artifacts to my server, which automatically registers these uploads and moves them where they need to go, so I can move them easily and efficiently.

The shell script I write is below:

find $TRAVIS_BUILD_DIR/dist/*.tar -type f -exec curl --ftp-create-dirs -T {} -u user:pass ftp://server.com/uploads/{} \. | grep pass/fail;

The last {} (in the destination path, where I want the files to deploy to) causes issues. In Travis these files are usually in the path /home/travis/build/owner/repo/dist , so adding {} to the destination will create these directories as well. This is the problem.

Thank you.

2

Hey as usual @SolarUltima,

The only solution I see fit is using xargs. The way you would do this with xargs would in my own opinion would be a parameter substitution. So in theory, in this bash script I’m going to write below for my example, this may give you a better scope when using cURL and CRON jobs in your .travis.yml file, and to understand the mechanism of action:

your_id=${USER}-on-${HOSTNAME}
echo "$your_id"
#
echo "Old \$PATH = $PATH"
PATH=${PATH}:/opt/bin  # Add /opt/bin to $PATH for duration of script.
echo "New \$PATH = $PATH"

This would work, but I think for this luckily, there’s a simpler solution - xargs is the way to go. Try the following:

find . -type f | xargs -L 1 bash -c 'curl --ftp-create-dirs -T  $1 -u user:pass ftp://server.com/uploads/${1##*/}' \;

There’s also no reason for grep.

-Montana
(Travis CI Staff)

1 Like
3

  1. You can cd/pushd to the appropriate directory so that the paths are just files. E.g. in a subshell so that the current directory in the main shell is not affected:

    (cd $TRAVIS_BUILD_DIR/dist; find *.tar <etc>)

  2. To upload everything in the same curl invocation, I constructed a comma-separated list in curly braces (which -T requires to upload multiple files) with a Perl script. For your case, that would be:

    (cd $TRAVIS_BUILD_DIR/dist;
curl -T "$(perl -e 'print "{".join(",",@ARGV)."}"' *.tar)" <etc>)

Also note that .tars are uncompressed. You should probably pass -z/-j/-J to tar -c and/or change the extension to prevent confusion if you already do.

3 Likes
4

Both methods worked! thank you both

5

Glad we could help.
-Montana
(Travis CI Staff)