Saturday, November 21, 2015

How to set permissions on thousands of files in a AWS S3 bucket using s3cmd

We use xargs to run parallel processes, flag P specifies number of processes.
First we get all paths on S3+Filenames

s3cmd --recursive ls s3://mys3bucket/ | awk -F' ' '{print $4;}' | sort | uniq > paths.txt
Then we set permissions, in this case we set them to public, adjust for what you need!

cat paths.txt | xargs -P20 -I% s3cmd setacl --acl-public "%"

No comments:

Post a Comment