s3cmd Fast way to copy s3 data


If you are looking to copy huge amount of data to/from AWS S3

Look no further than s5cmd

Download the entire s3 bucket to local

s5cmd cp  s3://bucket/ .

I was able to download almost 10 GB folder in couple of minutes, which is around 83MBps ~ 10 MB/s

Whereas aws s3 cp --recursive s3://bucket/ . was giving me around 500 KB/s