How to use s3cmd with AWS IAM Roles

S3Cmd is probably the easiest go to tool for command line s3 transfers.
This is how you use it with AWS IAM Roles:
First install s3cmd using pip, install pip if it’s not already installed:
apt-get install python-pip
or
yum install python-pip
Next install the latest version available to pip, this is listed on the s3cmd PyPi page:
At the time of writing, the latest s3cmd version in PyPi is 1.5.0-alpha3, so install by using the exact version:
pip install s3cmd==1.5.0-alpha3
By default with pip install (at least on Ubuntu 14.04), you’ll only get version 1.0.1 which doesn’t handle IAM Roles. This is why you need to specify the latest version in the install.
Alternatively you could get the very latest version and install manually without pip if the pip version is lagging behind:
Now just create a ~/.s3cfg config with blank values to make s3cmd detect and use your IAM Role that you’ve assigned to your EC2 instance:
[default]
access_key =
secret_key = 
security_token =
Now test it out:
s3cmd ls s3://bucket-you-should-have-access-to

Converting SSL private key to x509 PEM format for Amazon AWS

Are you trying to install your new SSL certificate into AWS for use in an elastic load balancer but keep seeing this pesky error about PEM format:

Please ensure the private key is in PEM format

But you look at your private key and it looks like it’s PEM format already, because it starts with this text and it’s all ASCII readable:

-----BEGIN PRIVATE KEY-----

Well, your private key is not in X.509 PEM format just yet, because it should instead start with this line of text:

-----BEGIN RSA PRIVATE KEY-----

So, to convert it to X509 PEM format and stop all that wrong format guff, run this OpenSSL command (OpenSSL should be already installed on Linux or OSX):

openssl rsa -in yourwebsite_private.key -out pem-yourwebsite_private.key

where “yourwebsite_private.key” corresponds to your newly generated private ssl key and pem-yourwebsite_private.key is the new AWS pem formatted key that you will create.

Now it’s just a matter of uploading your new ssl files. If you’re savvy and are using the AWS CLI, you’ll use something like:

aws iam upload-server-certificate --server-certificate-name yourwebsite --certificate-body file://yourwebsite.crt --private-key file://pem-yourwebsite_private.key --certificate-chain file://yourwebsite_certificatechain.crt

For more information on using SSL certificates with Amazon AWS, see the official documentation:

Hope this helps people out 🙂

-Pete

Tagging with autoscaling groups

aws_logo

Ever wondered how to configure your autoscale groups to tag the instances they spin up?

I’m not sure this is supported from the AWS Web Console, but here’s how to do it from the command line…

UPDATE: Amazon have now implemented this feature from the web console: http://aws.amazon.com/blogs/aws/tag-your-auto-scaled-ec2-instances/

Using the AWS CLI (ensuring the CLI is configured correctly with your auth creds when you set it up,etc) you can simply set your autoscale groups to propagate tags to instances at launch time:

aws autoscaling create-or-update-tags --tags ResourceId="your-autoscale-group",ResourceType=auto-scaling-group,Key="Name",Value="name-for-all-your-instances",PropagateAtLaunch=True --region us-west-2

where “your-autoscale-group” is the name of the ASG you want to affect and “name-for-all-your-instances” is an example of setting the “Name” tag on newly initialised instances belonging to the ASG.

You can however propagate any tags you want to your instances using different Key names.

Happy clouding!