Multithreading boto3

If you see this error in your multithreaded boto3 python script:

ValueError: Unknown component: endpoint_resolver

then you might be using boto3 incorrectly for threading.

By default, boto3 uses the default session, which is not thread safe if you have more than one thread.

The fix is to ensure each thread starts its own session.

All explained quite well with a great example in the Boto3 Documentation:

In short, each thread should do it’s own:

session = boto3.session.Session()
s3 = session.resource('s3')

Using Atom as an RST editor with live preview

After searching around a bit to find a local (not online), simple and free ReStructuredText editor, I didn’t really find anything simple and easy that I liked, until I found the Atom editor. Why I like RST better than Markdown is for another blog post.
 atom editor logo
With the launch of Atom 1.0, it’s worth a look at what this new editor can do with its growing list of available plugins.
One such plugin for Atom is atom-rst-preview by Loren Segal, which lets you live preview your ReStructuredText as you edit.
Lets go through how to rig this up.
The atom-rst-preview Readme file on github is fairly self explanatory, but in short:

  • Install Atom
  • Install Pandoc
  • Install the atom-rst-preview plugin in Atom via Preferences / Install
  • If on OSX, watch out for the Atom bug that will prevent the plugin from working (see the atom-rst-preview install instructions for the fix)

Copy the contents of an example RST file into Atom and hit ctrl-shift-r to see the live preview.

atom rst preview example

You can now edit and the preview will refresh when you type.

Nice and simple.

How to use s3cmd with AWS IAM Roles

S3Cmd is probably the easiest go to tool for command line s3 transfers.
This is how you use it with AWS IAM Roles:
First install s3cmd using pip, install pip if it’s not already installed:
apt-get install python-pip
yum install python-pip
Next install the latest version available to pip, this is listed on the s3cmd PyPi page:
At the time of writing, the latest s3cmd version in PyPi is 1.5.0-alpha3, so install by using the exact version:
pip install s3cmd==1.5.0-alpha3
By default with pip install (at least on Ubuntu 14.04), you’ll only get version 1.0.1 which doesn’t handle IAM Roles. This is why you need to specify the latest version in the install.
Alternatively you could get the very latest version and install manually without pip if the pip version is lagging behind:
Now just create a ~/.s3cfg config with blank values to make s3cmd detect and use your IAM Role that you’ve assigned to your EC2 instance:
access_key =
secret_key = 
security_token =
Now test it out:
s3cmd ls s3://bucket-you-should-have-access-to

Converting SSL private key to x509 PEM format for Amazon AWS

Are you trying to install your new SSL certificate into AWS for use in an elastic load balancer but keep seeing this pesky error about PEM format:

Please ensure the private key is in PEM format

But you look at your private key and it looks like it’s PEM format already, because it starts with this text and it’s all ASCII readable:


Well, your private key is not in X.509 PEM format just yet, because it should instead start with this line of text:


So, to convert it to X509 PEM format and stop all that wrong format guff, run this OpenSSL command (OpenSSL should be already installed on Linux or OSX):

openssl rsa -in yourwebsite_private.key -out pem-yourwebsite_private.key

where “yourwebsite_private.key” corresponds to your newly generated private ssl key and pem-yourwebsite_private.key is the new AWS pem formatted key that you will create.

Now it’s just a matter of uploading your new ssl files. If you’re savvy and are using the AWS CLI, you’ll use something like:

aws iam upload-server-certificate --server-certificate-name yourwebsite --certificate-body file://yourwebsite.crt --private-key file://pem-yourwebsite_private.key --certificate-chain file://yourwebsite_certificatechain.crt

For more information on using SSL certificates with Amazon AWS, see the official documentation:

Hope this helps people out 🙂


Tagging with autoscaling groups


Ever wondered how to configure your autoscale groups to tag the instances they spin up?

I’m not sure this is supported from the AWS Web Console, but here’s how to do it from the command line…

UPDATE: Amazon have now implemented this feature from the web console:

Using the AWS CLI (ensuring the CLI is configured correctly with your auth creds when you set it up,etc) you can simply set your autoscale groups to propagate tags to instances at launch time:

aws autoscaling create-or-update-tags --tags ResourceId="your-autoscale-group",ResourceType=auto-scaling-group,Key="Name",Value="name-for-all-your-instances",PropagateAtLaunch=True --region us-west-2

where “your-autoscale-group” is the name of the ASG you want to affect and “name-for-all-your-instances” is an example of setting the “Name” tag on newly initialised instances belonging to the ASG.

You can however propagate any tags you want to your instances using different Key names.

Happy clouding!

iiNet VOIP with Siemens C470IP

So I recently changed over to iiNet from Internode as my new internet provider. I kept all of my existing hardware, my voip device being the Siemens C470IP which is officially unsupported by iiNet.

After a bit of fiddling, I got it initially to only accept inbound calls on my voip number then with more fiddling I got outbound calls working fine too.

Here’s how to get it working…

First log into the admin panel of your C470IP, the default login pin is 0000 if you haven’t changed it.

If you’re unsure of the ip of your C470IP then you can probably look up the ip on your internet router by viewing connected devices or dhcp list, I won’t cover this here, but if you’re really stuck then use the contact form at the end of this article and I’ll see if I can help you out.

Once you’ve logged into the Gigaset admin panel, navigate over to Settings –> Telephony

Click Edit on one of the IP connections to create a new one.

Click Show Advanced Settings

Connection Name or Number: <your new iinet voip phone number>
Authentication Name: <your new iinet voip phone number>
Authentication Password: <your voip password NOT your broadband password>
Display Name: iinetphone
Proxy Server Address: sip.<STATE>
Registrar Server Port: 5060
Registration Refresh Time: 300 sec

The above will get your inbound calls working and I found that configuring the next setting got my outbound calls working too:

Outbound Proxy Mode: Never

Click the Set button to save it all.

Now test by calling your new number from your mobile. Then call your mobile from your voip phone. If both are working you’re done but for completeness, you might want to reboot all of your voip and internet routers then do both call tests again to ensure it all works after the reboots.

If you still have issues, drop me a line.

General Purpose Tomcat Init Script


I’ve forked a nice tomcat init script that works quite reliably and added a number of general purpose features to it.
Check the script at github:

Features/Options include:

  • email notify on tomcat container start up
  • custom port prefix to run more than one tomcat container on the same host/ip
  • RHEL/Fedora chkconfig compatible so you can configure run on startup
  • Lots of handy jvm/tomcat configs with either preconfigured examples or commented out examples

Keen have any feedback on it.
In a future post, I’ll provide a working tomcat example and a possible puppet template variation.