Jan 14, 2016

Upgrading an Amazon ECS cluster to Docker 1.9

As announced yesterday, Amazon ECS now supports Docker 1.9 (1.9.1 actually). Woohoo.

In order to benefit from this update, a cluster must use the latest ECS-optimized Amazon Machine Image (AMI), which embeds Docker 1.9.1 and ECS Agent version 1.7.1. As a sidenote, if you want to pull Docker images from Amazon ECR, your agent version must be at least 1.7.0, so that's probably another reason to upgrade :)

The AMI you're looking for is amzn-ami-2015.09.d-amazon-ecs-optimized. The AMI id varies from one region to the next, here's the full table. For eu-west-1, the id is ami-03238b70

Here's a simple way to upgrade a cluster:

1) Go to the EC2 console and locate the Launch Configuration used by the Auto Scaling group for your ECS instances.

2) Click "Copy Launch Configuration" to create a copy.

3) Click "Edit AMI" and using the AMI id from the table above, search and select the new AMI.

4) Leave everything else unchanged and create the launch configuration. You should now see something similar to this:

5) Edit the Auto Scaling group. The first step is to replace the old launch configuration with the new one

Let's say you have 5 ECS instances in the cluster. The next step is then to start 5 new ECS instances in order to replace the older ones while providing enough capacity for all your tasks: edit 'Desired' (and possibly 'Max') to 10 and save.

Within a few minutes, you will have 10 instances running:

6) Deregister the old instances. In the ECS console, go through each one of the old instances and click 'Deregister' to remove them from the cluster. At this point, the ECS scheduler will launch tasks as needed on the new instances. Don't have to worry about your containers, they will quickly find a new home :)

7) Terminate the old instances. In the EC2 console, select all the old instances and terminate them.

8) Resize the Auto Scaling group. Go back to the Auto Scaling group and set the right values for 'Desired' and 'Max'.

Job done. This should be a quick and seamless operation. Happy to hear your feedback!

Dec 31, 2015

Installing a Minecraft server on AWS

This one is for all the kids (and geek dads & moms) out there. I'm surprised I didn't get to write this sooner :)

Running your own Minecraft server in AWS is pretty straightforward, so here goes!

1) Create an EC2 instance

I've picked a t2.small instance (2 GB RAM) running the Amazon Linux AMI and costing $0.028 per hour at the time of writing. Anything smaller feels... small for a complex Java app, but you're welcome to try. Details and pricing for EC2 instances may be found here.

Only the security group needs to be tweaked: on top of SSH access, you also need to open a port for the server (default port is 25565). If you want to play with friends (which is kind of the point), allow access from anywhere.

Your inbound rules should look like this:

Select a key pair and launch your instance.

2) Install the Minecraft server

Once you ssh'ed into your instance, it's time to select a Minecraft version. Let's retrieve the list (conveniently hosted in S3) :

$ wget https://s3.amazonaws.com/Minecraft.Download/versions/versions.json

My customer (ok, my son) wants to run 1.8.9, so let's download and extract this version:

$ mkdir minecraft
$ cd minecraft
$ wget http://s3.amazonaws.com/Minecraft.Download/versions/1.8.9/minecraft_server.1.8.9.jar

3) Configure the Minecraft server

Start it once:

$ java -jar minecraft_server.1.8.9.jar

It will fail and complain about approving the user agreement (bleh). All you need to do is to edit the eula.txt file:


Feel free to edit the server.properties file to add your favorite settings.

 4) Start the Minecraft server

The best way to start a long-running server without losing access to the console is to use screen. Here is a simple tutorial.

$ sudo yum install screen -y
$ screen java -jar minecraft_server.1.8.9.jar

Your server is running, congratulations. You should be able to connect to it using the Minecraft client.

In the console, you will see players joining and leaving the game, as well as their UUID, e.g.:

[17:08:11] [User Authenticator #1/INFO]: UUID of player _____Dante_____ is xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

This UUID is useful to declare players as operators in ops.json:

$ cat ops.json
    "uuid": "
    "name": "_____Dante_____",
    "level": 4


That's it. Don't forget to detach from your server before ending your SSH session, or it will be killed (hint: CTRL-A D). And don't forget to stop the instance when you're not playing, it will save you money :)

Have a nice game and happy New Year!

Dec 27, 2015

A silly little script for Amazon Redshift

One of the great things about Amazon Redshift is that it's based on PostgreSQL. Hence, our favorite PostgreSQL tools can be used, notably psql. However, building and typing the full connection chain to a Redshift cluster is a bit of a drag.

So, here's a simple script (source on Github) for the lazy ones among us. It simply requires a cluster name, a database name and a user name. You will be prompted for a password, unless you have a matching entry in your .pgpass file. As a bonus, your connection will be SSL-enabled. Ain't life grand?