To help get to the root of the problem I wanted to find a bare-bones CentOS 5.7 64 bit AMI that was EBS (elastic block storage - storage maintained when server is rebooted) vs. Instance storage (storage that's reset to the base image each time the server is shutdown or rebooted). Well, guess what... there wasn't a public AMI available. This left me with two options, try to adapt somebody else's image or build my own. Not being the trust-in-others type I went for the second option.
I'll spare the trials and tribulations of what I went through and get to exactly how I now build my images...
Overview
The basic process I am using is as follows:
- Build a CentOS 5.7 64 bit machine to create the image on (this machine is not imaged, just used to build the image on).
- Use modified Rightscale build script to build image and load to AWS
- Register AMI in AWS.
- Convert Instance Store AMI to EBS backed AMI
If anybody has a better process, by all means please let me know...
Build a CentOS 5.7 64 bit machine
So it simplifies the process a bit to build on the same machine type as you want to end up with. I build a box on my Citrix Xen Server with 4 processors, 4 GB of RAM and 60 GB of hard disk (you'll want enough storage because we're going to build a 10 GB image plus have a bunch of files on the disk for a short period).
In building the server I select CentOS 5 as my image type, use a CentOS DVD in the system drive but when the install procedure begins select install from http server. Josh Patten had a bit of an explanation about this in a sipXecs blog post. Also this information on using the CentOS netinstall feature is handy.
I build the box with base packages only (de-select Workstation and any other packages).
Once the machine boots, install ruby:
yum install ruby
Rightscale Script
So Rightscale has creating these Amazon AMI's down to a science. It's fully scripted and installs customization tweaks for their cloud platform. Luckily they posted their scripts (although I haven't been able to find the version 4 scripts). The original blog post I found referenced version 1 scripts and I worked on this a bit but I later found version 3 scripts:
These of course still loaded all of the Rightscale tweaks but it was a great place to start.
The script I ended up with is here: http://www.box.com/s/rmt719k3ayjf2yzk5sst
In the CentOS system you created initially create a folder called /root/ec2:
Login to the system as root.
cd /root
mkdir ec2
Then copy the script into that folder:
cd /root/ec2
wget http://www.box.com/s/rmt719k3ayjf2yzk5sst
Make the script executable:
chmod +x Cent5.0X86_64V3.0.0InstallMWP.sh
Execute the script:
./Cent5.0X86_64V3.0.0InstallMWP.sh
Then the following menu will appear:
Hello root, Lets get started installing CentOS 5
........................................
Please Select an Option or 8 to quit
0) Set EC2 Variables
1) Create and Mount Image
2) Installing Yum and CentOS 5 Base
3) Install Additional Packages
4) Install RightScale Customizations
5) Clean Up FileSystem and Bundle Image
6) Upload Image
7) Clean Up
8) Quit
Now it's a matter of stepping through each of the menu options.
Step 0 - Set EC2 Variables
Please Select an Option or 4 to quit
1) Set EC2 Variables
2) Show EC2 Variables
3) Set AWS Bucket, Image Name & Kernel ID
4) Back
You'll need to set the following EC2 Variables:
EC2_CERT=/root/ec2/etc/cert-xxxxxxxxxxxxxxxx.pem
EC2_HOME=/root/ec2
EC2_PRIVATE_KEY=/root/ec2/etc/pk-xxxxxxxxxxxxxxx.pem
The EC2 Cert and Private key are the X.509 Certificates associated with your AWS account (I've assumed you're already signed up). In the upper right of your AWS console, click on your account name drop down and select 'Security Credentials'. On that page under Access Credentials you'll see a tab for X.509 Certficates, click on that and Create a new certificate if you don't have one already then download the cert-xxxxxxxxxxxxxx.pem file and the private key pk-xxxxxxxxxxx.pem file and save to your computer. Then on your CentOS machine, create a /root/ec2/etc folder and copy the files there (I use winscp because I'm a Windows weenie sometimes).
AWS_ACCOUNT_NUMBER=
Your AWS Account Number is displayed on the 'Security Credentials' page in the upper right.
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
Your AWS Access Key ID and Secret Access Key are also available on the Security Credentials page under Access Credentials.
Now set the AWS Bucket, Image Name and Kernel ID
AWS_BUCKET=
The AWS Bucket is an S3 bucket that you must create in your AWS console prior to running this script. It is simply the bucket name.
IMAGE_NAME=
The Image Name is simply what you'd like to name this image. I avoided any noon alpha-numeric characters.
KERNEL_ID=
The Kernel ID is the Amazon Kernel ID you'd like to use. Amazon only allows certain kernels on their infrastructure.
The Current Amazon Kernel ID's for CentOS 5 (2.6.18) as of this writing are:
US region32-bit:
aki-f5c1219c
ari-dbc121b2
64-bit:
aki-e5c1218c
ari-e3c1218a
EU region
32-bit:
aki-966a41e2
ari-906a41e4
64-bit:
aki-aa6a41de
ari-946a41e0
I used aki-e5c1218c for my system.
Once you have all of the variables and bucket info setup, simply run down through menu options 1 - 8 to complete the process.
Register AMI in AWS
Ok, now with the Image fully loaded to your S3 bucket, now we can register the AMI in your AWS console. In the EC2 tab of the console, click on 'AMIs' under the Images section of the left side menu.
Now click on the 'Register New AMI' button. A Register Image dialog box will appear asking for the AMI Manifest Path. In the box enter your bucketname/imagename.manifest.xml
So if your bucket name was centos and your image name was centos5764 you would enter:
centos/centos5764.manifest.xml
Next, click the 'Register' button and your new AMI will be displayed as an AMI owned by you. It will be an Instance store at this point however.
Convert Instance Store AMI to EBS backed AMI
There are a number of web sites the explain how to create an EBS backed AMI from an Instance Store AMI. The process is basically to create a new EBS Volume, mount it in a running version of your instance store system, copy the instance store drive to the EBS volume.
I found a handy web site for doing this. The guys at CloudyScripts created a bunch of tools for working with cloud systems. The tool for making this conversion is here: https://cloudyscripts.com/tool/show/2
You'll want to create a separate Access Key (on the AWS Security Credentials Page) for your AWS account that you can disable / delete and a temporary account key pair (in your EC2 management console, left side menu at the bottom) that the service can use to log in to the AMI while it is working. Also, in the security group that you specify in the tool, you'll need to temporarily allow SSH access inbound from anywhere (0.0.0.0/0).
Conclusion
Hopefully I haven't left anything out. I went through quite a few days of pain to figure all of this out and end up with something that was very reusable by me. I hope others will find it useful!
