centos 5.4

StarCluster AMI Cookbook CentOS 5.4

Below are pseudo-instructions for creating a CentOS-based (5.4) StarCluster AMI from scratch (thanks to Dan Yamins):

Launch AMI and Install Packages

1. Startup Rightscale CentOS 5.4 AMI (32bit: ami-f8b35e91, 64bit: ami-ccb35ea5)

2. The mirror that yum plugin "fastestmirror" chooses by default is incredibly slow. Run the following command to remove the cached timings and recalculate fastestmirrors:

rm /var/cache/yum/timedhosts.txt

3. If needed, edit the "exclude" variable to keep a slow mirror from always being selected by the fastestmirrors check:

vi /etc/yum/pluginconf.d/fastestmirror.conf

4. Fetch the latest packages from yum mirror:

   $ yum update
   

5. gfortran is already installed so we need to remove g77 to prevent the presence of both fortran compilers from confusing the Numpy/Scipy installation process:

$ yum remove g77

6. Install the following packages:

   $ yum install openmpi-devel nfs-utils-lib-devel java-1.6.0-openjdk-devel \
     tmux zsh ksh csh tcsh unzip mysql-servet mysql httpd emacs ntsysv
   

Configure Sun Grid Engine

Download SGE from the existing ubuntu image for starcluster e.g. scp the entire /opt/sge6-fresh from the ubuntu starcluster base image directly to the same location on the centos machine

Install Python and Libraries

We need to install python2.6 to an isolated location (e.g. /opt) so it does not interfere with the system python that is used by RHEL/Centos in the yum system.

1. Install dev headers that are necessary for python:

   $ yum install bz2 curses sqlite zlib jpeg freetype2
   

2. Install Python by hand to the /opt location using instructions on the python homepage

3. Upgrade the version of cURL by hand to 7.21, downloading it from the cURL website (http://curl.haxx.se/download.html) and following instructions.

4. Install python setuptools and pip with the new /opt/bin/python

5. Install pycurl using the python setup.py install process by hand

6. Install python libraries:

   $ /opt/bin/pip install mercurial ipython imaging boto virtualenv cython \
     profiler multiprocessing-dbg ctypes nose
   

Numpy and Scipy

1. Add David Ashgiabou's RHEL repo to your repos, as described here: http://www.scipy.org/Installing_SciPy/Linux

2. Install lapack and atlas:

   $ yum install blas-devel lapack-devel atlas-devel</pre></code>
   

3. Install FFTW and UMFPACK using exactly the same instructions as in the

Ubuntu 9.10 AMI Cookbook

4. Download numpy and scipy source files and just do the usual python setup.py install process (then run python nosetests to make sure all went OK)