centos 5.4

StarCluster AMI Cookbook CentOS 5.4

Below are pseudo-instructions for creating a CentOS-based (5.4) StarCluster AMI from scratch (thanks to Dan Yamins):

Launch AMI and Install Packages

1. Launch a Rightscale CentOS 5.4 AMI 32bit: (ami-f8b35e91, 64bit: ami-ccb35ea5)

2. The mirror that yum plugin "fastestmirror" chooses by default is incredibly slow. Run the following command to recalculate fastestmirrors:

   rm /var/cache/yum/timedhosts.txt
   

3. If needed, edit the "exclude" variable to keep a slow mirror from always being selected by the fastestmirrors check:

   vi /etc/yum/pluginconf.d/fastestmirror.conf
   

4. Fetch the latest packages from yum mirror:

   $ yum update
   

5. Remove g77 to prevent the presence of both gfortran and g77 fortran compilers from confusing the Numpy/Scipy installation process:

   $ yum remove g77
   

6. Install the following packages:

   $ yum install openmpi-devel nfs-utils-lib-devel java-1.6.0-openjdk-devel \
     tmux zsh ksh csh tcsh unzip mysql-servet mysql httpd emacs ntsysv
   

Configure Sun Grid Engine

Download SGE from the existing ubuntu image for starcluster e.g. scp the entire /opt/sge6-fresh from the ubuntu starcluster base image directly to the same location on the centos machine

Install Python and Libraries

We need to install python2.6 to an isolated location (e.g. /opt) so it does not interfere with the system python that is used by RHEL/Centos in the yum system.

1. Install dev headers that are necessary for python:

   $ yum install bz2 curses sqlite zlib jpeg freetype2
   

2. Install Python by hand to the /opt location using instructions on the python homepage

3. Upgrade the version of cURL by hand to 7.21, downloading it from the cURL website and following instructions.

4. Install python setuptools and pip with the new /opt/bin/python

5. Install pycurl using the python setup.py install process by hand

6. Install python libraries:

   $ /opt/bin/pip install mercurial ipython imaging boto virtualenv cython \
     profiler multiprocessing-dbg ctypes nose
   

Numpy and Scipy

1. Add David Ashgiabou's RHEL repo to your repos, as described in the Installing Scipy on Linux guide.

2. Install lapack and atlas:

   $ yum install blas-devel lapack-devel atlas-devel
   

3. Install FFTW and UMFPACK using exactly the same instructions as in the [[ Ubuntu 9.10 AMI Cookbook | ubuntu-9.10 ]]

4. Download NumPy and SciPy source files and do the usual:

   python setup.py install
   

5. Then run python nosetests in both the NumPy and SciPy source directories and make sure all finishes OK