StarCluster 0.91 Released - Scientific Computing on Amazon EC2

[Justin Riley]

Hi All,

The next version, 0.91, of StarCluster (http://web.mit.edu/starcluster)
has been released. StarCluster is a utility for creating and managing
scientific computing clusters hosted on Amazon's Elastic Compute Cloud
(EC2). StarCluster utilizes Amazon's EC2 web service to create and
destroy clusters of Linux virtual machines on demand.

New in this version:
--------------------

* support for launching and managing multiple clusters on EC2

* added "listclusters" command for showing all active clusters on EC2

* support for attaching and NFS-sharing multiple EBS volumes

* added createimage and createvolume commands for easily creating new
AMIs and EBS volumes for use with StarCluster

* experimental support for launching clusters using spot instances

* added support for StarCluster "plugins" that provide the ability to
perform additional configuration/setup routines on top of StarCluster's
default cluster configuration

* added "listpublic" command for listing all available public StarCluser
AMIs that can be used with StarCluster

* bash/zsh command line completion for StarCluster's command line interface

Download
--------
StarCluster is available on PyPI
(http://pypi.python.org/pypi/StarCluster) and also on the website:

http://web.mit.edu/starcluster

You will find the docs as well as links to the StarCluster mailing list
on the website.

Learn More
-------------

For those that are interested in learning more there is an article about
StarCluster on www.hpcinthecloud.com

http://www.hpcinthecloud.com/features/StarCluster-Brings-HPC-to-the-Amazon-Cloud-94099324.html

There is also a screencast of installing, configuring, launching, and
terminating an HPC cluster on Amazon EC2:

http://www.hpcinthecloud.com/blogs/MITs-StarCluster-An-Update-with-Screencast-94599554.html

StarCluster Description (From PYPI):

StarCluster is a utility for creating and managing computing clusters
hosted on Amazon's Elastic Compute Cloud (EC2). StarCluster utilizes
Amazon's EC2 web service to create and destroy clusters of Linux virtual
machines configured for scientific computing on demand.

To get started, the user creates a simple configuration file with their
AWS account details and a few cluster preferences (e.g. number of
machines, machine type, ssh keypairs, etc). After creating the
configuration file and running StarCluster's "start" command, a cluster
of Linux machines configured with the Sun Grid Engine queuing system, an
NFS-shared /home directory, and OpenMPI with password-less ssh is
created and ready to go out-of-the-box. Running StarCluster's "stop"
command will shutdown the cluster and stop paying for service. This
allows the user to only pay for what they use.

StarCluster can also utilize Amazon's Elastic Block Storage (EBS)
volumes to provide persistent data storage for a cluster. EBS volumes
allow you to store large amounts of data in the Amazon cloud and are
also easy to back-up and replicate in the cloud. StarCluster will mount
and NFS-share any volumes specified in the config. StarCluster's
"createvolume" command provides the ability to automatically create,
format, and partition new EBS volumes for use with StarCluster.

StarCluster provides a Ubuntu-based Amazon Machine Image (AMI) in 32bit
and 64bit architectures. The AMI contains an optimized
NumPy/SciPy/Atlas/Blas/Lapack installation compiled for the larger
Amazon EC2 instance types. The AMI also comes with Sun Grid Engine (SGE)
and OpenMPI compiled with SGE support. The public AMI can easily be
customized by launching a single instance of the public AMI, installing
additional software on the instance, and then using StarCluster's
"createimage" command to completely automate the process of creating a
new AMI from an EC2 instance.