Since July 1st, 2014,
GC3 merged into
S3IT.
This web site is only kept for historical
reasons and may be out-of-date.
Visit the S3IT website
for more up-to-date information.
As of July 1st, 2014,
GC3 merged into S3IT,
the new UZH unit for
Services and Support to Science IT.
(Read the full story here.)
This page remains for historical and archival purposes,
but updates and new material will be posted on
www.s3it.uzh.ch.
This is a list of software that is (mainly) developed at GC3. The list is sorted alphabetically.
AppPot is a way to implement generic application deployment on a computational Grid, and especially to enable users to provide their own software to the computing cluster.
AppPot uses User Mode Linux to provide users with a development and running environment they have full control over. User Mode Linux provides a virtualized environment in which users can build and run their own computational applications. AppPot augments this with scripts that can take a copy of the User-Mode Linux system image and submit it as a regular computational job using the ARC middleware.
If you're interested in AppPot, find out more at: http://apppot.googlecode.com/
elasticluster is a command line tool to create, manage and setup computing clusters hosted on cloud infrastructures (like Amazon's Elastic Compute Cloud EC2, Google Compute Engine or a private OpenStack cloud).
Its main goal is to get your own private computational cluster up and running with just a few commands; the video demoes the basic features of elasticluster.
The following configurations are available, tested, and supported:
SLURM/Ubuntu: Run a SLURM batch-queueing system on top of the Ubuntu operating system.
GridEngine: Run a GridEngine 6.2u5 batch-queueing system.
Hadoop: Run an Hadoop cluster to start your favorite map/reduce job.
IPython cluster: Run an IPython cluster to execute your python code in parallel over multiple virtual machines.
GlusterFS/Ceph/OrangeFS: Configure a distributed storage like GlusterFS or Ceph, or a parallel filesystem like OrangeFS (formerly known as PVFS2). You can also combine these storage with one of the previous cluster types!
TORQUE+MAUI: Run the TORQUE batch-queueing system together with the MAUI scheduler on top of a CentOS Linux distribution. (But note: due to license limitations of TORQUE and MAUI, this configuration relies on a private package repository. We can provide the source RPM so you can rebuild the repository, but we cannot distribute the binaries publicly.)
elasticluster can deploy cluster software on vanilla Ubuntu or CentOS virtual machines: the only requirement is that the VMs can be accessed via SSH and that Python >= 2.4 is installed.
Instructions on the deployment and configuration of the cluster software is encoded in Ansible playbooks, which are re-played on each newly-spawned VM. This allows running clusters on different cloud infrastructures or on top of different versions of the base OS, for instance. There is nothing special about the playbooks used by elasticluster -- you can modify them to customize your cluster, or re-use any Ansible playbook that you wrote for your local, "bare metal", HPC cluster.
If you are interested in elasticluster, please find out more at: http://gc3-uzh-ch.github.io/elasticluster/
Documentation for elasticluster is available on Read The Docs
Source code is available on GitHub
GC3Pie is a suite of Python classes (and command-line tools built upon them) to aid in submitting and controlling batch jobs to clusters and grid resources seamlessly. GC3Pie aims at providing the building blocks by which Python scripts that combine several applications in a dynamic workflow can be quickly developed.
You can find out more on GC3Pie at: http://gc3pie.googlecode.com
GridCertLib is a Java library providing services to create a SLCS/X.509 certificate and a Grid proxy (optionally VOMS-enabled), given the SAML2 assertion resulting from a Shibboleth2 authentication.
The library comes with some example servlets (cf. package ch.swing.gridcertlib.servlet) that provide sample code to use the GridCertLib features in a Java web services environment.
The main use case envisioned for GridCertLib is to provide seamless and secure access to Grid/X.509 certificates and proxies in web portals: when a user logs in to the portal using the regular SWITCHaai Shibboleth authentication, GridCertLib can automatically obtain a Grid X.509 certificate from the SLCS service and generate a VOMS proxy from it. What is more, all of this can happen without further interaction with the user.
If you're interested in GridCertLib, find out more at: http://gridcertlib.googlecode.com/
VM-MAD is a tool for dynamic expansion of compute clusters, creating new compute nodes in a cloud (any cloud provider supported by Apache LibCloud works) or from any execution backend available in GC3Pie.
Policies to decide when and how to start a new node and how long to keep it running are implemented in the Python language; the VM-MAD software provides an abstraction layer that allows expressing these policies in a way that is independent from the cluster batch system and actual node provider.
If you're interested in VM-MAD, find out more at: http://vm-mad.googlecode.com/