Installing the Cooperative Computing Tools

Last Modified October 2011

The Cooperative Computing Tools are Copyright (C) 2003-2004 Douglas Thain and Copyright (C) 2005-2011 The University of Notre Dame. This software is distributed under the GNU General Public License. See the file COPYING for details.

Overview

The Cooperative Computing Tools (cctools) are a collection of programs designed to assist users with the difficulties of building and managing complex, fault-prone distributed systems. The components of the cctools are:
  • Parrot - A personal virtual file system.
  • Chirp - A distributed file and storage system.
  • Makeflow - A workflow engine similar to Make.
  • Work Queue - A flexible master-worker library.
  • SAND - A scalable genome assembly framework.
  • All-Pairs - A computational abstraction.
  • Wavefront - A computational abstraction.
    • This short manual describes how to install the cctools. For more information about the tools themselves, see the links above.

    Installing

    The cctools may already be installed on your system. Simply run chirp -v. If you see something like 
    chirp version 1.0.1 built by dthain@hedwig on Sep 12 2004 at 20:04:51
    then the cctools are installed, and you are ready to go. Otherwise, you must install the cctools yourself.

    Installation From a Binary Package (UNIX)

    From any other location, we recommend that you install a binary package from the cctools web page Simply unpack the tarball in any directory that you like, and then add the bin directory to your path. For example, to install in $HOME/cctools: 
    % cd $HOME
% gunzip cctools-xxx-yyy.tar.gz
% tar xvf cctools-xxx-yyy.tar
% setenv PATH $HOME/cctools/bin:$PATH

    Installation From a Binary Package (DOS)

    From any other location, we recommend that you install the binary package for the Windows operating system from the cctools web page Simply unzip the zipfile in any directory that you like, and then add the bin directory to your path. For example, to install in %UserProfile%\cctools using the DOS shell: 
    % cd %UserProfile%
% winzip32 -e cctools-xxx-yyy.zip 
% set PATH=%UserProfile%\cctools\bin;%PATH%

    Installation From a Source Package

    Building the CCTools from source is very easy. Just download a source package:
  • CCTools Download(Required)
    • And follow this recipe while logged in as any ordinary user: 
    % gunzip cctools-xxx-src.tar.gz
% tar xvf cctools-xxx-src.tar
% cd cctools-xxx-src
% ./configure
% make
% make install
% setenv PATH ${HOME}/cctools/bin:$PATH
    The software will happily build and run without installing any external packages. Optionally, the cctools will interoperate with a variety of external packages for security and data access. To use these, you must download and install them separately:
  • FUSE (Optional)
  • Hadoop (Optional)
  • xRootD (Optional)
  • iRODS (Optional)
  • Kerberos (Optional)
  • Globus (Optional) (Tested with Globus 2.2, 2.4, and 4.0.)
    • Once the desired packages are correctly installed, unpack the cctools and then issue a configure command that points to all of the other installations. Then, make and install. For example: 
    % gunzip cctools-xxx.tar.gz
% tar xvf cctools-xxx.tar
% cd cctools-xxx
% ./configure --with-krb5-path /usr/kerberos --with-globus-path /usr/local/globus ...
% make
% make install
% setenv PATH ${HOME}/cctools/bin:$PATH
    Cooperative Computing Tools Documentation Index