CCL Home

Research

Software Community Operations

Building Scalable Elastic Applications using Makeflow and Work Queue

Tutorial at CCGrid 2013, May 13 2013, Delft University of Technology, The Netherlands. By Dinesh Rajan and Prof. Douglas Thain.

Introduction

This tutorial will provide an introduction to writing scalable applications that are robust, elastic, and portable across different distributed systems. We will use Makeflow and Work Queue, developed by the Cooperative Computing Lab, to build such applications. These tools are used at Notre Dame and around the world to attack large problems in fields such as chemistry, data mining, economics, biology, physics, and more.

Target Audience

This tutorial is aimed at scientists, researchers, developers, graduate students, and other participants interested in:

  • writing applications that transparently adapt and scale to the size of the allocated resources
  • writing applications that are robust, fault-tolerant, and portable.
  • Prerequisites

  • Familiarity with Unix
  • Ability to program in Python, Perl, or C (for the Work Queue portion of tutorial).
  • Tutorial Materials

    The tutorial will consist of a lecture and a hands-on instruction session. The lecture will give an overview of the Makeflow and Work Queue software. The hands-on instruction will demonstrate and walk through the installation and use of Work Queue and Makeflow to write scalable programs. A set of practice problems will also be provided to try on completion of the tutorial.

      Makeflow

  • Lecture slides
  • Tutorial
  • Practice problems
  •   Work Queue

  • Lecture slides
  • Tutorial
  • Practice problems
  • Software Resources

  • Software Download
  • Makeflow Web Page
  • Work Queue Web Page
  • Getting Help
  • Additional Reading

    (Showing papers with tag tutorial. See all papers instead.)