CCL Home

Research

Software Community Operations

Building Scalable Scientific Applications using Makeflow and Work Queue

Tutorial at XSEDE13, July 22 2013, San Diego.

Introduction

This tutorial will provide an introduction to writing scalable scientific applications that are robust, elastic, and portable across different distributed systems. We will use Makeflow and Work Queue, developed by the Cooperative Computing Lab, to build such applications. These tools are used at Notre Dame and around the world to attack large problems in fields such as chemistry, data mining, economics, biology, physics, and more.

Target Audience

This tutorial is aimed at scientists, researchers, developers, graduate students, and other participants interested in:

  • writing scientific applications that easily scale to the size of the allocated resources
  • writing applications that are robust, fault-tolerant, and portable
  • Prerequisites

  • Familiarity with Unix.
  • Basic knowledge of Python, Perl, or C (for the Work Queue portion of tutorial).
  • Tutorial Materials

    The tutorial will consist of a lecture and a hands-on instruction session. The lecture will give an overview of the Makeflow and Work Queue software. The hands-on instruction will demonstrate and walk through the installation and use of Work Queue and Makeflow to write scalable programs. A set of practice problems will also be provided to try on completion of the tutorial.

      Makeflow

  • Lecture slides
  • Tutorial
  • Practice problems
  •   Work Queue

  • Lecture slides
  • Tutorial
  • Practice problems
  •   Demo

  • Slides
  • Audience guide
  • Software Resources

  • Software Download
  • Makeflow Web Page
  • Work Queue Web Page
  • Getting Help
  • Additional Reading

    (Showing papers with tag tutorial. See all papers instead.)