Tutorial: Introduction to scalable programming using Makeflow and Work Queue

Tutorial at CCL Workshop, Room 112 Notre Dame Conference Center (McKenna Hall), October 10, 2013.

This tutorial will provide a brief introduction to writing scalable applications using Makeflow and Work Queue, developed by the Cooperative Computing Lab. The tutorial consists of a lecture that gives an overview of the software and applications, an hands-on instruction that will demonstrate and walk through the installation and use of Work Queue and Makeflow to write scalable programs, and practice problems to try after the tutorial.

Prerequisites

Tutorial Materials

  Makeflow

  Work Queue

Software Resources

Additional Reading

(Showing papers with tag tutorial. See all papers instead.)

Michael Albrecht, Dinesh Rajan, Douglas Thain, Making Work Queue Cluster-Friendly for Data Intensive Scientific Applications, IEEE International Conference on Cluster Computing, September, 2013. DOI: 10.1109/CLUSTER.2013.6702628

Dinesh Rajan, Andrew Thrasher, Badi Abdul-Wahid, Jesus A Izaguirre, Scott Emrich, and Douglas Thain, Case Studies in Designing Elastic Applications, 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), May, 2013. DOI: 0.1109/CCGrid.2013.46

Christopher Moretti, Andrew Thrasher, Li Yu, Michael Olson, Scott Emrich, and Douglas Thain, A Framework for Scalable Genome Assembly on Clusters, Clouds, and Grids, IEEE Transactions on Parallel and Distributed Systems, 23(12), December, 2012. DOI: 10.1109/TPDS.2012.80

Badi Abdul-Wahid, Li Yu, Dinesh Rajan, Haoyun Feng, Eric Darve, Douglas Thain, Jesus A. Izaguirre, Folding Proteins at 500 ns/hour with Work Queue, 8th IEEE International Conference on eScience (eScience 2012), October, 2012. DOI: 10.1109/eScience.2012.6404429

Michael Albrecht, Patrick Donnelly, Peter Bui, and Douglas Thain, Makeflow: A Portable Abstraction for Data Intensive Computing on Clusters, Clouds, and Grids, Workshop on Scalable Workflow Enactment Engines and Technologies (SWEET) at ACM SIGMOD, May, 2012. DOI: 10.1145/2443416.2443417

Dinesh Rajan, Anthony Canino, Jesus A Izaguirre, and Douglas Thain, Converting a High Performance Application to an Elastic Cloud Application, The 3rd IEEE International Conference on Cloud Computing Technology and Science (CloudCom 2011), November, 2011.

Peter Bui, Dinesh Rajan, Badi Abdul-Wahid, Jesus Izaguirre, Douglas Thain, Work Queue + Python: A Framework For Scalable Scientific Ensemble Applications, Workshop on Python for High Performance and Scientific Computing (PyHPC) at the ACM/IEEE International Conference for High Performance Computing, Networking, Storage, and Analysis (Supercomputing) , November, 2011.

Andrew Thrasher, Rory Carmichael, Peter Bui, Li Yu, Douglas Thain, and Scott Emrich, Taming Complex Bioinformatics Workflows with Weaver, Makeflow, and Starch, Workshop on Workflows in Support of Large Scale Science, pages 1-6, November, 2010. DOI: 10.1109/WORKS.2010.5671858