A3: Chirp Performance Study

<title>A3: Chirp Performance Study</title>
<h1>A3: Chirp Performance Study</h1>

In this assignment, you will set up and operate a personal Chirp server, and learn how to access it using all available methods.  You will carry out a performance study of using the server to manipulate a large number of files, and explain the nature of the results.
<p>
<h2>Objectives</h2>
At the conclusion of this assignment, students will be able to:
<dir>
<li> Set up and operate a Chirp server.
<li> Combine Condor, Parrot, and Chirp into a single working system.
<li> Access Chirp servers through Parrot, tools, and the API.
<li> Explain the performance tradeoffs of each mechanism.
</dir>

<h2>Part 1: Set Up a Chirp Server</h2>
<dir>
<li> Familiarize yourself with the <a href=http://www.cse.nd.edu/~ccl/software/chirp/>Chirp Manual and API</a>
<li> Start your own private Chirp server on one of the student cluster machines.  (Make sure to use /tmp/YOURNAME as a storage directory, to get predictable performance.
<li> Learn how to combine Parrot, Chirp, and Condor.  Modify your solution to assignment 1 to run the ray tracer with Parrot and have it write the output files directly to your Chirp server.  Call this program condor_parrot_chirp_povray
<li> Generate 1000 frames of output at 1024x768, to use in the next section.
</dir>

<h2>Part 2: Performance Study</H2>
Measure the time to checksum all 1000 files as fast as possible, using the following methods:
<dir>
<li> Write a script that uses Parrot and /usr/bin/md5sum.
<li> Write a C program that uses chirp_reli_getfile and md5_file to download and checksum each file on the local disk.
<li> Write a C program that uses chirp_reli_md5 to checksum each file in place.
</dir>

Measure the time to convert all 1000 .png files into .jpg files with /usr/bin/convert:
<dir>
<li> Write a script that users Parrot and /usr/bin/convert
<li> Write a C program that uses chirp_reli_getfile, calls /usr/bin/convert, then chirp_reli_putfile
<li> Write a C program that uses the chirp_job active storage functions 
</dir>

<h2>Part 3: Lab Report</h2>
Write a short lab report that summarizes your methods, results, and conclusions.  Be sure to carefully explain why the results are the way they are.  You may find it illuminating to turn on debugging flags by passing -d chirp to Parrot, or by called debug_flags_set("chirp") in your custom programs.

<h2>What to Turn In</h2>

Your dropbox directory is:
<pre>
/afs/nd.edu/courses/cse/cse40771.01/dropbox/YOURNAME/a3
</pre>

Turn in the following:

<dir>
<li> The source code for all of your code and scripts.
<li> A Makefile that builds everything.
<li> A short lab report (.txt or .doc) that outlines your methods, results, and conclusions.
</dir>

This assignment is due on 18 March at the beginning of class.