Practice Problems for Work Queue
-
Problem 1: Write a Work Queue program to compute the maxima of a simulator.
-
Objectives
-
Problem Statement
-
Hints
-
Problem 2: Run Work Queue on multiple platforms
-
Objectives
-
Problem Statement
-
Problem 3: Apply Work Queue in your research
-
Problem Statement
Use the information in the
Work Queue page to learn more about Work Queue, if necessary.
Problem 1
Write a Work Queue program to compute the maxima of a simulator.
Objectives
This problem will illustrate the following:
- Work Queue's support for the construction of large dynamic workflows.
- The execution model of Work Queue.
Problem Statement
This problem involves a simulator that takes an integer as input (x),
applies this input in a function (y=f(x)), and returns the evaluated
value (y) as output.
The simulator for this problem is a Python executable. It can be downloaded using:
> wget http://www.nd.edu/~ccl/software/tutorials/ndtut13/wq/simulator.py
It can be executed by specifying an integer as a command-line input argument.
For example, to run the simulator with an input of 10 and write the output
value to a file named 10.output:
> python simulator.py 10 > 10.output
Write a Work Queue program that determines the maxima
of this simulator. You will achieve this without any knowledge of the
function (f(x)) implemented in the simulator. That is, you will
only use the output values returned by the simulator for the given inputs to
determine its maxima. Or in other words, you will treat the simulator as a
black box.
Hints
- Determine the search space (values of x) that you want to
explore (e.g., from -300 to 300). Also decide on the granularity at which you want to explore the
search space (e.g., in increments of 20).
- Select values from the search space with the chosen granularity to
feed as inputs to the simulator. Create a Work Queue task to run the simulator
with a given input value.
- You can apply techniques such as binary search to narrow down the
search space and lower your search granularity as you gather
outputs for the input values you fed to the simulator.
- When you have narrowed down your search space to a single value, you
have found the maxima. Note that this maxima is valid only for the
chosen search space. Therefore, to determine the global maxima, make sure
to chose your search space and search granularity carefully.
For the given simulator, the value of the (global) maxima can be found
here.
Problem 2
Configure Work Queue to run on multiple platforms
Objectives
This problem will familiarize participants with the following:
- Setup Work Queue programs to harness resources from multiple platforms.
Problem Statement
For this assigment, you will need to download the following files used in
the tutorial:
- Work Queue
master program
- simulation.py
(used by the master program)
In this assignment, you will use the catalog server and project name feature
to have the workers automatically find and establish connection with the
master without being provided with the master's hostname and port.
To do this, the master needs to be provided with a project name that
will be advertised to the catalog server. You will modify the
Work Queue master program
to specify a project name to the catalog server through
the
specify_name() API.
An example of their usage is given below:
try:
Q = WorkQueue(port = 0)
except:
sys.exit(1)
Q.specify_name("dinesh-wq")
You will then start workers for your Work Queue master by
specifying the option to use the catalog server (-a option) and the
project name of your master (-N option). Example:
work_queue_worker -d all -a -N dinesh-wq
To successfully run Work Queue workers on multiple platforms, you may need to
build and install CCTools on those platforms.
Follow these instructions to download and install CCTools.
Problem 3
Apply Work Queue in your research.
Problem Statement
In completing Problem 1 and Problem
2, you have mastered the fundamentals of transforming and running large
dynamic worfklows as Work Queue programs!
Now, can you think of one or two instances in your research where you can
apply Work Queue to run computation/simulation workflows?
|