The work queue provides an implementation of the master-worker computing model using TCP sockets, Unix applications, and files as intermediate buffers. A master process uses work_queue_create to create a queue, then work_queue_submit to submit tasks. Once tasks are running, call work_queue_wait to wait for completion. The generic worker
program can be run on any machine, and simply needs to be told the host and port of the master.
#include "timestamp.h"
Go to the source code of this file.
Data Structures | |
struct | work_queue_task |
A task description. More... | |
struct | work_queue_stats |
Statistics describing a work queue. More... | |
Functions | |
work_queue * | work_queue_create (int port, time_t stoptime) |
Create a new work queue. | |
void | work_queue_delete (struct work_queue *q) |
Delete a work queue. | |
void | work_queue_get_stats (struct work_queue *q, struct work_queue_stats *s) |
Get queue statistics. | |
int | work_queue_activate_fast_abort (struct work_queue *q, double multiplier) |
Turn on or off fast abort functionality for a given queue. | |
int | work_queue_specify_scheduler (struct work_queue *q, int alg) |
Change the worker selection algorithm for a given queue. | |
work_queue_task * | work_queue_wait (struct work_queue *q, int timeout) |
Wait for tasks to complete. | |
void | work_queue_submit (struct work_queue *q, struct work_queue_task *t) |
Submit a job to a work queue. | |
void | work_queue_task_delete (struct work_queue_task *t) |
Delete a task specification. | |
work_queue_task * | work_queue_task_create (const char *full_command) |
Create a new task specification. | |
void | work_queue_task_specify_tag (struct work_queue_task *t, const char *tag) |
Attach a user defined logical name to the task. | |
int | work_queue_task_specify_scheduler (struct work_queue_task *t, int alg) |
Further define a task specification. | |
void | work_queue_task_specify_input_buf (struct work_queue_task *t, const char *buf, int length, const char *rname) |
Further define a task specification. | |
void | work_queue_task_specify_input_file (struct work_queue_task *t, const char *fname, const char *rname) |
Further define a task specification. | |
void | work_queue_task_specify_output_file (struct work_queue_task *t, const char *rname, const char *fname) |
Further define a task specification. | |
int | work_queue_hungry (struct work_queue *q) |
Determine whether the queue can support more tasks. | |
int | work_queue_empty (struct work_queue *q) |
Determine whether there are any known tasks queued, running, or waiting to be collected. | |
int | work_queue_shut_down_workers (struct work_queue *q, int n) |
Shut down workers connected to the work_queue system. | |
Variables | |
double | wq_option_fast_abort_multiplier |
Initial setting for fast abort multiplier upon creating queue. | |
int | wq_option_scheduler |
Initial setting for algorithm to assign tasks to workers upon creating queue. |
|
Create a new work queue.
|
|
Delete a work queue.
|
|
Get queue statistics.
|
|
Turn on or off fast abort functionality for a given queue.
|
|
Change the worker selection algorithm for a given queue.
|
|
Wait for tasks to complete. This call will block until the timeout has elapsed.
|
|
Submit a job to a work queue. It is safe to re-submit a task returned by work_queue_wait.
|
|
Delete a task specification. This may be called on tasks after they are returned from work_queue_wait.
|
|
Create a new task specification. Once created, the task may be passed to work_queue_submit.
|
|
Attach a user defined logical name to the task. This field is not interpreted by the work queue, but simply maintained to help the user track tasks.
|
|
Further define a task specification. Once completed, the task may be passed to work_queue_submit.
|
|
Further define a task specification. Once completed, the task may be passed to work_queue_submit.
|
|
Further define a task specification. Once completed, the task may be passed to work_queue_submit.
|
|
Further define a task specification. Once completed, the task may be passed to work_queue_submit. If no file is defined, the program will have default (no) output files retrieved.
|
|
Determine whether the queue can support more tasks. Returns the number of additional tasks it can support if "hungry" and 0 if "sated".
|
|
Determine whether there are any known tasks queued, running, or waiting to be collected. Returns 0 if there are tasks remaining in the system, 1 if the system is "empty".
|
|
Shut down workers connected to the work_queue system. Gives a best effort and then returns the number of workers given the shut down order.
|
|
Initial setting for fast abort multiplier upon creating queue. Turned off if less than 0. Change prior to calling work_queue_create, after queue is created this variable is not considered and changes must be made through the API calls. |
|
Initial setting for algorithm to assign tasks to workers upon creating queue. Change prior to calling work_queue_create, after queue is created this variable is not considered and changes must be made through the API calls. |