The work queue provides an implementation of the master-worker computing model using TCP sockets, Unix applications, and files as intermediate buffers. A master process uses work_queue_create to create a queue, then work_queue_submit to submit tasks. Once tasks are running, call work_queue_wait to wait for completion. The generic worker
program can be run on any machine, and simply needs to be told the host and port of the master.
#include "timestamp.h"
Go to the source code of this file.
Data Structures | |
struct | work_queue_task |
A task description. More... | |
struct | work_queue_stats |
Statistics describing a work queue. More... | |
Functions | |
work_queue * | work_queue_create (int port, time_t stoptime) |
Create a new work queue. | |
void | work_queue_delete (struct work_queue *q) |
Delete a work queue. | |
void | work_queue_get_stats (struct work_queue *q, struct work_queue_stats *s) |
Get queue statistics. | |
int | work_queue_activate_fast_abort (struct work_queue *q, double multiplier) |
Turn on or off fast abort functionality for a given queue. | |
work_queue_task * | work_queue_wait (struct work_queue *q, int timeout) |
Wait for tasks to complete. | |
void | work_queue_submit (struct work_queue *q, struct work_queue_task *t) |
Submit a job to a work queue. | |
void | work_queue_task_delete (struct work_queue_task *t) |
Delete a task specification. | |
work_queue_task * | work_queue_task_create (const char *full_command) |
Create a new task specification. | |
INT64_T | work_queue_task_specify_tag (struct work_queue_task *t, const char *tag) |
Further define a task specification. | |
INT64_T | work_queue_task_specify_algorithm (struct work_queue_task *t, int alg) |
Further define a task specification. | |
INT64_T | work_queue_task_specify_input_buf (struct work_queue_task *t, const char *buf, int length, const char *rname) |
Further define a task specification. | |
INT64_T | work_queue_task_specify_input_file (struct work_queue_task *t, const char *fname, const char *rname) |
Further define a task specification. | |
INT64_T | work_queue_task_specify_output_file (struct work_queue_task *t, const char *rname, const char *fname) |
Further define a task specification. | |
int | work_queue_hungry (struct work_queue *q) |
Determine whether the queue can support more tasks. | |
int | work_queue_empty (struct work_queue *q) |
Determine whether there are any known tasks queued, running, or waiting to be collected. | |
int | work_queue_shut_down_workers (struct work_queue *q, int n) |
Shut down workers connected to the work_queue system. | |
Variables | |
double | wq_option_fast_abort_multiplier |
Initial setting for fast abort multiplier upon creating queue. | |
int | wq_option_worker_selection_algorithm |
Initial setting for algorithm to assign tasks to workers upon creating queue. |
|
Create a new work queue.
|
|
Delete a work queue.
|
|
Get queue statistics.
|
|
Turn on or off fast abort functionality for a given queue.
|
|
Wait for tasks to complete. This call will block until the timeout has elapsed.
|
|
Submit a job to a work queue.
|
|
Delete a task specification. This may be called on tasks after they are returned from work_queue_wait.
|
|
Create a new task specification. Once created, the task may be passed to work_queue_submit.
|
|
Further define a task specification. Once completed, the task may be passed to work_queue_submit.
|
|
Further define a task specification. Once completed, the task may be passed to work_queue_submit.
|
|
Further define a task specification. Once completed, the task may be passed to work_queue_submit.
|
|
Further define a task specification. Once completed, the task may be passed to work_queue_submit.
|
|
Further define a task specification. Once completed, the task may be passed to work_queue_submit. If no file is defined, the program will have default (no) output files retrieved.
|
|
Determine whether the queue can support more tasks. Returns the number of additional tasks it can support if "hungry" and 0 if "sated".
|
|
Determine whether there are any known tasks queued, running, or waiting to be collected. Returns 1 if so and 0 if "empty".
|
|
Shut down workers connected to the work_queue system. Gives a best effort and then returns the number of workers given the shut down order.
|
|
Initial setting for fast abort multiplier upon creating queue. Turned off if less than 0. Change prior to calling work_queue_create, after queue is created this variable is not considered and changes must be made through the API calls. |
|
Initial setting for algorithm to assign tasks to workers upon creating queue. Change prior to calling work_queue_create, after queue is created this variable is not considered and changes must be made through the API calls. |