This class uses a dictionary to map between the task pointer objects and the work_queue::Task.
Public Member Functions | |
def | __init__ |
Create a new work queue. | |
def | name |
Get the project name of the queue. | |
def | port |
Get the listening port of the queue. | |
def | stats |
Get queue statistics. | |
def | stats_hierarchy |
Get worker hierarchy statistics. | |
def | enable_monitoring |
Enables resource monitoring of tasks in the queue. | |
def | activate_fast_abort |
Turn on or off fast abort functionality for a given queue. | |
def | empty |
Determine whether there are any known tasks queued, running, or waiting to be collected. | |
def | hungry |
Determine whether the queue can support more tasks. | |
def | specify_algorithm |
Set the worker selection algorithm for queue. | |
def | specify_task_order |
Set the order for dispatching submitted tasks in the queue. | |
def | specify_name |
Change the project name for the given queue. | |
def | specify_priority |
Change the project priority for the given queue. | |
def | specify_master_mode |
Specify the master mode for the given queue. | |
def | specify_catalog_server |
Specify the catalog server the master should report to. | |
def | specify_log |
Specify a log file that records the states of connected workers and submitted tasks. | |
def | specify_password |
Add a mandatory password that each worker must present. | |
def | specify_password_file |
Add a mandatory password file that each worker must present. | |
def | cancel_by_taskid |
Cancel task identified by its taskid and remove from the given queue. | |
def | cancel_by_tasktag |
Cancel task identified by its tag and remove from the given queue. | |
def | shutdown_workers |
Shutdown workers connected to queue. | |
def | specify_keepalive_interval |
Change keepalive interval for a given queue. | |
def | specify_keepalive_timeout |
Change keepalive timeout for a given queue. | |
def | estimate_capacity |
Turn on master capacity measurements. | |
def | reset |
Reset a work queue and all attached workers. | |
def | tune |
Tune advanced parameters for work queue. | |
def | submit |
Submit a task to the queue. | |
def | wait |
Wait for tasks to complete. |
def work_queue::WorkQueue::__init__ | ( | self, | ||
port = WORK_QUEUE_DEFAULT_PORT , |
||||
name = None , |
||||
catalog = False , |
||||
exclusive = True , |
||||
shutdown = False | ||||
) |
Create a new work queue.
self | Reference to the current work queue object. | |
port | The port number to listen on. If zero is specified, then the default is chosen, and if -1 is specified, a random port is chosen. | |
name | The project name to use. | |
catalog | Whether or not to enable catalog mode. | |
exclusive | Whether or not the workers should be exclusive. | |
shutdown | Automatically shutdown workers when queue is finished. Disabled by default. |
def work_queue::WorkQueue::name | ( | self | ) |
Get the project name of the queue.
Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print q.name
def work_queue::WorkQueue::port | ( | self | ) |
Get the listening port of the queue.
Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print q.port
def work_queue::WorkQueue::stats | ( | self | ) |
Get queue statistics.
Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print q.stats
>>> print q.stats.workers_busy
def work_queue::WorkQueue::stats_hierarchy | ( | self | ) |
Get worker hierarchy statistics.
Note: This is defined using property decorator. So it must be called without parentheses (). For example:
>>> print q.stats
>>> print q.stats.workers_busy
def work_queue::WorkQueue::enable_monitoring | ( | self, | ||
summaryfile | ||||
) |
Enables resource monitoring of tasks in the queue.
And writes a summary of the monitored information to a file.
Returns 1 on success, 0 on failure (i.e., monitoring was not enabled).
self | Reference to the current work queue object. | |
summaryfile | Filename for the summary log (If NULL, writes to wq-<pid>-resource-usage). |
def work_queue::WorkQueue::activate_fast_abort | ( | self, | ||
multiplier | ||||
) |
Turn on or off fast abort functionality for a given queue.
self | Reference to the current work queue object. | |
multiplier | The multiplier of the average task time at which point to abort; if negative (the default) fast_abort is deactivated. |
def work_queue::WorkQueue::empty | ( | self | ) |
Determine whether there are any known tasks queued, running, or waiting to be collected.
Returns 0 if there are tasks remaining in the system, 1 if the system is "empty".
self | Reference to the current work queue object. |
def work_queue::WorkQueue::hungry | ( | self | ) |
Determine whether the queue can support more tasks.
Returns the number of additional tasks it can support if "hungry" and 0 if "sated".
self | Reference to the current work queue object. |
def work_queue::WorkQueue::specify_algorithm | ( | self, | ||
algorithm | ||||
) |
Set the worker selection algorithm for queue.
self | Reference to the current work queue object. | |
algorithm | One of the following algorithms to use in assigning a task to a worker: |
def work_queue::WorkQueue::specify_task_order | ( | self, | ||
order | ||||
) |
Set the order for dispatching submitted tasks in the queue.
self | Reference to the current work queue object. | |
order | One of the following algorithms to use in dispatching submitted tasks to workers: |
def work_queue::WorkQueue::specify_name | ( | self, | ||
name | ||||
) |
Change the project name for the given queue.
self | Reference to the current work queue object. | |
name | The new project name. |
def work_queue::WorkQueue::specify_priority | ( | self, | ||
priority | ||||
) |
Change the project priority for the given queue.
self | Reference to the current work queue object. | |
priority | An integer that presents the priorty of this work queue master. The higher the value, the higher the priority. |
def work_queue::WorkQueue::specify_master_mode | ( | self, | ||
mode | ||||
) |
Specify the master mode for the given queue.
self | Reference to the current work queue object. | |
mode | This may be one of the following values: WORK_QUEUE_MASTER_MODE_STANDALONE or WORK_QUEUE_MASTER_MODE_CATALOG. |
def work_queue::WorkQueue::specify_catalog_server | ( | self, | ||
hostname, | ||||
port | ||||
) |
Specify the catalog server the master should report to.
self | Reference to the current work queue object. | |
hostname | The hostname of the catalog server. | |
port | The port the catalog server is listening on. |
def work_queue::WorkQueue::specify_log | ( | self, | ||
logfile | ||||
) |
Specify a log file that records the states of connected workers and submitted tasks.
self | Reference to the current work queue object. | |
logfile | Filename. |
def work_queue::WorkQueue::specify_password | ( | self, | ||
password | ||||
) |
Add a mandatory password that each worker must present.
self | Reference to the current work queue object. | |
password | The password. |
def work_queue::WorkQueue::specify_password_file | ( | self, | ||
file | ||||
) |
Add a mandatory password file that each worker must present.
self | Reference to the current work queue object. | |
file | Name of the file containing the password. |
def work_queue::WorkQueue::cancel_by_taskid | ( | self, | ||
id | ||||
) |
Cancel task identified by its taskid and remove from the given queue.
self | Reference to the current work queue object. | |
id | The taskid returned from submit. |
def work_queue::WorkQueue::cancel_by_tasktag | ( | self, | ||
tag | ||||
) |
Cancel task identified by its tag and remove from the given queue.
self | Reference to the current work queue object. | |
tag | The tag assigned to task using work_queue_task_specify_tag. |
def work_queue::WorkQueue::shutdown_workers | ( | self, | ||
n | ||||
) |
Shutdown workers connected to queue.
Gives a best effort and then returns the number of workers given the shutdown order.
self | Reference to the current work queue object. | |
n | The number to shutdown. To shut down all workers, specify "0". |
def work_queue::WorkQueue::specify_keepalive_interval | ( | self, | ||
interval | ||||
) |
Change keepalive interval for a given queue.
self | Reference to the current work queue object. | |
interval | Minimum number of seconds to wait before sending new keepalive checks to workers. |
def work_queue::WorkQueue::specify_keepalive_timeout | ( | self, | ||
timeout | ||||
) |
Change keepalive timeout for a given queue.
self | Reference to the current work queue object. | |
timeout | Minimum number of seconds to wait for a keepalive response from worker before marking it as dead. |
def work_queue::WorkQueue::estimate_capacity | ( | self | ) |
Turn on master capacity measurements.
self | Reference to the current work queue object. |
def work_queue::WorkQueue::reset | ( | self, | ||
flags | ||||
) |
Reset a work queue and all attached workers.
self | Reference to the current work queue object. | |
flags | Flags to indicate what to reset: |
def work_queue::WorkQueue::tune | ( | self, | ||
name, | ||||
value | ||||
) |
Tune advanced parameters for work queue.
self | Reference to the current work queue object. | |
name | The name fo the parameter to tune. Can be one of following:
| |
value | The value to set the parameter to. |
def work_queue::WorkQueue::submit | ( | self, | ||
task | ||||
) |
Submit a task to the queue.
It is safe to re-submit a task returned by wait.
self | Reference to the current work queue object. | |
task | A task description created from work_queue::Task. |
def work_queue::WorkQueue::wait | ( | self, | ||
timeout = WORK_QUEUE_WAITFORTASK | ||||
) |
Wait for tasks to complete.
This call will block until the timeout has elapsed
self | Reference to the current work queue object. | |
timeout | The number of seconds to wait for a completed task before returning. Use an integer to set the timeout or the constant WORK_QUEUE_WAITFORTASK to block until a task has completed. |