CCTools 3.5.0 Released

The software may be downloaded at http://www.nd.edu/~ccl/software/download.

This is a minor release which adds numerous features and fixes several bugs:

• Batch Job Hadoop module for submitting Hadoop jobs supports Hadoop 0.21.0. [Michael Albrecht, Patrick Donnelly]
• Improvements to WorkQueue for worker accounting and management. WorkQueue now does master capacity estimation and automatic worker removal. [Li Yu]
• WorkQueue now supports cancelling of a submitted task. [Dinesh Rajan]
• WorkQueue workers can now report task execution time. [Li Yu]
• Improved Batch Job local execution to fork and exec instead of fork and system.
• Swig detection improved. [Peter Bui]
• Improved and made consistent time formats for catalog server. [Li Yu]
• Various corrections to Parrot's directory handling. [Douglas Thain]
• Corrected numerous memory leaks and software bugs using Valgrind/ccpcheck. [Peter Bui]
• WorkQueue workers now check for low disk space. [Dinesh Rajan]
• Parrot now supports writable memory mapped files. [Douglas Thain]
• WorkQueue MOAB support improved. [Peter Bui, Michael Albrecht]
• WorkQueue now has prototype support for work_queue_pool resource management of multiple masters. work_queue_pool is now capable of automatically requesting resources from the underlying batch system as needed by the masters subject to a constraint file. [Li Yu]
• WorkQueue now supports FIFO and LIFO task dispatch to workers. [Dinesh Rajan]
• WorkQueue now has work_queue_version to differentiate versions of the library. [Peter Bui]
• Chirp client status output is now properly sent to stderr. [Patrick Donnelly]
• WorkQueue taskid assignment moved to submit from create. Submit now returns this unique id. [Dinesh Rajan]
• Makeflow/WorkQueue/Chirp now support selecting an arbitrary port in a range using environment variables TCP_LOW_PORT and TCP_HIGH_PORT. [Patrick Donnelly]
• Improved debug output for non-blocking tcp connections. [Li Yu]
• WorkQueue task status is now appropriately set to complete when tasks are moved to complete list. [Dinesh Rajan]
• Parrot now supports iRODS version 3.1. [Douglas Thain]
• Parrot now allows an identity-boxed process to write to a world-writable file. (such as /dev/null) [Douglas Thain]
• WorkQueue workers now have a tunable exponential backoff for reconnecting to masters. [Dinesh Rajan]
• Updated WorkQueue documentation and examples. [Dinesh Rajan]
• Various improvements to WorkQueue Python binding. [Peter Bui, Dinesh Rajan]
• Numerous API/code improvements made to WorkQueue. [Li Yu, Dinesh Rajan, Douglas Thain]
• Various compatibility improvements for building CCTools. [Douglas Thain, Patrick Donnelly]

Enjoy!

Monday, June 11, 2012 - Permalink