chirp_reli.h is designed to look similar to the Unix I/O interface. It is called "reli" because it is "reliable". Each function call here has the capabaility to detect and retry a large number of network and server errors with an exponential backoff, until a user-defined time limit is reached. The caller need not worry about connecting to or disconnecting from servers.
All functions in this file have several common calling conventions.
host | A hostname may be a domain name or an IP address, followed by an optional colon and port number. If not given, the port number is assumed to by the default Chirp port of 9094. | |
path | A pathname identifies a file from the root of the given file server, and must start with a slash. The Chirp protocol allows pathnames to contain any printable ASCII character except a newline. | |
stoptime | All functions accept a final argument time_t stoptime which indicates the absolute time at which to abort. For example, to try an operation for 60 seconds, pass time(0)+60 as stoptime . Any transient network failures will be silently retried until this timeout is reached. |
errno
may have any arbitrary value. On failure, all return an integer less than zero, and set errno to the reason for the failure. (chirp_reli_open is the only exception to this rule.) The caller may invoke strerror(errno)
to generate a human-readable string representing the error.
#include "chirp_types.h"
#include <sys/types.h>
#include <stdio.h>
Go to the source code of this file.
Functions | |
chirp_file * | chirp_reli_open (const char *host, const char *path, INT64_T flags, INT64_T mode, time_t stoptime) |
Creates or opens a file in preparation for I/O. | |
INT64_T | chirp_reli_close (struct chirp_file *file, time_t stoptime) |
Closes an open file. | |
INT64_T | chirp_reli_pread (struct chirp_file *file, void *buffer, INT64_T length, INT64_T offset, time_t stoptime) |
Read data from a file. | |
INT64_T | chirp_reli_pwrite (struct chirp_file *file, const void *buffer, INT64_T length, INT64_T offset, time_t stoptime) |
Write data to a file. | |
INT64_T | chirp_reli_pread_unbuffered (struct chirp_file *file, void *buffer, INT64_T length, INT64_T offset, time_t stoptime) |
Read data from a file without buffering. | |
INT64_T | chirp_reli_pwrite_unbuffered (struct chirp_file *file, const void *buffer, INT64_T length, INT64_T offset, time_t stoptime) |
Write data to a file without buffering. | |
INT64_T | chirp_reli_sread (struct chirp_file *file, void *buffer, INT64_T length, INT64_T stride_length, INT64_T stride_skip, INT64_T offset, time_t stoptime) |
Strided read from a file. | |
INT64_T | chirp_reli_swrite (struct chirp_file *file, const void *buffer, INT64_T length, INT64_T stride_length, INT64_T stride_skip, INT64_T offset, time_t stoptime) |
Strided write to a file. | |
INT64_T | chirp_reli_fstat (struct chirp_file *file, struct chirp_stat *info, time_t stoptime) |
Get file status. | |
INT64_T | chirp_reli_fstatfs (struct chirp_file *file, struct chirp_statfs *info, time_t stoptime) |
Get file system status. | |
INT64_T | chirp_reli_fchown (struct chirp_file *file, INT64_T uid, INT64_T gid, time_t stoptime) |
Change the ownership of a file. | |
INT64_T | chirp_reli_fchmod (struct chirp_file *file, INT64_T mode, time_t stoptime) |
Change the mode bits of a file. | |
INT64_T | chirp_reli_ftruncate (struct chirp_file *file, INT64_T length, time_t stoptime) |
Truncate an open file. | |
INT64_T | chirp_reli_flush (struct chirp_file *file, time_t stoptime) |
Flush any pending changes to a file. | |
INT64_T | chirp_reli_getfile (const char *host, const char *path, FILE *stream, time_t stoptime) |
Get an entire file efficiently. | |
INT64_T | chirp_reli_getfile_buffer (const char *host, const char *path, char **buffer, time_t stoptime) |
Get an entire file efficiently to memory. | |
INT64_T | chirp_reli_putfile (const char *host, const char *path, FILE *stream, INT64_T mode, INT64_T length, time_t stoptime) |
Put an entire file efficiently. | |
INT64_T | chirp_reli_putfile_buffer (const char *host, const char *path, const char *buffer, INT64_T mode, INT64_T length, time_t stoptime) |
Put an entire file efficiently from memory. | |
INT64_T | chirp_reli_getlongdir (const char *host, const char *path, chirp_longdir_t callback, void *arg, time_t stoptime) |
Get a detailed directory listing. | |
INT64_T | chirp_reli_getdir (const char *host, const char *path, chirp_dir_t callback, void *arg, time_t stoptime) |
Get a simple directory listing. | |
chirp_dir * | chirp_reli_opendir (const char *host, const char *path, time_t stoptime) |
Open a directory for listing. | |
chirp_dirent * | chirp_reli_readdir (struct chirp_dir *dir) |
Read one item from a directory. | |
void | chirp_reli_closedir (struct chirp_dir *dir) |
Close a directory. | |
INT64_T | chirp_reli_getacl (const char *host, const char *path, chirp_dir_t callback, void *arg, time_t stoptime) |
Get an access control list. | |
INT64_T | chirp_reli_setacl (const char *host, const char *path, const char *subject, const char *rights, time_t stoptime) |
Modify an access control list. | |
INT64_T | chirp_reli_resetacl (const char *host, const char *path, const char *rights, time_t stoptime) |
Reset an access control list. | |
INT64_T | chirp_reli_whoami (const char *host, char *subject, INT64_T length, time_t stoptime) |
Return the caller's identity. | |
INT64_T | chirp_reli_whoareyou (const char *host, const char *rhost, char *subject, INT64_T length, time_t stoptime) |
Return the server's identity against another server. | |
INT64_T | chirp_reli_mkfifo (const char *host, const char *path, time_t stoptime) |
Create a named pipe (FIFO). | |
INT64_T | chirp_reli_unlink (const char *host, const char *path, time_t stoptime) |
Delete a file. | |
INT64_T | chirp_reli_rename (const char *host, const char *path, const char *newpath, time_t stoptime) |
Rename a file or directory. | |
INT64_T | chirp_reli_link (const char *host, const char *path, const char *newpath, time_t stoptime) |
Create a hard link. | |
INT64_T | chirp_reli_symlink (const char *host, const char *path, const char *newpath, time_t stoptime) |
Create a symbolic link. | |
INT64_T | chirp_reli_readlink (const char *host, const char *path, char *buf, INT64_T length, time_t stoptime) |
Examine a symbolic link. | |
INT64_T | chirp_reli_mkdir (const char *host, const char *path, INT64_T mode, time_t stoptime) |
Create a new directory. | |
INT64_T | chirp_reli_mkdir_recursive (const char *host, const char *path, INT64_T mode, time_t stoptime) |
Create a new directory recursively. | |
INT64_T | chirp_reli_rmdir (const char *host, const char *path, time_t stoptime) |
Delete a directory if it is empty. | |
INT64_T | chirp_reli_rmall (const char *host, const char *path, time_t stoptime) |
Delete a directory recursively. | |
INT64_T | chirp_reli_stat (const char *host, const char *path, struct chirp_stat *info, time_t stoptime) |
Get file status. | |
INT64_T | chirp_reli_lstat (const char *host, const char *path, struct chirp_stat *info, time_t stoptime) |
Get file or link status. | |
INT64_T | chirp_reli_statfs (const char *host, const char *path, struct chirp_statfs *info, time_t stoptime) |
Get filesystem status. | |
INT64_T | chirp_reli_access (const char *host, const char *path, INT64_T flags, time_t stoptime) |
Check access permissions. | |
INT64_T | chirp_reli_chmod (const char *host, const char *path, INT64_T mode, time_t stoptime) |
Change mode bits. | |
INT64_T | chirp_reli_chown (const char *host, const char *path, INT64_T uid, INT64_T gid, time_t stoptime) |
Change the ownership of a file. | |
INT64_T | chirp_reli_lchown (const char *host, const char *path, INT64_T uid, INT64_T gid, time_t stoptime) |
Change the ownership of a file or link. | |
INT64_T | chirp_reli_truncate (const char *host, const char *path, INT64_T length, time_t stoptime) |
Truncate a file. | |
INT64_T | chirp_reli_utime (const char *host, const char *path, time_t actime, time_t modtime, time_t stoptime) |
Change the modification times of a file. | |
INT64_T | chirp_reli_md5 (const char *host, const char *path, unsigned char digest[16], time_t stoptime) |
Checksum a remote file. | |
INT64_T | chirp_reli_localpath (const char *host, const char *path, char *localpath, int length, time_t stoptime) |
Return the local path of a file. | |
INT64_T | chirp_reli_audit (const char *host, const char *path, struct chirp_audit **list, time_t stoptime) |
Measure remote space consumption. | |
INT64_T | chirp_reli_thirdput (const char *host, const char *path, const char *thirdhost, const char *thirdpath, time_t stoptime) |
Third party transfer. | |
INT64_T | chirp_reli_mkalloc (const char *host, const char *path, INT64_T size, INT64_T mode, time_t stoptime) |
Create a space allocation. | |
INT64_T | chirp_reli_lsalloc (const char *host, const char *path, char *allocpath, INT64_T *total, INT64_T *inuse, time_t stoptime) |
List a space allocation. | |
INT64_T | chirp_reli_group_create (const char *host, char *group, time_t stoptime) |
Create a new access control group. | |
INT64_T | chirp_reli_group_list (const char *host, const char *group, chirp_dir_t callback, void *arg, time_t stoptime) |
List members of an access control group. | |
INT64_T | chirp_reli_group_add (const char *host, char *group, char *user, time_t stoptime) |
Add a user to a group. | |
INT64_T | chirp_reli_group_remove (const char *host, char *group, char *user, time_t stoptime) |
Remove a user from a group. | |
INT64_T | chirp_reli_group_lookup (const char *host, const char *group, const char *user, time_t stoptime) |
Test membership in a group. | |
INT64_T | chirp_reli_job_begin (const char *host, const char *cwd, const char *input, const char *output, const char *error, const char *cmdline, time_t stoptime) |
Create a new active storage job. | |
INT64_T | chirp_reli_job_commit (const char *host, INT64_T jobid, time_t stoptime) |
Commit an active storage job. | |
INT64_T | chirp_reli_job_wait (const char *host, INT64_T jobid, struct chirp_job_state *state, int wait_time, time_t stoptime) |
Get status of an active storage job. | |
INT64_T | chirp_reli_job_kill (const char *host, INT64_T jobid, time_t stoptime) |
Kill an active storage job. | |
INT64_T | chirp_reli_job_remove (const char *host, INT64_T jobid, time_t stoptime) |
Remove an active storage job. | |
INT64_T | chirp_reli_job_list (const char *host, chirp_joblist_t callback, void *arg, time_t stoptime) |
List all active storage jobs. | |
INT64_T | chirp_reli_bulkio (struct chirp_bulkio *list, int count, time_t stoptime) |
Perform multiple I/O operations simultaneously. | |
INT64_T | chirp_reli_blocksize_get () |
Return the current buffer block size. | |
void | chirp_reli_blocksize_set (INT64_T bs) |
Set the buffer block size. | |
void | chirp_reli_cleanup_before_fork () |
Prepare to fork in a parallel program. |
|
Creates or opens a file in preparation for I/O.
|
|
Closes an open file.
Note that a close may need to write buffered data to disk before completing, so chirp_reli_close can fail. If chirp_reli_close indicates failures, the
|
|
Read data from a file. Small reads may be buffered into large reads for efficiency.
|
|
Write data to a file. Small writes may be buffered together into large writes for efficiency.
|
|
Read data from a file without buffering.
|
|
Write data to a file without buffering.
|
|
Strided read from a file.
Reads
|
|
Strided write to a file.
Writes
|
|
Get file status.
|
|
Get file system status.
|
|
Change the ownership of a file.
|
|
Change the mode bits of a file. Note that the current Chirp file server ignores the mode bits, except to determine whether a program is executable. See chirp_reli_setacl instead.
|
|
Truncate an open file.
|
|
Flush any pending changes to a file. To improve performance, Chirp buffers small writes to files. These writes might not be forced to disk until a later write or a call to chirp_reli_close. To force any buffered writes to disk, call this function.
|
|
Get an entire file efficiently. Reads an entire remote file, and write the contents to a standard FILE stream. To get an entire directory tree, see chirp_recursive_get instead.
|
|
Get an entire file efficiently to memory. Reads an entire remote file into newly allocated memory. To get an entire directory tree, see chirp_recursive_get instead.
|
|
Put an entire file efficiently. Reads data out of a standard I/O stream and writes it to a remote file. To put an entire directory tree, see chirp_recursive_put instead.
|
|
Put an entire file efficiently from memory. Reads data out of memory and writes it to a remote file. To put an entire directory tree, see chirp_recursive_put instead.
|
|
Get a detailed directory listing. Gets a detailed directory listing from a Chirp server, and then calls the callback once for each element in the directory.
|
|
Get a simple directory listing. Gets a simple directory listing from a Chirp server, and then calls the callback once for each element in the directory. This is a low-level function, you may find chirp_reli_opendir easier to use.
|
|
Open a directory for listing. This function returns a pointer to an opened directory. You may then call chirp_reli_readdir to read directory elements one by one.
|
|
Read one item from a directory. Accepts a pointer to a directory opened by chirp_reli_opendir and returns the next chirp_dirent object, which describes the name and properties of the next item in the list. Returns null when the list is complete. Note that this function has no timeout because it operates solely on memory structures.
|
|
Close a directory. This function releases the chirp_dir object returned by chirp_reli_opendir. It should be called after chirp_reli_readdir returns null to indicate the end of the directory. Note that this function has no timeoutbecause it operates solely on memory structures.
|
|
Get an access control list.
|
|
Modify an access control list.
|
|
Reset an access control list. This call will remove all entries from the access control list and grant to the calling user only those rights stated here.
|
|
Return the caller's identity.
|
|
Return the server's identity against another server. This causes the server to call another Chirp server and invoke chirp_reli_whoami.
|
|
Create a named pipe (FIFO). A named pipe (FIFO) is a rendezvous that appears as a file. Programs that read from the named pipe will block until another program connects and issues a write.
|
|
Delete a file.
|
|
Rename a file or directory.
|
|
Create a hard link.
|
|
Create a symbolic link.
|
|
Examine a symbolic link.
|
|
Create a new directory.
|
|
Create a new directory recursively.
|
|
Delete a directory if it is empty.
|
|
Delete a directory recursively. Deletes a directory recursively, even if it is not empty. The recursion is performed on the file server, so this call is efficient to perform over the network.
|
|
Get file status. If called on a symbolic link, chirp_reli_stat will follow that link and obtain the status of the underlying file.
|
|
Get file or link status. If called on a symbolic link, chirp_reli_lstat will return the status of the link itself.
|
|
Get filesystem status.
|
|
Check access permissions.
|
|
Change mode bits. Note that the current Chirp file server ignores the mode bits, except to determine whether a program is executable. See chirp_reli_setacl instead.
|
|
Change the ownership of a file.
|
|
Change the ownership of a file or link.
|
|
Truncate a file.
|
|
Change the modification times of a file.
|
|
Checksum a remote file. This MD5 checksum is performed remotely by the file server, so it is much more efficient than computing one by invoking a local command. Note that the data is returned in binary digest form. Use md5_string to convert the digest into a human readable form.
|
|
Return the local path of a file.
This function allows the caller to find out the local path where a file is stored, which is useful if you intend to execute a program on the host by some other means to access the file. Note that the local path will only be accessible if the directory ACL has been readable to the user
|
|
Measure remote space consumption. This routine causes the server to internally measure the space consumed by each user of the system. This could be a very long running function call. It then allocates a list of chirp_audit structures describing the current space usage. The caller is responsible for free()ing the list when done.
|
|
Third party transfer. Directs the server to transfer a file or directory to another (third-party) server. If a directory is mentioned, the transfer will be performed recursively, and will preserve the access controls present in the source directory.
|
|
Create a space allocation. Creates a new directory with a firm guarantee that the user will be able to store a specific amount of data there.
|
|
List a space allocation.
|
|
Create a new access control group. Note that group is deleted by calling chirp_reli_unlink on the group name.
|
|
List members of an access control group.
|
|
Add a user to a group.
|
|
Remove a user from a group.
|
|
Test membership in a group.
|
|
Create a new active storage job. This function creates a new job with a given command line and the explicitly names the current working directory and standard I/O files. A unique job identifier will be returned, which must be given to chirp_reli_job_commit, which will permit the job to run.
|
|
Commit an active storage job. A job created by chirp_reli_job_begin is not allowed to run until it is committed by calling chirp_reli_job_commit. This is called a two phase commit and is necessary to prevent runaway jobs of which the caller does not know the job ID. After this function is called, the job is free to run within the constraints of the local scheduler.
|
|
Get status of an active storage job. This function will obtain the detailed status of an active storage job, optionally waiting until it is complete. If wait_time is zero, then the job's status will be immediately returned. If wait_time is greater than zero, then this function will wait until the job reaches a completed state, or the wait_time expires. Note that stoptime must be greater than waittime, otherwise you will never receive a response.
|
|
Kill an active storage job. Forces the named job into the CHIRP_JOB_STATE_KILLED state. The job record must still be removed by calling chirp_reli_job_remove. The caller must be the owner of this job.
|
|
Remove an active storage job. Deletes the record and all other state associated with an active storage job. If the job is not yet complete, it will be killed first. The caller must be the owner of this job.
|
|
List all active storage jobs. Returns status about all known jobs on a server, regardless of their owner or state.
|
|
Perform multiple I/O operations simultaneously. This call invokes any number of I/O operations simultaneously, using pipelining and parallelism to complete quickly. Multiple operations may affect a single host, multiple hosts, or a mix of the two. The list of operations is given by an array of chirp_bulkio structures, each of which specifies an individual operation from chirp_reli.h. Note that only a select few operations are available through this interface. Future versions of the API may be more complete.
|
|
Return the current buffer block size. This module performs input and output buffering to improve the performance of small I/O operations. Operations larger than the buffer size are sent directly over the network, while those smaller are aggregated together. This function returns the current buffer size.
|
|
Set the buffer block size. This module performs input and output buffering to improve the performance of small I/O operations. Operations larger than the buffer size are sent directly over the network, while those smaller are aggregated together. This function sets the current buffer size.
|
|
Prepare to fork in a parallel program. The Chirp library is not thread-safe, but it can be used in a program that exploits parallelism by calling fork(). Before calling fork, this function must be invoked to clean up shared state such as TCP connections. After forking, each process will maintain its own connection to each Chirp server. |