This module manages the storage of very large matrices (ranging from gigabytes to terabytes) by distributing the data across multiple Chirp servers. By harnessing the aggregate memory and storage of multiple machines, the time to process data is dramatically reduced.
#include "chirp_types.h"
Go to the source code of this file.
Functions | |
chirp_matrix * | chirp_matrix_create (const char *host, const char *path, int width, int height, int element_size, int nhosts, time_t stoptime) |
Create a new distributed matrix. | |
chirp_matrix * | chirp_matrix_open (const char *host, const char *path, time_t stoptime) |
Open an existing matrix. | |
int | chirp_matrix_get_row (struct chirp_matrix *matrix, int y, void *data, time_t stoptime) |
Get all values in a row. | |
int | chirp_matrix_set_row (struct chirp_matrix *matrix, int y, const void *data, time_t stoptime) |
Set all values in a row. | |
int | chirp_matrix_get_col (struct chirp_matrix *matrix, int x, void *data, time_t stoptime) |
Get all values in a column. | |
int | chirp_matrix_set_col (struct chirp_matrix *matrix, int x, const void *data, time_t stoptime) |
Set all values in a column. | |
int | chirp_matrix_get_range (struct chirp_matrix *matrix, int x, int y, int width, int height, void *data, time_t stoptime) |
Get a range of data. | |
int | chirp_matrix_set_range (struct chirp_matrix *matrix, int x, int y, int width, int height, const void *data, time_t stoptime) |
Set a range of data. | |
int | chirp_matrix_get (struct chirp_matrix *matrix, int x, int y, void *data, time_t stoptime) |
Get a single element. | |
int | chirp_matrix_set (struct chirp_matrix *matrix, int x, int y, const void *data, time_t stoptime) |
Set a single element. | |
int | chirp_matrix_setacl (const char *host, const char *path, const char *subject, const char *rights, time_t stoptime) |
Set the acls on a matrix. | |
int | chirp_matrix_width (struct chirp_matrix *matrix) |
Get the width of a matrix. | |
int | chirp_matrix_height (struct chirp_matrix *matrix) |
Get the height of a matrix. | |
int | chirp_matrix_element_size (struct chirp_matrix *matrix) |
Get the element size of a matrix. | |
int | chirp_matrix_nhosts (struct chirp_matrix *matrix) |
Get the number of hosts used by a matrix. | |
int | chirp_matrix_nfiles (struct chirp_matrix *matrix) |
Get the number of files used by a matrix. | |
void | chirp_matrix_fsync (struct chirp_matrix *matrix, time_t stoptime) |
Force all data to disk. | |
void | chirp_matrix_close (struct chirp_matrix *matrix, time_t stoptime) |
Close a matrix and free all related resources. | |
int | chirp_matrix_delete (const char *host, const char *path, time_t stoptime) |
Delete a matrix. |
|
Create a new distributed matrix. The host and path specified here will be used to store a small index file that contains the configuration of the matrix. The actual pieces of the matrix will be spread across multiple hosts. To tell Chirp where to store those pieces, you must create a hosts file that is a simple list of host names separated by newlines. Chirp will look for this file in $CHIRP_HOSTS, then in $HOME/.chirp/hosts. If neither is available, this call will fail.
|
|
Open an existing matrix.
|
|
Get all values in a row. This is the most efficient way to access data in a matrix.
|
|
Set all values in a row. This is the most efficient way to access data in a matrix.
|
|
Get all values in a column. Note that accessing columns is not as efficient as accessing rows. If possible, use chirp_matrix_get_row instead.
|
|
Set all values in a column. Note that accessing columns is not as efficient as accessing rows. If possible, use chirp_matrix_set_row instead.
|
|
Get a range of data.
|
|
Set a range of data.
|
|
Get a single element. Note: Reading a single element at a time is very inefficient. If possible, get multiple elements at once using chirp_matrix_get_row.
|
|
Set a single element. Note: Writing a single element at a time is very inefficient. If possible, set multiple elements at once using chirp_matrix_set_row.
|
|
Set the acls on a matrix.
|
|
Get the width of a matrix.
|
|
Get the height of a matrix.
|
|
Get the element size of a matrix.
|
|
Get the number of hosts used by a matrix.
|
|
Get the number of files used by a matrix. This value might be greater than the number of hosts, if the matrix is very large and it is necessary to break it into multiple files of one gigabyte or less.
|
|
Force all data to disk.
|
|
Close a matrix and free all related resources.
|
|
Delete a matrix.
|