API Documentation

Supported Compression Methods

In the table below, "Name String" refers to the term that you should provide as the -Z option for the wandiocat tool to produce compressed output in that particular format. The "Identifier" is the enumerated type that corresponds to the compression_type argument that must be provided when calling wandio_wcreate().

Method	Name String	Identifier	Supports Read	Supports Write
No Compression	NA	WANDIO_COMPRESS_NONE	Yes	Yes
Gzip / Zlib	"gzip"	WANDIO_COMPRESS_ZLIB	Yes	Yes
Bzip2	"bzip2"	WANDIO_COMPRESS_BZ2	Yes	Yes
LZO	"lzo"	WANDIO_COMPRESS_LZO	No	Yes
LZMA (.xz)	"lzma"	WANDIO_COMPRESS_LZMA	Yes	Yes
zstd	"zstd"	WANDIO_COMPRESS_ZSTD	Yes	Yes
LZ4	"lz4"	WANDIO_COMPRESS_LZ4	Yes	Yes

Note that support for each compression method is dependent on you having the appropriate development library installed on your system at compile time, e.g. LZ4 support is only available if liblz4-dev (or equivalent) is present on your system. If you've installed libwandio via a package, all of the supported methods will be available.

File Reading API

The following functions are the API for reading a file (compressed or uncompressed) using libwandio.

io_t *wandio_create(const char *filename);

Creates a new libwandio reader instance and opens the provided file for reading.

Arguments:

• filename -- the name of the file to open (full path to the file is recommended)

Returns a pointer to the new libwandio reader instance. This pointer will be required to call any of the other reading API functions.

The compression format for the file will be determined automatically by peeking at the initial bytes of the file and comparing them against known signatures for each supported compression format. If no signature is matched, the file will be assumed to be uncompressed.

io_t *wandio_create_uncompressed(const char *filename);

Creates a new libwandio reader instance and opens the provided file for reading.

Arguments:

• filename -- the name of the file to open (full path to the file is recommended)

Returns a pointer to the new libwandio reader instance. This pointer will be required to call any of the other reading API functions.

Unlike wandio_create(), this function will not attempt to detect the compression format for the file and instead automatically assume the file is uncompressed. This method is useful for dealing with situations where the initial bytes of a file coincidentally match the signature of one of the compression formats, which would cause wandio_create() to incorrectly infer that the file was compressed.

void wandio_destroy(io_t *io);

Destroys a libwandio reader instance, closing the file and freeing any memory associated with the reader.

Arguments:

• io -- the reader to be destroyed

int64_t wandio_read(io_t *io, void *buffer, int64_t len);

Reads data from a file using a libwandio reader, decompresses it (if required) and then writes it into the provided buffer.

Arguments:

• io -- the reader that is to perform the read
• buffer -- A pointer to a memory block that the decompressed data will be written into
• len -- The size of the buffer, in bytes.

Returns the number of bytes that have been written into the buffer as a result of the read operation. If this function returns 0, EOF has been reached and there is no more data to be read. If this function returns -1, an error has occurred.

Note that a successful read may not necessarily fill the provided buffer, i.e. return len.

Where possible, the contents of the buffer upon returning will be decompressed data, even if the original file is compressed. In other words, you should never need to worry about decompressing the buffer contents yourself.

int64_t wandio_peek(io_t *io, void *buffer, int64_t len);

Reads data from a file using a libwandio reader, decompresses it (if required) and then writes it into the provided buffer. Unlike wandio_read(), this function will NOT update the internal read pointer; the data read by this function call will not be "consumed".

Arguments:

• io -- the reader that is to perform the read
• buffer -- A pointer to a memory block that the decompressed data will be written into
• len -- The size of the buffer, in bytes.

Returns the number of bytes that have been written into the buffer as a result of the read operation. If this function returns 0, EOF has been reached and there is no more data to be read. If this function returns -1, an error has occurred.

Behaves exactly like wandio_read(), except that any subsequent read (or peek) operation will begin reading from the same point in the file that this peek operation did. This allows callers to look ahead in the file that they are reading from without affecting their primary reading and processing behaviour.

int64_t wandio_seek(io_t *io, int64_t offset, int whence);

Changes the read pointer offset for a libwandio reader to a specified value.

Arguments:

• io -- the reader that is to be modified
• offset -- the new offset for the read pointer
• whence -- a flag indicating how to interpret the given offset. Can be one of SEEK_SET (relative to the start of the file), SEEK_CUR (relative to the current read pointer), or SEEK_END (relative to the end of the file).

Returns the updated value for the read pointer offset. If an error occurred, returns -1 instead.

This function is the equivalent of lseek(2) for libwandio readers. Consult the lseek(2) manpage for more details on the meaning of the arguments.

Note that not all input formats will support seek operations. If this is the case, -1 will be returned and errno will be set to -ENOSYS.

int64_t wandio_tell(io_t *io);

Returns the current offset of the read pointer for a libwandio reader.

Arguments:

• io -- the reader that is to be queried

Note that not all input formats will support tell operations. If this is the case, -1 will be returned and errno will be set to -ENOSYS.

File Writing API

The following functions are the API for writing a (compressed) file using libwandio.

iow_t *wandio_wcreate(const char *filename, int compression_type, int compression_level, int flags);

Creates a new libwandio writer instance and opens the output file for writing.

Arguments:

• filename -- the name of the output file to open
• compression_type -- the type of compression to be applied when writing to the file. Must be one of the "Identifier" values listed in the table at the top of this page.
• compression_level -- the compression level to use when writing to the file. This can range from 0 - 9 inclusive. 0 typically represents no compression. Larger values will often result in a better compression ratio, at the cost of requiring significantly more CPU time to run. Note that some compression methods do not support the use of levels to modify behaviour; in these cases, the value of the level does not matter. If you are unsure, 1 is a good starting point for the compression level.
• flags -- flags to apply when opening the file, just like those used by open(2). Examples of flags would be O_CREAT and O_DIRECT but consult the open(2) manpage for more information.

Returns a pointer to the new libwandio writer instance which will be required to call any of the other writing API functions.

void wandio_wdestroy(iow_t *iow);

Destroys a libwandio writer instance, closing the file and freeing any memory associated with the writer structure.

Arguments:

• iow -- the writer to be destroyed

Note: this function will automatically ensure that any outstanding data to be written will be flushed, so you do not need to call wandio_flush() beforehand.

int64_t wandio_wwrite(iow_t *iow, const void *buffer, int64_t len);

Writes the contents of the provided buffer to a file using a libwandio writer instance. If the writer is configured to compress the output, the buffer contents will be compressed before being written.

Arguments:

• iow -- the writer that is to perform the write
• buffer -- A pointer to a block of memory containing the data that is to be written
• len -- The amount of writable data in the buffer, in bytes.

Returns the number of bytes from the provided buffer that were successfully consumed by the underlying write method. This should almost always match the value of the len argument, i.e. the function will block until all of the provided buffer contents are consumed. If an error occurs, -1 will be returned instead.

int wandio_wflush(iow_t *iow);

Flushes any output data that has been retained internally by the libwandio writer (or the libraries that it is depending on) to the output file.

Arguments:

• iow -- the writer that needs to be flushed

Returns the amount of data (in bytes) that was written to the file after the flush was completed (which may be zero, of course). If an error occurred, -1 is returned instead.