Contents

ZFP

ZFP is a compression library for floating-point data stored in multidimensional arrays. Compression of d-dimensional (1D, 2D, 3D, and 4D) array of floating-point values is happening by partitioning the source array into cubical blocks of 4d values, i.e. 4, 16, 64, or 256 values respectively. Each such block is (de)compressed independently into a fixed or variable length bit string, and these bit strings are concatenated into a single stream of bits.

ZFP usually truncates each bit string to a fixed number of bits to meet a storage budget or to some variable length needed to meet a given error tolerance, as dictated by the compressibility of the data. The bit string representing any given block may be truncated at any point and still yield a valid approximation. The early bits are most important. The later bits progressively refine the approximation, similar to how the last few bits in a floating-point number have less significance than the first several bits and can often be discarded (zeroed) with limited impact on accuracy.

For more information about this algorithm see the official documentation.

oneDTL Specific

oneDTL ZFP compression is based on the open-source ZFP compression library. oneDTL implementation encoded bitstream must be compatible with it as well.

ZFP compressor provides multiple modes of work and data types. The implementation must support modes marked with v in the table below, other modes are optional but may become a requirement in the future:

Compression modes

1D

2D

3D

4D

fixed-rate

x

x

v

x

fixed-precision

x

x

x

x

fixed-accuracy

x

x

x

x

expert

x

x

x

x

Chunks

Multiple blocks may be processed at once. Such combination of blocks is called a chunk.

Index Array

Index array must be provided for all modes except fixed-rate to correctly perform decompression. Values of this array must point to the beginning of each block in the compressed stream.

The following types of indexing should be supported:

enum oneapi::dtl::data_compression::zfp_index_type

Type of index array.

Values:

enumerator none

No index array produce. Only for fix rate mode.

enumerator offset

Array of offset between chunks produces.

enumerator length

Array of chunks length produces.

enumerator hybrid

Hybrid mode with offset and lengths.

previous

Data Compression

next

Compression Modes