Package org.apache.druid.frame.wire

Class FrameCompression

java.lang.Object

org.apache.druid.frame.wire.FrameCompression

public class FrameCompression
extends Object

Utility class for compressing and decompressing data using LZ4 compression with a specific envelope format. Compressed format: - 1 byte: compression type: CompressionStrategy.getId(). Currently, only LZ4 is supported. - 8 bytes: compressed data length, little-endian long - 8 bytes: uncompressed data length, little-endian long - NNN bytes: LZ4-compressed data - 8 bytes: 64-bit xxhash checksum of prior content, including 17-byte header and compressed data, little-endian long

Field Summary

Fields

Modifier and Type	Field	Description
static final int	COMPRESSED_DATA_ENVELOPE_SIZE
static final int	COMPRESSED_DATA_HEADER_SIZE
static final int	COMPRESSED_DATA_TRAILER_SIZE

Method Summary

Modifier and Type	Method	Description
static ByteBuffer	compress(ByteBuffer data, ByteBuffer compressionBuffer)	Compresses data into the provided compression buffer.
static ByteBuffer	compress(org.apache.datasketches.memory.Memory data, long dataLength, ByteBuffer compressionBuffer)	Compresses data from Memory into the provided compression buffer.
static int	compressionBufferSize(long dataBytes)	Minimum size of compression buffer required by compress(org.apache.datasketches.memory.Memory, long, java.nio.ByteBuffer).
static byte[]	decompress(org.apache.datasketches.memory.Memory memory, long position, long length)	Decompresses the provided memory and returns a byte array with the decompressed data.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

COMPRESSED_DATA_HEADER_SIZE

public static final int COMPRESSED_DATA_HEADER_SIZE

See Also:

• Constant Field Values

COMPRESSED_DATA_TRAILER_SIZE

public static final int COMPRESSED_DATA_TRAILER_SIZE

See Also:

• Constant Field Values

COMPRESSED_DATA_ENVELOPE_SIZE

public static final int COMPRESSED_DATA_ENVELOPE_SIZE

See Also:

• Constant Field Values

Method Details

compressionBufferSize

public static int compressionBufferSize(long dataBytes)

Minimum size of compression buffer required by compress(org.apache.datasketches.memory.Memory, long, java.nio.ByteBuffer).

compress

public static ByteBuffer compress(org.apache.datasketches.memory.Memory data,
 long dataLength,
 ByteBuffer compressionBuffer)

Compresses data from Memory into the provided compression buffer.

Parameters:

data - the memory region to compress

dataLength - the number of bytes to compress from memory

compressionBuffer - buffer to hold compressed output; must be at least compressionBufferSize(long) bytes

Returns:

the compression buffer with position 0 and limit set to the number of bytes written

compress

public static ByteBuffer compress(ByteBuffer data,
 ByteBuffer compressionBuffer)

Compresses data into the provided compression buffer.

Parameters:

data - the data to compress (position and limit are used)

compressionBuffer - buffer to hold compressed output; must be at least compressionBufferSize(long) bytes

Returns:

the compression buffer with position 0 and limit set to the number of bytes written

decompress

public static byte[] decompress(org.apache.datasketches.memory.Memory memory,
 long position,
 long length)

Decompresses the provided memory and returns a byte array with the decompressed data. This operation is safe even on corrupt data: it validates position, length, and checksum prior to decompressing. This operation allocates memory on-heap to store the decompressed data.

Parameters:

memory - the memory containing compressed data

position - the position in memory where compressed data starts

length - the length of compressed data

Returns:

a byte array containing the decompressed data