Package org.apache.druid.segment.data

Class FrontCodedIndexedWriter

java.lang.Object
org.apache.druid.segment.data.FrontCodedIndexedWriter
All Implemented Interfaces:
DictionaryWriter<byte[]>, Serializer
public class FrontCodedIndexedWriter extends Object implements DictionaryWriter<byte[]>
DictionaryWriter for a FrontCodedIndexed, written to a SegmentWriteOutMedium. Values MUST be added to this dictionary writer in sorted order, which is enforced. Front coding is a type of delta encoding for byte arrays, where values are grouped into buckets. The first value of the bucket is written entirely, and remaining values are stored as pairs of an integer which indicates how much of the first byte array of the bucket to use as a prefix, (or the preceding value of the bucket if using 'incremental' buckets) followed by the remaining value bytes after the prefix. This writer is designed for use with UTF-8 encoded strings that are written in an order compatible with String.compareTo(String).
See Also:

  • Constructor Details

    • FrontCodedIndexedWriter

      public FrontCodedIndexedWriter(SegmentWriteOutMedium segmentWriteOutMedium, ByteOrder byteOrder, int bucketSize, byte version)

  • Method Details

    • open

      public void open() throws IOException
      Description copied from interface: DictionaryWriter
      Prepares the writer for writing
      Specified by:
      open in interface DictionaryWriter<byte[]>
      Throws:
      IOException - if there is a problem with IO

    • write

      public int write(@Nullable byte[] value) throws IOException
      Description copied from interface: DictionaryWriter
      Writes an object to the dictionary.

      Returns the index of the value that was just written. This is defined as the `int` value that can be passed into DictionaryWriter.get(int) such that it will return the same value back.

      Specified by:
      write in interface DictionaryWriter<byte[]>
      Parameters:
      value - object to be written to the dictionary
      Returns:
      index of the value that was just written
      Throws:
      IOException - if there is a problem with IO

    • getSerializedSize

      public long getSerializedSize() throws IOException
      Description copied from interface: Serializer
      Returns the number of bytes, that this Serializer will write to the output _channel_ (not smoosher) on a Serializer.writeTo(java.nio.channels.WritableByteChannel, org.apache.druid.segment.file.SegmentFileBuilder) call.
      Specified by:
      getSerializedSize in interface Serializer
      Throws:
      IOException

    • writeTo

      public void writeTo(WritableByteChannel channel, SegmentFileBuilder fileBuilder) throws IOException
      Description copied from interface: Serializer
      Writes the serialized form of this object. The entire object may be written to the provided channel, or the object may be split over the provided channel and files added to the {@link SegmentFileBuilder], where additional channels can be created via SegmentFileBuilder.addWithChannel(String, long). The latter approach is useful when the serialized form of the object is too large for a single smoosh container. At the time this javadoc was written, the max smoosh container size is limit to the max ByteBuffer size.
      Specified by:
      writeTo in interface Serializer
      Throws:
      IOException

    • isSorted

      public boolean isSorted()
      Specified by:
      isSorted in interface DictionaryWriter<byte[]>

    • get

      @Nullable public byte[] get(int index) throws IOException
      Description copied from interface: DictionaryWriter
      Returns an object that has already been written via the DictionaryWriter.write(T) method.
      Specified by:
      get in interface DictionaryWriter<byte[]>
      Parameters:
      index - index of the object to return
      Returns:
      the object identified by the given index
      Throws:
      IOException - if there is a problem with IO

    • getCardinality

      public int getCardinality()
      Description copied from interface: DictionaryWriter
      Returns the number of items that have been written so far in this dictionary. Any number lower than this cardinality can be passed into DictionaryWriter.get(int) and a value will be returned. If a value greater than or equal to the cardinality is passed into DictionaryWriter.get(int) all sorts of things could happen, but likely none of them are good.
      Specified by:
      getCardinality in interface DictionaryWriter<byte[]>
      Returns:
      the number of items that have been written so far

    • writeBucketV0

      public static int writeBucketV0(ByteBuffer buffer, byte[][] values, int numValues)
      Write bucket of values to a ByteBuffer. The first value is written completely, subsequent values are written with an integer to indicate how much of the first value in the bucket is a prefix of the value, followed by the remaining bytes of the value. Uses VByte encoded integers to indicate prefix length and value length.

    • writeBucketV1

      public static int writeBucketV1(ByteBuffer buffer, byte[][] values, int numValues)
      Write bucket of values to a ByteBuffer. The first value is written completely, subsequent values are written with an integer to indicate how much of the preceding value in the bucket is a prefix of the value, followed by the remaining bytes of the value. Uses VByte encoded integers to indicate prefix length and value length.

    • writeValue

      public static int writeValue(ByteBuffer buffer, byte[] bytes)
      Write a variable length byte[] value to a ByteBuffer, storing the length as a VByte encoded integer followed by the value itself. Returns the number of bytes written to the buffer. This method returns a negative value if there is no room available in the buffer, so that it can be grown if needed.