Package org.apache.druid.segment.data

Class GenericIndexedWriter<T>

java.lang.Object
org.apache.druid.segment.data.GenericIndexedWriter<T>
All Implemented Interfaces:
DictionaryWriter<T>, Serializer
public class GenericIndexedWriter<T> extends Object implements DictionaryWriter<T>
Streams arrays of objects out in the binary format described by GenericIndexed The version EncodedStringDictionaryWriter.VERSION is reserved and must never be specified as the GenericIndexed version byte, else it will interfere with string column deserialization.

  • Field Details

  • Constructor Details

  • Method Details

    • ofCompressedByteBuffers

      public static GenericIndexedWriter<ByteBuffer> ofCompressedByteBuffers(SegmentWriteOutMedium segmentWriteOutMedium, String filenameBase, CompressionStrategy compressionStrategy, int bufferSize, int fileSizeLimit, Closer closer)
      Creates a new writer that accepts byte buffers and compresses them.
      Parameters:
      segmentWriteOutMedium - supplier of temporary files
      filenameBase - base filename to be used for secondary files, if multiple files are needed
      compressionStrategy - compression strategy to apply
      bufferSize - size of the buffers that will be passed in
      fileSizeLimit - limit for files created by the writer. In production code, this should always be MAX_FILE_SIZE. The parameter is exposed only for testing.
      closer - closer to attach temporary compression buffers to

    • compressedByteBuffersWriteObjectStrategy

      public static ObjectStrategy<ByteBuffer> compressedByteBuffersWriteObjectStrategy(CompressionStrategy compressionStrategy, int bufferSize, Closer closer)

    • generateValueFileName

      public static String generateValueFileName(String fileNameBase, int fileNum)

    • generateHeaderFileName

      public static String generateHeaderFileName(String fileNameBase)

    • open

      public void open() throws IOException
      Description copied from interface: DictionaryWriter
      Prepares the writer for writing
      Specified by:
      open in interface DictionaryWriter<T>
      Throws:
      IOException - if there is a problem with IO

    • setObjectsNotSorted

      public void setObjectsNotSorted()

    • isSorted

      public boolean isSorted()
      Specified by:
      isSorted in interface DictionaryWriter<T>

    • write

      public int write(@Nullable T objectToWrite) throws IOException
      Description copied from interface: DictionaryWriter
      Writes an object to the dictionary.

      Returns the index of the value that was just written. This is defined as the `int` value that can be passed into DictionaryWriter.get(int) such that it will return the same value back.

      Specified by:
      write in interface DictionaryWriter<T>
      Parameters:
      objectToWrite - object to be written to the dictionary
      Returns:
      index of the value that was just written
      Throws:
      IOException - if there is a problem with IO

    • get

      @Nullable public T get(int index) throws IOException
      Description copied from interface: DictionaryWriter
      Returns an object that has already been written via the DictionaryWriter.write(T) method.
      Specified by:
      get in interface DictionaryWriter<T>
      Parameters:
      index - index of the object to return
      Returns:
      the object identified by the given index
      Throws:
      IOException - if there is a problem with IO

    • getCardinality

      public int getCardinality()
      Description copied from interface: DictionaryWriter
      Returns the number of items that have been written so far in this dictionary. Any number lower than this cardinality can be passed into DictionaryWriter.get(int) and a value will be returned. If a value greater than or equal to the cardinality is passed into DictionaryWriter.get(int) all sorts of things could happen, but likely none of them are good.
      Specified by:
      getCardinality in interface DictionaryWriter<T>
      Returns:
      the number of items that have been written so far

    • getSerializedSize

      public long getSerializedSize()
      Description copied from interface: Serializer
      Returns the number of bytes, that this Serializer will write to the output _channel_ (not smoosher) on a Serializer.writeTo(java.nio.channels.WritableByteChannel, org.apache.druid.segment.file.SegmentFileBuilder) call.
      Specified by:
      getSerializedSize in interface Serializer

    • writeTo

      public void writeTo(WritableByteChannel channel, @Nullable SegmentFileBuilder fileBuilder) throws IOException
      Description copied from interface: Serializer
      Writes the serialized form of this object. The entire object may be written to the provided channel, or the object may be split over the provided channel and files added to the {@link SegmentFileBuilder], where additional channels can be created via SegmentFileBuilder.addWithChannel(String, long). The latter approach is useful when the serialized form of the object is too large for a single smoosh container. At the time this javadoc was written, the max smoosh container size is limit to the max ByteBuffer size.
      Specified by:
      writeTo in interface Serializer
      Throws:
      IOException