Package org.apache.iotdb.tsfile.encoding.encoder

Class RleEncoder<T extends Comparable<T>>

  • Type Parameters:
    T - data type T for RLE
    Direct Known Subclasses:
    IntRleEncoder, LongRleEncoder
    public abstract class RleEncoder<T extends Comparable<T>>
extends Encoder
    Encodes values using a combination of run length encoding and bit packing, according to the following grammar: 
    
 rle-bit-packing-hybrid: <length> <bitwidth> <encoded-data>
 length := length of the <bitwidth> <encoded-data> in bytes stored as 4 bytes little endian
 bitwidth := bitwidth for all encoded data in <encoded-data>
 encoded-data := <run>*
 run := <bit-packed-run> | <rle-run>
 bit-packed-run := <bit-packed-header> <lastBitPackedNum> <bit-packed-values>
 bit-packed-header := varint-encode(<bit-pack-count> << 1 | 1)
 lastBitPackedNum := the number of useful value in last bit-pack may be less than 8, so
 lastBitPackedNum indicates how many values are useful
 bit-packed-values :=  bit packed
 rle-run := <rle-header> <repeated-value>
 rle-header := varint-encode( (number of times repeated) << 1)
 repeated-value := value that is repeated, using a fixed-width of round-up-to-next-byte(bit-width)

    • Field Detail

      • values

        protected List<T extends Comparable<T>> values
        we save all value in a list and calculate its bitwidth.

      • bitWidth

        protected int bitWidth
        the bit width used for bit-packing and rle.

      • repeatCount

        protected int repeatCount
        for a given value now buffered, how many times it occurs.

      • bitPackedGroupCount

        protected int bitPackedGroupCount
        the number of group which using bit packing, it is saved in header.

      • numBufferedValues

        protected int numBufferedValues
        the number of buffered value in array.

      • bytesBuffer

        protected List<byte[]> bytesBuffer
        we will write all bytes using bit-packing to OutputStream once. Before that, all bytes are saved in list.

      • isBitPackRun

        protected boolean isBitPackRun
        flag which indicate encoding mode false -- rle true -- bit-packing.

      • preValue

        protected T extends Comparable<T> preValue
        previous value written, used to detect repeated values.

      • bufferedValues

        protected T extends Comparable<T>[] bufferedValues
        array to buffer values temporarily.

      • isBitWidthSaved

        protected boolean isBitWidthSaved

      • byteCache

        protected ByteArrayOutputStream byteCache
        output stream to buffer <bitwidth> <encoded-data>.

    • Constructor Detail

      • RleEncoder

        protected RleEncoder()
        constructor.

    • Method Detail

      • reset

        protected void reset()

      • writeRleRun

        protected abstract void writeRleRun()
                             throws IOException
        Write bytes to OutputStream using rle. rle format: [header][value] header: (repeated value) << 1
        Throws:
        IOException - cannot write RLE run

      • writeOrAppendBitPackedRun

        public void writeOrAppendBitPackedRun()
        Start a bit-packing run transform values to bytes and buffer them in cache.

      • endPreviousBitPackedRun

        protected void endPreviousBitPackedRun​(int lastBitPackedNum)
        End a bit-packing run write all bit-packing group to OutputStream bit-packing format: [header][lastBitPackedNum][bit-packing group]+ [bit-packing group]+ are saved in List<byte[]> bytesBuffer .
        Parameters:
        lastBitPackedNum - - in last bit-packing group, it may have useful values less than 8. This param indicates how many values are useful

      • encodeValue

        protected void encodeValue​(T value)
        Encode T value using rle or bit-packing. It may not write to OutputStream immediately
        Parameters:
        value - - value to encode

      • clearBuffer

        protected abstract void clearBuffer()
        clean all useless value in bufferedValues and set 0.

      • convertBuffer

        protected abstract void convertBuffer()