Package org.apache.parquet.column.values.bitpacking

Class BytePacker

java.lang.Object
org.apache.parquet.column.values.bitpacking.BytePacker
public abstract class BytePacker extends Object
Packs and unpacks into bytes

packing unpacking treats: - n values at a time (with n % 8 == 0) - bitWidth * (n/8) bytes at a time.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected int
    unpackCount
    Number of integer values to be unpacked at a time.

  • Method Summary

    Modifier and Type
    Method
    Description
    final int
    getBitWidth()
     
    int
    getUnpackCount()
     
    abstract void
    pack32Values(int[] input, int inPos, byte[] output, int outPos)
    pack 32 values from input at inPos into bitWidth * 4 bytes in output at outPos.
    abstract void
    pack8Values(int[] input, int inPos, byte[] output, int outPos)
    pack 8 values from input at inPos into bitWidth bytes in output at outPos.
    void
    unpack32Values(byte[] input, int inPos, int[] output, int outPos)
    Deprecated.
    abstract void
    unpack32Values(ByteBuffer input, int inPos, int[] output, int outPos)
    unpack bitWidth * 4 bytes from input at inPos into 32 values in output at outPos.
    void
    unpack8Values(byte[] input, int inPos, int[] output, int outPos)
    Deprecated.
    abstract void
    unpack8Values(ByteBuffer input, int inPos, int[] output, int outPos)
    unpack bitWidth bytes from input at inPos into 8 values in output at outPos.
    void
    unpackValuesUsingVector(byte[] input, int inPos, int[] output, int outPos)
    unpack bitWidth bytes from input at inPos into {unpackCount} values in output at outPos using Java Vector API.
    void
    unpackValuesUsingVector(ByteBuffer input, int inPos, int[] output, int outPos)
    unpack bitWidth bytes from input at inPos into {unpackCount} values in output at outPos using Java Vector API.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • unpackCount

      protected int unpackCount
      Number of integer values to be unpacked at a time. unpackCount is a multiple of 8 For AVX512, the register is 512 bits, so output values at a time maybe different when different bitWidth.

  • Method Details

    • getBitWidth

      public final int getBitWidth()
      Returns:
      the width in bits used for encoding, also how many bytes are packed/unpacked at a time by pack8Values/unpack8Values

    • getUnpackCount

      public int getUnpackCount()

    • pack8Values

      public abstract void pack8Values(int[] input, int inPos, byte[] output, int outPos)
      pack 8 values from input at inPos into bitWidth bytes in output at outPos. nextPosition: inPos += 8; outPos += getBitWidth()
      Parameters:
      input - the input values
      inPos - where to read from in input
      output - the output bytes
      outPos - where to write to in output

    • pack32Values

      public abstract void pack32Values(int[] input, int inPos, byte[] output, int outPos)
      pack 32 values from input at inPos into bitWidth * 4 bytes in output at outPos. nextPosition: inPos += 32; outPos += getBitWidth() * 4
      Parameters:
      input - the input values
      inPos - where to read from in input
      output - the output bytes
      outPos - where to write to in output

    • unpack8Values

      public abstract void unpack8Values(ByteBuffer input, int inPos, int[] output, int outPos)
      unpack bitWidth bytes from input at inPos into 8 values in output at outPos. nextPosition: inPos += getBitWidth(); outPos += 8
      Parameters:
      input - the input bytes
      inPos - where to read from in input
      output - the output values
      outPos - where to write to in output

    • unpack8Values

      @Deprecated public void unpack8Values(byte[] input, int inPos, int[] output, int outPos)
      Deprecated.
      Compatibility API
      Parameters:
      input - the input bytes
      inPos - where to read from in input
      output - the output values
      outPos - where to write to in output

    • unpack32Values

      public abstract void unpack32Values(ByteBuffer input, int inPos, int[] output, int outPos)
      unpack bitWidth * 4 bytes from input at inPos into 32 values in output at outPos. nextPosition: inPos += getBitWidth() * 4; outPos += 32
      Parameters:
      input - the input bytes
      inPos - where to read from in input
      output - the output values
      outPos - where to write to in output

    • unpack32Values

      @Deprecated public void unpack32Values(byte[] input, int inPos, int[] output, int outPos)
      Deprecated.
      Compatibility API
      Parameters:
      input - the input bytes
      inPos - where to read from in input
      output - the output values
      outPos - where to write to in output

    • unpackValuesUsingVector

      public void unpackValuesUsingVector(byte[] input, int inPos, int[] output, int outPos)
      unpack bitWidth bytes from input at inPos into {unpackCount} values in output at outPos using Java Vector API.
      Parameters:
      input - the input bytes
      inPos - where to read from in input
      output - the output values
      outPos - where to write to in output

    • unpackValuesUsingVector

      public void unpackValuesUsingVector(ByteBuffer input, int inPos, int[] output, int outPos)
      unpack bitWidth bytes from input at inPos into {unpackCount} values in output at outPos using Java Vector API.
      Parameters:
      input - the input bytes
      inPos - where to read from in input
      output - the output values
      outPos - where to write to in output