Package org.apache.parquet.format

Class ColumnIndex

java.lang.Object
org.apache.parquet.format.ColumnIndex
All Implemented Interfaces:
Serializable, Cloneable, Comparable<ColumnIndex>, org.apache.thrift.TBase<ColumnIndex,ColumnIndex._Fields>, org.apache.thrift.TSerializable
@Generated(value="Autogenerated by Thrift Compiler (0.23.0)", date="2026-08-06") public class ColumnIndex extends Object implements org.apache.thrift.TBase<ColumnIndex,ColumnIndex._Fields>, Serializable, Cloneable, Comparable<ColumnIndex>
Optional statistics for each data page in a ColumnChunk. Forms part the page index, along with OffsetIndex. If this structure is present, OffsetIndex must also be present. For each field in this structure, [i] refers to the page at OffsetIndex.page_locations[i]
See Also:

  • Field Details

    • null_pages

      public List<Boolean> null_pages
      A list of Boolean values to determine the validity of the corresponding min and max values. If true, a page contains only null values, and writers have to set the corresponding entries in min_values and max_values to byte[0], so that all lists have the same length. If false, the corresponding entries in min_values and max_values must be valid.

    • min_values

      public List<ByteBuffer> min_values
      Two lists containing lower and upper bounds for the values of each page determined by the ColumnOrder of the column. These may be the actual minimum and maximum values found on a page, but can also be (more compact) values that do not exist on a page. For example, instead of storing "Blart Versenwald III", a writer may set min_values[i]="B", max_values[i]="C". Such more compact values must still be valid values within the column's logical type. Readers must make sure that list entries are populated before using them by inspecting null_pages. For columns of physical type FLOAT or DOUBLE, or logical type FLOAT16, NaN values are not to be included in these bounds. If all non-null values of a page are NaN, then a writer must do the following: - If the order of this column is TYPE_ORDER, then a column index must not be written for this column chunk. While this is unfortunate for performance, it is necessary to avoid conflict with legacy files that still included NaN in min_values and max_values even if the page had non-NaN values. To mitigate this, IEEE754_TOTAL_ORDER is recommended. - If the order of this column is IEEE754_TOTAL_ORDER, then min_values[i] and max_values[i] of that page must be set to the smallest and largest NaN values as defined by IEEE 754 total order.

    • max_values

      public List<ByteBuffer> max_values

    • boundary_order

      public BoundaryOrder boundary_order
      Stores whether both min_values and max_values are ordered and if so, in which direction. This allows readers to perform binary searches in both lists. Readers cannot assume that max_values[i] <= min_values[i+1], even if the lists are ordered.
      See Also:

    • null_counts

      public List<Long> null_counts
      A list containing the number of null values for each page Writers SHOULD always write this field even if no null values are present or the column is not nullable. Readers MUST distinguish between null_counts not being present and null_count being 0. If null_counts are not present, readers MUST NOT assume all null counts are 0.

    • repetition_level_histograms

      public List<Long> repetition_level_histograms
      Contains repetition level histograms for each page concatenated together. The repetition_level_histogram field on SizeStatistics contains more details. When present the length should always be (number of pages * (max_repetition_level + 1)) elements. Element 0 is the first element of the histogram for the first page. Element (max_repetition_level + 1) is the first element of the histogram for the second page.

    • definition_level_histograms

      public List<Long> definition_level_histograms
      Same as repetition_level_histograms except for definitions levels.

    • nan_counts

      public List<Long> nan_counts
      A list containing the number of NaN values for each page. Only present for columns of physical type FLOAT or DOUBLE, or logical type FLOAT16. If this field is not present, readers MUST assume that there might be NaN values in any page.

    • metaDataMap

      public static final Map<ColumnIndex._Fields,org.apache.thrift.meta_data.FieldMetaData> metaDataMap

  • Constructor Details

  • Method Details

    • deepCopy

      public ColumnIndex deepCopy()
      Specified by:
      deepCopy in interface org.apache.thrift.TBase<ColumnIndex,ColumnIndex._Fields>

    • clear

      public void clear()
      Specified by:
      clear in interface org.apache.thrift.TBase<ColumnIndex,ColumnIndex._Fields>

    • getNull_pagesSize

      public int getNull_pagesSize()

    • getNull_pagesIterator

      public Iterator<Boolean> getNull_pagesIterator()

    • addToNull_pages

      public void addToNull_pages(boolean elem)

    • getNull_pages

      public List<Boolean> getNull_pages()
      A list of Boolean values to determine the validity of the corresponding min and max values. If true, a page contains only null values, and writers have to set the corresponding entries in min_values and max_values to byte[0], so that all lists have the same length. If false, the corresponding entries in min_values and max_values must be valid.

    • setNull_pages

      public ColumnIndex setNull_pages(List<Boolean> null_pages)
      A list of Boolean values to determine the validity of the corresponding min and max values. If true, a page contains only null values, and writers have to set the corresponding entries in min_values and max_values to byte[0], so that all lists have the same length. If false, the corresponding entries in min_values and max_values must be valid.

    • unsetNull_pages

      public void unsetNull_pages()

    • isSetNull_pages

      public boolean isSetNull_pages()
      Returns true if field null_pages is set (has been assigned a value) and false otherwise

    • setNull_pagesIsSet

      public void setNull_pagesIsSet(boolean value)

    • getMin_valuesSize

      public int getMin_valuesSize()

    • getMin_valuesIterator

      public Iterator<ByteBuffer> getMin_valuesIterator()

    • addToMin_values

      public void addToMin_values(ByteBuffer elem)

    • getMin_values

      public List<ByteBuffer> getMin_values()
      Two lists containing lower and upper bounds for the values of each page determined by the ColumnOrder of the column. These may be the actual minimum and maximum values found on a page, but can also be (more compact) values that do not exist on a page. For example, instead of storing "Blart Versenwald III", a writer may set min_values[i]="B", max_values[i]="C". Such more compact values must still be valid values within the column's logical type. Readers must make sure that list entries are populated before using them by inspecting null_pages. For columns of physical type FLOAT or DOUBLE, or logical type FLOAT16, NaN values are not to be included in these bounds. If all non-null values of a page are NaN, then a writer must do the following: - If the order of this column is TYPE_ORDER, then a column index must not be written for this column chunk. While this is unfortunate for performance, it is necessary to avoid conflict with legacy files that still included NaN in min_values and max_values even if the page had non-NaN values. To mitigate this, IEEE754_TOTAL_ORDER is recommended. - If the order of this column is IEEE754_TOTAL_ORDER, then min_values[i] and max_values[i] of that page must be set to the smallest and largest NaN values as defined by IEEE 754 total order.

    • setMin_values

      public ColumnIndex setMin_values(List<ByteBuffer> min_values)
      Two lists containing lower and upper bounds for the values of each page determined by the ColumnOrder of the column. These may be the actual minimum and maximum values found on a page, but can also be (more compact) values that do not exist on a page. For example, instead of storing "Blart Versenwald III", a writer may set min_values[i]="B", max_values[i]="C". Such more compact values must still be valid values within the column's logical type. Readers must make sure that list entries are populated before using them by inspecting null_pages. For columns of physical type FLOAT or DOUBLE, or logical type FLOAT16, NaN values are not to be included in these bounds. If all non-null values of a page are NaN, then a writer must do the following: - If the order of this column is TYPE_ORDER, then a column index must not be written for this column chunk. While this is unfortunate for performance, it is necessary to avoid conflict with legacy files that still included NaN in min_values and max_values even if the page had non-NaN values. To mitigate this, IEEE754_TOTAL_ORDER is recommended. - If the order of this column is IEEE754_TOTAL_ORDER, then min_values[i] and max_values[i] of that page must be set to the smallest and largest NaN values as defined by IEEE 754 total order.

    • unsetMin_values

      public void unsetMin_values()

    • isSetMin_values

      public boolean isSetMin_values()
      Returns true if field min_values is set (has been assigned a value) and false otherwise

    • setMin_valuesIsSet

      public void setMin_valuesIsSet(boolean value)

    • getMax_valuesSize

      public int getMax_valuesSize()

    • getMax_valuesIterator

      public Iterator<ByteBuffer> getMax_valuesIterator()

    • addToMax_values

      public void addToMax_values(ByteBuffer elem)

    • getMax_values

      public List<ByteBuffer> getMax_values()

    • setMax_values

      public ColumnIndex setMax_values(List<ByteBuffer> max_values)

    • unsetMax_values

      public void unsetMax_values()

    • isSetMax_values

      public boolean isSetMax_values()
      Returns true if field max_values is set (has been assigned a value) and false otherwise

    • setMax_valuesIsSet

      public void setMax_valuesIsSet(boolean value)

    • getBoundary_order

      public BoundaryOrder getBoundary_order()
      Stores whether both min_values and max_values are ordered and if so, in which direction. This allows readers to perform binary searches in both lists. Readers cannot assume that max_values[i] <= min_values[i+1], even if the lists are ordered.
      See Also:

    • setBoundary_order

      public ColumnIndex setBoundary_order(BoundaryOrder boundary_order)
      Stores whether both min_values and max_values are ordered and if so, in which direction. This allows readers to perform binary searches in both lists. Readers cannot assume that max_values[i] <= min_values[i+1], even if the lists are ordered.
      See Also:

    • unsetBoundary_order

      public void unsetBoundary_order()

    • isSetBoundary_order

      public boolean isSetBoundary_order()
      Returns true if field boundary_order is set (has been assigned a value) and false otherwise

    • setBoundary_orderIsSet

      public void setBoundary_orderIsSet(boolean value)

    • getNull_countsSize

      public int getNull_countsSize()

    • getNull_countsIterator

      public Iterator<Long> getNull_countsIterator()

    • addToNull_counts

      public void addToNull_counts(long elem)

    • getNull_counts

      public List<Long> getNull_counts()
      A list containing the number of null values for each page Writers SHOULD always write this field even if no null values are present or the column is not nullable. Readers MUST distinguish between null_counts not being present and null_count being 0. If null_counts are not present, readers MUST NOT assume all null counts are 0.

    • setNull_counts

      public ColumnIndex setNull_counts(List<Long> null_counts)
      A list containing the number of null values for each page Writers SHOULD always write this field even if no null values are present or the column is not nullable. Readers MUST distinguish between null_counts not being present and null_count being 0. If null_counts are not present, readers MUST NOT assume all null counts are 0.

    • unsetNull_counts

      public void unsetNull_counts()

    • isSetNull_counts

      public boolean isSetNull_counts()
      Returns true if field null_counts is set (has been assigned a value) and false otherwise

    • setNull_countsIsSet

      public void setNull_countsIsSet(boolean value)

    • getRepetition_level_histogramsSize

      public int getRepetition_level_histogramsSize()

    • getRepetition_level_histogramsIterator

      public Iterator<Long> getRepetition_level_histogramsIterator()

    • addToRepetition_level_histograms

      public void addToRepetition_level_histograms(long elem)

    • getRepetition_level_histograms

      public List<Long> getRepetition_level_histograms()
      Contains repetition level histograms for each page concatenated together. The repetition_level_histogram field on SizeStatistics contains more details. When present the length should always be (number of pages * (max_repetition_level + 1)) elements. Element 0 is the first element of the histogram for the first page. Element (max_repetition_level + 1) is the first element of the histogram for the second page.

    • setRepetition_level_histograms

      public ColumnIndex setRepetition_level_histograms(List<Long> repetition_level_histograms)
      Contains repetition level histograms for each page concatenated together. The repetition_level_histogram field on SizeStatistics contains more details. When present the length should always be (number of pages * (max_repetition_level + 1)) elements. Element 0 is the first element of the histogram for the first page. Element (max_repetition_level + 1) is the first element of the histogram for the second page.

    • unsetRepetition_level_histograms

      public void unsetRepetition_level_histograms()

    • isSetRepetition_level_histograms

      public boolean isSetRepetition_level_histograms()
      Returns true if field repetition_level_histograms is set (has been assigned a value) and false otherwise

    • setRepetition_level_histogramsIsSet

      public void setRepetition_level_histogramsIsSet(boolean value)

    • getDefinition_level_histogramsSize

      public int getDefinition_level_histogramsSize()

    • getDefinition_level_histogramsIterator

      public Iterator<Long> getDefinition_level_histogramsIterator()

    • addToDefinition_level_histograms

      public void addToDefinition_level_histograms(long elem)

    • getDefinition_level_histograms

      public List<Long> getDefinition_level_histograms()
      Same as repetition_level_histograms except for definitions levels.

    • setDefinition_level_histograms

      public ColumnIndex setDefinition_level_histograms(List<Long> definition_level_histograms)
      Same as repetition_level_histograms except for definitions levels.

    • unsetDefinition_level_histograms

      public void unsetDefinition_level_histograms()

    • isSetDefinition_level_histograms

      public boolean isSetDefinition_level_histograms()
      Returns true if field definition_level_histograms is set (has been assigned a value) and false otherwise

    • setDefinition_level_histogramsIsSet

      public void setDefinition_level_histogramsIsSet(boolean value)

    • getNan_countsSize

      public int getNan_countsSize()

    • getNan_countsIterator

      public Iterator<Long> getNan_countsIterator()

    • addToNan_counts

      public void addToNan_counts(long elem)

    • getNan_counts

      public List<Long> getNan_counts()
      A list containing the number of NaN values for each page. Only present for columns of physical type FLOAT or DOUBLE, or logical type FLOAT16. If this field is not present, readers MUST assume that there might be NaN values in any page.

    • setNan_counts

      public ColumnIndex setNan_counts(List<Long> nan_counts)
      A list containing the number of NaN values for each page. Only present for columns of physical type FLOAT or DOUBLE, or logical type FLOAT16. If this field is not present, readers MUST assume that there might be NaN values in any page.

    • unsetNan_counts

      public void unsetNan_counts()

    • isSetNan_counts

      public boolean isSetNan_counts()
      Returns true if field nan_counts is set (has been assigned a value) and false otherwise

    • setNan_countsIsSet

      public void setNan_countsIsSet(boolean value)

    • setFieldValue

      public void setFieldValue(ColumnIndex._Fields field, Object value)
      Specified by:
      setFieldValue in interface org.apache.thrift.TBase<ColumnIndex,ColumnIndex._Fields>

    • getFieldValue

      public Object getFieldValue(ColumnIndex._Fields field)
      Specified by:
      getFieldValue in interface org.apache.thrift.TBase<ColumnIndex,ColumnIndex._Fields>

    • isSet

      public boolean isSet(ColumnIndex._Fields field)
      Returns true if field corresponding to fieldID is set (has been assigned a value) and false otherwise
      Specified by:
      isSet in interface org.apache.thrift.TBase<ColumnIndex,ColumnIndex._Fields>

    • equals

      public boolean equals(Object that)
      Overrides:
      equals in class Object

    • equals

      public boolean equals(ColumnIndex that)

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • compareTo

      public int compareTo(ColumnIndex other)
      Specified by:
      compareTo in interface Comparable<ColumnIndex>

    • fieldForId

      public ColumnIndex._Fields fieldForId(int fieldId)
      Specified by:
      fieldForId in interface org.apache.thrift.TBase<ColumnIndex,ColumnIndex._Fields>

    • read

      public void read(org.apache.thrift.protocol.TProtocol iprot) throws org.apache.thrift.TException
      Specified by:
      read in interface org.apache.thrift.TSerializable
      Throws:
      org.apache.thrift.TException

    • write

      public void write(org.apache.thrift.protocol.TProtocol oprot) throws org.apache.thrift.TException
      Specified by:
      write in interface org.apache.thrift.TSerializable
      Throws:
      org.apache.thrift.TException

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • validate

      public void validate() throws org.apache.thrift.TException
      Throws:
      org.apache.thrift.TException