Package org.apache.parquet.format

Class SizeStatistics

java.lang.Object
org.apache.parquet.format.SizeStatistics
All Implemented Interfaces:
Serializable, Cloneable, Comparable<SizeStatistics>, org.apache.thrift.TBase<SizeStatistics,SizeStatistics._Fields>, org.apache.thrift.TSerializable
@Generated(value="Autogenerated by Thrift Compiler (0.23.0)", date="2026-08-06") public class SizeStatistics extends Object implements org.apache.thrift.TBase<SizeStatistics,SizeStatistics._Fields>, Serializable, Cloneable, Comparable<SizeStatistics>
A structure for capturing metadata for estimating the unencoded, uncompressed size of data written. This is useful for readers to estimate how much memory is needed to reconstruct data in their memory model and for fine grained filter pushdown on nested structures (the histograms contained in this structure can help determine the number of nulls at a particular nesting level and maximum length of lists).
See Also:

  • Field Details

    • unencoded_byte_array_data_bytes

      public long unencoded_byte_array_data_bytes
      The number of physical bytes stored for BYTE_ARRAY data values assuming no encoding. This is exclusive of the bytes needed to store the length of each byte array. In other words, this field is equivalent to the `(size of PLAIN-ENCODING the byte array values) - (4 bytes * number of values written)`. To determine unencoded sizes of other types readers can use schema information multiplied by the number of non-null and null values. The number of null/non-null values can be inferred from the histograms below. For example, if a column chunk is dictionary-encoded with dictionary ["a", "bc", "cde"], and a data page contains the indices [0, 0, 1, 2], then this value for that data page should be 7 (1 + 1 + 2 + 3). This field should only be set for types that use BYTE_ARRAY as their physical type.

    • repetition_level_histogram

      public List<Long> repetition_level_histogram
      When present, there is expected to be one element corresponding to each repetition (i.e. size=max repetition_level+1) where each element represents the number of times the repetition level was observed in the data. This field may be omitted if max_repetition_level is 0 without loss of information.

    • definition_level_histogram

      public List<Long> definition_level_histogram
      Same as repetition_level_histogram except for definition levels. This field may be omitted if max_definition_level is 0 or 1 without loss of information.

    • metaDataMap

      public static final Map<SizeStatistics._Fields,org.apache.thrift.meta_data.FieldMetaData> metaDataMap

  • Constructor Details

    • SizeStatistics

      public SizeStatistics()

    • SizeStatistics

      public SizeStatistics(SizeStatistics other)
      Performs a deep copy on other.

  • Method Details

    • deepCopy

      public SizeStatistics deepCopy()
      Specified by:
      deepCopy in interface org.apache.thrift.TBase<SizeStatistics,SizeStatistics._Fields>

    • clear

      public void clear()
      Specified by:
      clear in interface org.apache.thrift.TBase<SizeStatistics,SizeStatistics._Fields>

    • getUnencoded_byte_array_data_bytes

      public long getUnencoded_byte_array_data_bytes()
      The number of physical bytes stored for BYTE_ARRAY data values assuming no encoding. This is exclusive of the bytes needed to store the length of each byte array. In other words, this field is equivalent to the `(size of PLAIN-ENCODING the byte array values) - (4 bytes * number of values written)`. To determine unencoded sizes of other types readers can use schema information multiplied by the number of non-null and null values. The number of null/non-null values can be inferred from the histograms below. For example, if a column chunk is dictionary-encoded with dictionary ["a", "bc", "cde"], and a data page contains the indices [0, 0, 1, 2], then this value for that data page should be 7 (1 + 1 + 2 + 3). This field should only be set for types that use BYTE_ARRAY as their physical type.

    • setUnencoded_byte_array_data_bytes

      public SizeStatistics setUnencoded_byte_array_data_bytes(long unencoded_byte_array_data_bytes)
      The number of physical bytes stored for BYTE_ARRAY data values assuming no encoding. This is exclusive of the bytes needed to store the length of each byte array. In other words, this field is equivalent to the `(size of PLAIN-ENCODING the byte array values) - (4 bytes * number of values written)`. To determine unencoded sizes of other types readers can use schema information multiplied by the number of non-null and null values. The number of null/non-null values can be inferred from the histograms below. For example, if a column chunk is dictionary-encoded with dictionary ["a", "bc", "cde"], and a data page contains the indices [0, 0, 1, 2], then this value for that data page should be 7 (1 + 1 + 2 + 3). This field should only be set for types that use BYTE_ARRAY as their physical type.

    • unsetUnencoded_byte_array_data_bytes

      public void unsetUnencoded_byte_array_data_bytes()

    • isSetUnencoded_byte_array_data_bytes

      public boolean isSetUnencoded_byte_array_data_bytes()
      Returns true if field unencoded_byte_array_data_bytes is set (has been assigned a value) and false otherwise

    • setUnencoded_byte_array_data_bytesIsSet

      public void setUnencoded_byte_array_data_bytesIsSet(boolean value)

    • getRepetition_level_histogramSize

      public int getRepetition_level_histogramSize()

    • getRepetition_level_histogramIterator

      public Iterator<Long> getRepetition_level_histogramIterator()

    • addToRepetition_level_histogram

      public void addToRepetition_level_histogram(long elem)

    • getRepetition_level_histogram

      public List<Long> getRepetition_level_histogram()
      When present, there is expected to be one element corresponding to each repetition (i.e. size=max repetition_level+1) where each element represents the number of times the repetition level was observed in the data. This field may be omitted if max_repetition_level is 0 without loss of information.

    • setRepetition_level_histogram

      public SizeStatistics setRepetition_level_histogram(List<Long> repetition_level_histogram)
      When present, there is expected to be one element corresponding to each repetition (i.e. size=max repetition_level+1) where each element represents the number of times the repetition level was observed in the data. This field may be omitted if max_repetition_level is 0 without loss of information.

    • unsetRepetition_level_histogram

      public void unsetRepetition_level_histogram()

    • isSetRepetition_level_histogram

      public boolean isSetRepetition_level_histogram()
      Returns true if field repetition_level_histogram is set (has been assigned a value) and false otherwise

    • setRepetition_level_histogramIsSet

      public void setRepetition_level_histogramIsSet(boolean value)

    • getDefinition_level_histogramSize

      public int getDefinition_level_histogramSize()

    • getDefinition_level_histogramIterator

      public Iterator<Long> getDefinition_level_histogramIterator()

    • addToDefinition_level_histogram

      public void addToDefinition_level_histogram(long elem)

    • getDefinition_level_histogram

      public List<Long> getDefinition_level_histogram()
      Same as repetition_level_histogram except for definition levels. This field may be omitted if max_definition_level is 0 or 1 without loss of information.

    • setDefinition_level_histogram

      public SizeStatistics setDefinition_level_histogram(List<Long> definition_level_histogram)
      Same as repetition_level_histogram except for definition levels. This field may be omitted if max_definition_level is 0 or 1 without loss of information.

    • unsetDefinition_level_histogram

      public void unsetDefinition_level_histogram()

    • isSetDefinition_level_histogram

      public boolean isSetDefinition_level_histogram()
      Returns true if field definition_level_histogram is set (has been assigned a value) and false otherwise

    • setDefinition_level_histogramIsSet

      public void setDefinition_level_histogramIsSet(boolean value)

    • setFieldValue

      public void setFieldValue(SizeStatistics._Fields field, Object value)
      Specified by:
      setFieldValue in interface org.apache.thrift.TBase<SizeStatistics,SizeStatistics._Fields>

    • getFieldValue

      public Object getFieldValue(SizeStatistics._Fields field)
      Specified by:
      getFieldValue in interface org.apache.thrift.TBase<SizeStatistics,SizeStatistics._Fields>

    • isSet

      public boolean isSet(SizeStatistics._Fields field)
      Returns true if field corresponding to fieldID is set (has been assigned a value) and false otherwise
      Specified by:
      isSet in interface org.apache.thrift.TBase<SizeStatistics,SizeStatistics._Fields>

    • equals

      public boolean equals(Object that)
      Overrides:
      equals in class Object

    • equals

      public boolean equals(SizeStatistics that)

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • compareTo

      public int compareTo(SizeStatistics other)
      Specified by:
      compareTo in interface Comparable<SizeStatistics>

    • fieldForId

      public SizeStatistics._Fields fieldForId(int fieldId)
      Specified by:
      fieldForId in interface org.apache.thrift.TBase<SizeStatistics,SizeStatistics._Fields>

    • read

      public void read(org.apache.thrift.protocol.TProtocol iprot) throws org.apache.thrift.TException
      Specified by:
      read in interface org.apache.thrift.TSerializable
      Throws:
      org.apache.thrift.TException

    • write

      public void write(org.apache.thrift.protocol.TProtocol oprot) throws org.apache.thrift.TException
      Specified by:
      write in interface org.apache.thrift.TSerializable
      Throws:
      org.apache.thrift.TException

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • validate

      public void validate() throws org.apache.thrift.TException
      Throws:
      org.apache.thrift.TException