Package io.delta.kernel.internal.stats

Class FileSizeHistogram

Object
io.delta.kernel.internal.stats.FileSizeHistogram
public class FileSizeHistogram extends Object
A histogram that tracks file size distributions and their counts.

  • Field Details

    • FULL_SCHEMA

      public static final StructType FULL_SCHEMA

  • Constructor Details

    • FileSizeHistogram

      @VisibleForTesting public FileSizeHistogram(long[] sortedBinBoundaries, long[] fileCounts, long[] totalBytes)

  • Method Details

    • createDefaultHistogram

      public static FileSizeHistogram createDefaultHistogram()
      Creates a default FileSizeHistogram with predefined bin boundaries and zero counts.

    • fromColumnVector

      public static Optional<FileSizeHistogram> fromColumnVector(ColumnVector vector, int rowId)
      Creates a FileSizeHistogram from a column vector.

    • fromFileSizeHistogramResult

      public static FileSizeHistogram fromFileSizeHistogramResult(FileSizeHistogramResult fileSizeHistogramResult)

    • insert

      public void insert(long fileSize)
      Adds a file size to the histogram, incrementing the appropriate bin's count and total bytes. The appropriate bin refers to a bin with boundary that is less than or equal to the file size. Files larger than the maximum bin boundary (256 GB) are placed in the last bin.
      Parameters:
      fileSize - The size of the file in bytes
      Throws:
      IllegalArgumentException - if fileSize is negative or if getBinIndex returns an invalid index

    • remove

      public void remove(long fileSize)
      Removes a file size from the histogram, decrementing the appropriate bin's count and total bytes.
      Parameters:
      fileSize - The size of the file in bytes
      Throws:
      IllegalArgumentException - if fileSize is negative

    • toRow

      public Row toRow()
      Encode as a Row object with the schema FULL_SCHEMA.

    • captureFileSizeHistogramResult

      public FileSizeHistogramResult captureFileSizeHistogramResult()

    • plus

      public FileSizeHistogram plus(FileSizeHistogram other)
      Adds two histograms together by combining their counts and total bytes. Both histograms must have the same bin boundaries.
      Parameters:
      other - The histogram to add to this histogram
      Returns:
      A new histogram with combined statistics
      Throws:
      IllegalArgumentException - if the histograms have different bin boundaries

    • minus

      public FileSizeHistogram minus(FileSizeHistogram other)
      Subtracts another histogram from this one. Both histograms must have the same bin boundaries. The result will ensure no negative counts or bytes.
      Parameters:
      other - The histogram to subtract from this histogram
      Returns:
      A new histogram with the difference in statistics
      Throws:
      IllegalArgumentException - if the histograms have different bin boundaries
      IllegalArgumentException - if subtraction would result in negative counts or bytes

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object