Package io.delta.kernel.internal.metrics

Class TransactionMetrics

Object
io.delta.kernel.internal.metrics.TransactionMetrics
public class TransactionMetrics extends Object
Stores the metrics for an ongoing transaction. These metrics are updated and recorded throughout the transaction using this class.

At report time, we create an immutable TransactionMetricsResult from an instance of TransactionMetrics to capture the metrics collected during the transaction. The TransactionMetricsResult interface exposes getters for any metrics collected in this class.

  • Field Details

    • totalCommitTimer

      public final Timer totalCommitTimer

    • commitAttemptsCounter

      public final Counter commitAttemptsCounter

    • totalActionsCounter

      public final Counter totalActionsCounter

  • Method Details

    • forNewTable

      public static TransactionMetrics forNewTable()
      Returns:
      a fresh TransactionMetrics object with a default tableFileSizeHistogram (with 0 counts)

    • withExistingTableFileSizeHistogram

      public static TransactionMetrics withExistingTableFileSizeHistogram(Optional<FileSizeHistogram> tableFileSizeHistogram)
      Returns:
      a fresh TransactionMetrics object with an initial tableFileSizeHistogram as provided

    • updateForAddFile

      public void updateForAddFile(long addFileSize)
      Updates the metrics for a seen AddFile with size addFileSize. Specifically, updates addFilesCounter, addFilesSizeInBytesCounter, and tableFileSizeHistogram. Note, it does NOT increment totalActionsCounter, this needs to be done separately.
      Parameters:
      addFileSize - the size of the add file to update the metrics for

    • updateForRemoveFile

      public void updateForRemoveFile(long removeFileSize)
      Updates the metrics for a seen RemoveFile with size removeFileSize. Specifically, updates removeFilesCounter, removeFilesSizeInBytesCounter, and tableFileSizeHistogram. Note, it does NOT increment totalActionsCounter, this needs to be done separately.
      Parameters:
      removeFileSize - the size of the remove file to update the metrics for

    • resetActionMetricsForRetry

      public void resetActionMetricsForRetry()
      Resets any action metrics for a failed commit to prepare them for retrying. Specifically,
      • Resets addFilesCounter, removeFilesCounter, totalActionsCounter, addFilesSizeInBytesCounter, and removeFilesSizeInBytesCounter to 0
      • Sets tableFileSizeHistogram to be empty since we don't know the updated distribution after the conflicting txn committed
      Action counters / tableFileSizeHistogram may be partially incremented if an action iterator is not read to completion (i.e. if an exception interrupts a file write). This allows us to reset the counters so that we can increment them correctly from 0 on a retry.

    • captureTransactionMetricsResult

      public TransactionMetricsResult captureTransactionMetricsResult()

    • toString

      public String toString()
      Overrides:
      toString in class Object