Package io.delta.kernel.internal

Class DeltaHistoryManager

Object
io.delta.kernel.internal.DeltaHistoryManager
public final class DeltaHistoryManager extends Object

  • Method Details

    • getVersionAtOrAfterTimestamp

      public static long getVersionAtOrAfterTimestamp(Engine engine, Path logPath, long millisSinceEpochUTC, SnapshotImpl latestSnapshot, List<ParsedCatalogCommitData> catalogCommits)
      Returns the latest version that was committed at or after millisSinceEpochUTC. If no version exists, throws a KernelException

      Specifically:

      • if a commit version exactly matches the provided timestamp, we return it
      • else, we return the earliest commit version with a timestamp greater than the provided one
      • If the provided timestamp is larger than the timestamp of any committed version, we throw an error.
      Parameters:
      millisSinceEpochUTC - the number of milliseconds since midnight, January 1, 1970 UTC
      catalogCommits - parsed log Deltas to use (must be sorted and contiguous)
      Returns:
      latest commit that happened at or before timestamp.
      Throws:
      KernelException - if the timestamp is more than the timestamp of any committed version

    • getVersionBeforeOrAtTimestamp

      public static long getVersionBeforeOrAtTimestamp(Engine engine, Path logPath, long millisSinceEpochUTC, SnapshotImpl latestSnapshot, List<ParsedCatalogCommitData> catalogCommits)
      Returns the latest version that was committed before or at millisSinceEpochUTC. If no version exists, throws a KernelException

      Specifically:

      • if a commit version exactly matches the provided timestamp, we return it
      • else, we return the latest commit version with a timestamp less than the provided one
      • If the provided timestamp is less than the timestamp of any committed version, we throw an error.
      Parameters:
      millisSinceEpochUTC - the number of milliseconds since midnight, January 1, 1970 UTC
      catalogCommits - parsed log Deltas to use (must be sorted and contiguous)
      Returns:
      latest commit that happened before or at timestamp.
      Throws:
      KernelException - if the timestamp is less than the timestamp of any committed version

    • getActiveCommitAtTimestamp

      public static DeltaHistoryManager.Commit getActiveCommitAtTimestamp(Engine engine, SnapshotImpl latestSnapshot, Path logPath, long timestamp, boolean mustBeRecreatable, boolean canReturnLastCommit, boolean canReturnEarliestCommit, List<ParsedCatalogCommitData> catalogCommits) throws TableNotFoundException
      Returns the latest commit that happened at or before timestamp.

      If the timestamp is outside the range of [earliestCommit, latestCommit] then use parameters canReturnLastCommit and canReturnEarliestCommit to control whether an exception is thrown or the corresponding earliest/latest commit is returned.

      Parameters:
      engine - instance of Engine to use
      logPath - the _delta_log path of the table
      timestamp - the timestamp find the version for in milliseconds since the unix epoch
      mustBeRecreatable - whether the state at the returned commit should be recreatable
      canReturnLastCommit - whether we can return the latest version of the table if the provided timestamp is after the latest commit
      canReturnEarliestCommit - whether we can return the earliest version of the table if the provided timestamp is before the earliest commit
      catalogCommits - parsed log Deltas to use (must be sorted and contiguous)
      Throws:
      KernelException - if the provided timestamp is before the earliest commit and canReturnEarliestCommit is false
      KernelException - if the provided timestamp is after the latest commit and canReturnLastCommit is false
      TableNotFoundException - when there is no Delta table at the given path

    • getEarliestRecreatableCommit

      public static long getEarliestRecreatableCommit(Engine engine, Path logPath, Optional<Long> earliestRatifiedCommitVersion) throws TableNotFoundException
      Gets the earliest commit that we can recreate. Note that this version isn't guaranteed to exist when performing an action as a concurrent operation can delete the file during cleanup. This value must be used as a lower bound.

      We search for the earliest checkpoint we have, or whether we have the 0th delta file. This method assumes that the commits are contiguous.

      Throws:
      TableNotFoundException

    • getEarliestDeltaFile

      public static long getEarliestDeltaFile(Engine engine, Path logPath, Optional<Long> earliestRatifiedCommitVersion) throws TableNotFoundException
      Get the earliest commit available for this table. Note that this version isn't guaranteed to exist when performing an action as a concurrent operation can delete the file during cleanup. This value must be used as a lower bound.
      Throws:
      TableNotFoundException