Package io.delta.kernel.internal

Class SnapshotImpl

Object
io.delta.kernel.internal.SnapshotImpl
All Implemented Interfaces:
Snapshot
public class SnapshotImpl extends Object implements Snapshot
Implementation of Snapshot.

  • Constructor Details

  • Method Details

    • getPath

      public String getPath()
      Specified by:
      getPath in interface Snapshot
      Returns:
      the file system path to this table

    • getVersion

      public long getVersion()
      Specified by:
      getVersion in interface Snapshot
      Returns:
      the version of this snapshot in the Delta table

    • getPartitionColumnNames

      public List<String> getPartitionColumnNames()
      Description copied from interface: Snapshot
      Get the names of the partition columns in the Delta table at this snapshot.

      The partition column names are returned in the order they are defined in the Delta table schema. If the table does not define any partition columns, this method returns an empty list.

      Specified by:
      getPartitionColumnNames in interface Snapshot
      Returns:
      a list of partition column names, or an empty list if the table is not partitioned.

    • getTimestamp

      public long getTimestamp(Engine engine)
      Get the timestamp (in milliseconds since the Unix epoch) of the latest commit in this Snapshot.

      When InCommitTimestampTableFeature is enabled, the timestamp is retrieved from the CommitInfo of the latest commit in this Snapshot, which can result in an IO operation.

      For non-ICT tables, this is the same as the file modification time of the latest commit in this Snapshot.

      Specified by:
      getTimestamp in interface Snapshot
      Parameters:
      engine - the engine to use for IO operations
      Returns:
      the timestamp of the latest commit

    • getSchema

      public StructType getSchema()
      Specified by:
      getSchema in interface Snapshot
      Returns:
      the schema of the Delta table at this snapshot

    • getDomainMetadata

      public Optional<String> getDomainMetadata(String domain)
      Description copied from interface: Snapshot
      Returns the configuration for the provided domain if it exists in the snapshot. Returns empty if the domain is not present in the snapshot.
      Specified by:
      getDomainMetadata in interface Snapshot
      Parameters:
      domain - the domain to look up
      Returns:
      the domain configuration or empty

    • getTableProperties

      public Map<String,String> getTableProperties()
      Description copied from interface: Snapshot
      Get all table properties for the Delta table at this snapshot.
      Specified by:
      getTableProperties in interface Snapshot
      Returns:
      a Map of table properties.

    • getStatistics

      public SnapshotStatistics getStatistics()
      Specified by:
      getStatistics in interface Snapshot
      Returns:
      statistics about this snapshot

    • getScanBuilder

      public ScanBuilder getScanBuilder()
      Specified by:
      getScanBuilder in interface Snapshot
      Returns:
      a scan builder to construct a Scan to read data from this snapshot

    • buildUpdateTableTransaction

      public UpdateTableTransactionBuilder buildUpdateTableTransaction(String engineInfo, Operation operation)
      Specified by:
      buildUpdateTableTransaction in interface Snapshot
      Returns:
      a UpdateTableTransactionBuilder to build an update table transaction

    • publish

      public Snapshot publish(Engine engine) throws PublishFailedException
      Description copied from interface: Snapshot
      Publishes all catalog commits at this table version. Applicable only to catalog-managed tables. This method is a no-op for filesystem-managed tables, if the committer doesn't support publishing, or if there's no catalog commits to publish.

      Publishing copies ratified catalog commits to the Delta log as published Delta files, reducing catalog storage requirements and enabling some table maintenance operations, like checkpointing.

      Specified by:
      publish in interface Snapshot
      Parameters:
      engine - the engine to use for publishing commits
      Returns:
      a new Snapshot reflecting the published state
      Throws:
      PublishFailedException - if the publish operation fails
      See Also:

    • writeChecksum

      public void writeChecksum(Engine engine, Snapshot.ChecksumWriteMode mode) throws IOException
      Description copied from interface: Snapshot
      Writes a checksum file for this snapshot using the specified mode:
      • SIMPLE: Uses pre-computed CRC information already loaded in memory. This is the fastest approach but requires CRC info to be available. Throws IllegalStateException if CRC information is not available.
      • FULL: Computes the necessary CRC information by replaying the delta log since the latest checksum (if present). This may be expensive for large tables when CRC information is not available.

      Use SnapshotStatistics.getChecksumWriteMode() to check if writing is needed and to determine the appropriate mode.

      This method should only be called if a checksum file does not already exist at this version. If it already does, this method is a no-op.

      If a concurrent writer creates the checksum file for this version between when this snapshot was loaded and when this method is called, the method will detect the existing checksum and return successfully without error. This ensures safe concurrent checksum writing.

      Specified by:
      writeChecksum in interface Snapshot
      Parameters:
      engine - the engine to use for writing the checksum file and potentially reading the log
      mode - the mode specifying how to write the checksum (SIMPLE or FULL)
      Throws:
      IOException - if an I/O error occurs during checksum computation or writing
      See Also:

    • writeCheckpoint

      public void writeCheckpoint(Engine engine) throws IOException
      Description copied from interface: Snapshot
      Writes a checkpoint for the current snapshot.
      Specified by:
      writeCheckpoint in interface Snapshot
      Parameters:
      engine - The execution engine used to write the checkpoint and, if necessary, read log entries required to compute it.
      Throws:
      IOException - If an I/O error occurs while computing or writing the checkpoint.

    • buildReplaceTableTransaction

      public ReplaceTableTransactionBuilder buildReplaceTableTransaction(StructType schema, String engineInfo)

    • getCommitter

      public Committer getCommitter()

    • getLogPath

      public Path getLogPath()

    • getDataPath

      public Path getDataPath()

    • wasBuiltAsLatest

      public boolean wasBuiltAsLatest()
      Returns true if this snapshot was built as a "latest" snapshot query (i.e., no time-travel parameters were provided). This is intent-based - it indicates what the user requested, not whether the snapshot is actually the latest version.

    • getProtocol

      public Protocol getProtocol()

    • getSnapshotReport

      public SnapshotReport getSnapshotReport()

    • getPhysicalClusteringColumns

      public Optional<List<Column>> getPhysicalClusteringColumns()
      Returns the clustering columns for this snapshot.
      • Optional.empty() - unclustered table (clustering is not enabled)
      • Optional.of([]) - clustered table with no clustering columns (clustering is enabled)
      • Optional.of([col1, col2]) - clustered table with the given physical clustering columns
      Returns:
      the physical clustering columns in this snapshot

    • getActiveDomainMetadataMap

      public Map<String,DomainMetadata> getActiveDomainMetadataMap()
      Get the domain metadata map from the log replay, which lazily loads and replays a history of domain metadata actions, resolving them to produce the current state of the domain metadata. Only active domain metadata are included in this map.
      Returns:
      A map where the keys are domain names and the values are DomainMetadata objects.

    • getCurrentCrcInfo

      public Optional<CRCInfo> getCurrentCrcInfo()
      Returns the crc info for the current snapshot if the checksum file is read

    • getMetadata

      public Metadata getMetadata()

    • getLogSegment

      public LogSegment getLogSegment()

    • getLazyLogSegment

      @VisibleForTesting public Lazy<LogSegment> getLazyLogSegment()

    • getCreateCheckpointIterator

      public CreateCheckpointIterator getCreateCheckpointIterator(Engine engine)

    • getLatestTransactionVersion

      public Optional<Long> getLatestTransactionVersion(Engine engine, String applicationId)
      Get the latest transaction version for given applicationId. This information comes from the transactions identifiers stored in Delta transaction log. This API is not a public API. For now keep this internal to enable Flink upgrade to use Kernel.
      Parameters:
      applicationId - Identifier of the application that put transaction identifiers in Delta transaction log
      Returns:
      Last transaction version or Optional.empty() if no transaction identifier exists for this application.