All Classes and Interfaces
Class
Description
This class takes as input a list of delta files (.json, .checkpoint.parquet) and produces an
iterator of (ColumnarBatch, isFromCheckpoint) tuples, where the schema of the ColumnarBatch
semantically represents actions (or, a subset of action fields) parsed from the Delta Log.
Internal wrapper class holding information needed to perform log replay.
This class takes an iterator of (
ColumnarBatch, isFromCheckpoint), where the columnar
data inside the columnar batch represents has top level columns "add" and "remove", and produces
an iterator of FilteredColumnarBatch with only the "add" column and with a selection
vector indicating which AddFiles are still active in the table (have not been tombstoned).Metadata about
cdc action in the Delta Log.Delta log action representing an `AddFile`
Predicate which always evaluates to
false.Predicate which always evaluates to
true.AND expressionRepresent
array data typeAbstraction to represent a single array value in a
ColumnVector.This implements Base85 using the 4 byte block aligned encoding and character set from Z85.
Base class for all primitive types
DataType.The data type representing
byte[] values.Data type representing
boolean type values.The data type representing
byte type values.A map that is case-insensitive in its keys.
Committer sub-interface for catalog-managed tables.Thrown when trying to create a checkpoint at version
v, but there already exists a
checkpoint at version v.Class to load and write the
CheckpointMetaData from `_last_checkpoint` file.Write a new checkpoint at the version committed by the txn.
Metadata about Delta checkpoint.
Action representing a checkpointMetadata action in a top-level V2 checkpoint file
A post-commit hook that writes a new checksum file at the version committed by the transaction.
Utility method to load protocol and metadata from the Delta log checksum files.
A post-commit hook that writes a new checksum file at the version committed by the transaction.
Utility methods for computing and writing checksums for Delta tables.
Writers for writing checksum files from a snapshot
A
Row implementation that wraps a set of child vectors for a specific rowId.An interface to represent clocks, so that they can be mocked out in unit tests.
Extend the Java
Iterable interface to provide a way to close the iterator.Closeable extension of
IteratorRepresents the result of applying the filter condition in the
CloseableIterator.breakableFilter(Function) method of a CloseableIterator.Represents the metadata domain for clustering.
Identifies collation for string type.
An expression type that refers to a column (case-sensitive) in the input.
Represents zero or more rows of records with same schema type.
Row abstraction around a columnar batch and a particular row within the columnar batch.
Utilities class for TableFeature "allowColumnDefaults".
Utilities related to the column mapping feature.
Represents zero or more values of a single column.
Represents all actions from a single commit version in a table.
Implementation of
CommitActions.Delta log action representing a commit information action.
Contains all information (excluding the iterator of finalized actions) required to commit changes
to a Delta table.
Represents the different types of commits based on filesystem-managed and catalog-managed state
transitions.
Represents a range of contiguous commits in a Delta Lake table with a defined start and end
version.
A builder for creating
CommitRange instances that define a contiguous range of commits in
a Delta Lake table.Defines a boundary (start or end) of a commit range in a Delta Lake table.
An implementation of
CommitRangeBuilder.Implementation of
CommitRange.Exception thrown when Kernel cannot find any commit files in the requested version range.
Response container for the result of a commit operation.
Exception thrown when the Delta transaction commit system cannot determine whether a previous
commit attempt succeeded or failed, making it unsafe to continue with automatic retries.
Interface for committing changes to Delta tables, supporting both filesystem-managed and
catalog-managed tables.
Thrown when concurrent transaction both attempt to update the table with same transaction
identifier set through
TransactionBuilder.withTransactionId(Engine, String, long)
(String)}.Thrown when a concurrent transaction has written data after the current transaction has started.
Class containing the conflict resolution logic when writing to a Delta table.
Class containing the rebase state from winning transactions that the current transaction needs
to rebase against before attempting the commit.
A long counter that uses
AtomicLong to count events.Replays a history of actions from the transaction log to reconstruct the checkpoint state of the
table.
Builder for creating a
Transaction to create a new Delta table.Encapsulates statistics for a data file in a Delta Lake table and provides methods to serialize
those stats to JSON with basic physical-type validation.
Extends
FileStatus to include additional details such as column level statistics of the
data file in the Delta Lake table.Specification for the data layout of a Delta table, including partitioning and clustering
configurations.
A
Predicate with a set of columns referenced by the expression.Base class for all data types.
Serialize and deserialize Delta data types
DataType to JSON and from JSON class based on
the serialization
rules outlined in the Delta Protocol.Contains the context for writing data to Delta table.
Implements the
DataWriteContext interface.A date type, supporting "0001-01-01" through "9999-12-31".
The data type representing
java.math.BigDecimal values.This wraps an existing
Row and allows overriding values for some particular ordinals.Information about a deletion vector attached to a file action.
Bitmap for a Deletion Vector, implemented as a thin wrapper around a Deletion Vector Descriptor.
Utility methods regarding deletion vectors.
Contains methods to create user-facing Delta exceptions.
Contains methods to create developer-facing exceptions.
Exposes APIs to read the raw actions within the *commit files* of the _delta_log.
Represents a Delta action.
Internal wrapper class holding information needed to perform log replay.
Defines the common fields that are shared by reports for Delta operations
Basic POJO implementation of
DeltaOperationReportUtility class for creating Delta directories based on commit version and protocol features.
Thrown when attempting to remove a domain metadata that does not exist in the read snapshot.
Delta log action representing an `DomainMetadata` action
The data type representing
double type values.Interface encapsulating all clients needed by the Delta Kernel in order to read the Delta table.
APIs that are meant to evolve towards becoming stable APIs, but are not stable APIs yet.
APIs that are still under active development and are expected to change.
Base interface for all Kernel expressions.
Interface for implementing an
Expression evaluator.Provides expression evaluation capability to Delta Kernel.
Defines behavior for
TableFeature that can be automatically enabled via a change in a
table's metadata, e.g., through setting particular values of certain feature-specific table
properties.The metadata for a given
StructField.Builder class for
FieldMetadata.Represents a request to read a range of bytes from a given file.
The result of reading a batch of data in a file.
A histogram that tracks file size distributions and their counts.
Stores the file size histogram information to track file size distribution and their counts.
Class for encapsulating metadata about a file in Delta Lake table.
Provides file system related functionalities to Delta Kernel.
Represents a filtered version of
ColumnarBatch.The data type representing
float type values.Utilities to convert Iceberg add/removes to Delta Kernel add/removes
A generic implementation of
ColumnVector that wraps a list of values.Exposes a given map of values as a
RowThe data type representing geography values.
The data type representing geometry values.
Utility for parsing and formatting WKT POINT strings for geometry/geography stats.
Contains interfaces and common utility classes for defining the iceberg conversion compatibility
checks and metadata updates.
Defines the input context for the metadata validator and updater.
Utility methods for validation and compatibility checks for Iceberg V2.
Utility methods for validation and compatibility checks for Iceberg V3.
Utility class that enforces dependencies of UNIVERSAL_FORMAT_* options.
Performs the validations and updates necessary to support the table feature IcebergWriterCompatV1
when it is enabled by the table property "delta.enableIcebergWriterCompatV1".
IN expressionThe data type representing
integer type values.Utilities to extract information out of the scan file rows returned by
Scan.getScanFiles(Engine).Copy of `org/apache/spark/sql/catalyst/util/SparkIntervalUtils.scala` from Apache Spark.
Thrown when an illegal value is specified for a table property.
Thrown when an invalid table is encountered; the table's log and/or checkpoint files are in an
invalid state.
Provides JSON handling functionality to Delta Kernel.
Abstract class representing a metadata domain, whose configuration string is a JSON serialization
of a domain object.
Throws when the
Engine encountered an error while executing an operation.Thrown when Kernel cannot execute the requested operation due to the operation being invalid or
unsupported.
A literal value.
A post-commit hook that performs inline log compaction.
Utility for writing out log compactions.
Replays a history of actions, resolving them to produce the current state of the table.
The data type representing
long type values.A clock whose time can be manually set and modified.
Data type representing a
map type.Abstraction to represent a single map value in a
ColumnVector.A collection of helper methods for working with materialized row tracking columns.
Thrown when the metadata of the Delta table has changed between the time of transaction start and
the time of commit.
Enumeration of metadata columns recognized by Delta Kernel.
Interface containing the metrics for a given operation.
Interface for reporting metrics for operations to a Delta table
Provides Jackson ObjectMapper configuration for serializing
MetricsReport typesAn operation that can be performed on a Delta table.
OR expressionPage Token Class for Pagination Support
Extension of
Scan that supports pagination.An iterator over
FilteredColumnarBatch, each representing a batch of Scan Files in a
paginated scan.Implementation of
PaginatedScanFilesIteratorImplementation of
PaginatedScanPaginationContext carries pagination-related information.Provides Parquet file related functionalities to Delta Kernel.
A catalog commit represents an atomic change to a table.
Abstract checkpoint file that contains a complete snapshot of table state at a specific version.
Version checksum file containing table state information for integrity validation.
Classic checkpoint stored as a single Parquet file.
Base class for Delta types that represent atomic changes to a table.
Log compaction file containing compacted delta entries across a version range.
Abstract representation of any valid log type in the Delta log.
Multi-part checkpoint split across multiple Parquet files for parallel reading.
A published Delta commit file represent an atomic change to a table.
V2 checkpoint with UUID-based naming.
Expression to decode the serialized partition value into partition type value according the
Delta Protocol spec.
Names a file or directory in a FileSystem.
A hook for executing operation after a transaction commit.
Static convenience methods that help a method or constructor check whether it was invoked
correctly (that is, whether its preconditions were met).
Defines predicate scalar expression which is an extension of
ScalarExpression that
evaluates to true, false, or null for each input row.Special interface for evaluating
Predicate on input batch and return a selection vector
containing one value for each row in input batch indicating whether the row has passed the
predicate or not.Thrown when the protocol of the Delta table has changed between the time of transaction start and
the time of commit.
Static utility class for loading Protocol and Metadata from Delta log files.
Result of loading Protocol and Metadata from a LogSegment.
Exception thrown when publishing catalog commits to the Delta log fails.
Metadata required for publishing catalog commits to the Delta log.
Metadata about
remove action in the Delta Log.Builds a
Transaction to replace an existing Delta table.A 64-bit extension of [[RoaringBitmap]] that is optimized for cases that usually fit within a
32-bit bitmap, but may run over by a few bits on occasion.
Represent a single record
An abstract base class for Delta Log actions that are backed by a
Row.A collection of helper methods for working with row tracking.
Represents the metadata domain for row tracking.
Scalar SQL expressions which take zero or more inputs and for each input row generate one output
value.
Represents a scan of a Delta table.
Builder to construct
Scan object.Implementation of
ScanBuilder.Implementation of
ScanStores the metrics for an ongoing scan.
Stores the metrics results for a
ScanReportDefines the metadata and metrics for a Scan
MetricsReportA basic POJO implementation of
ScanReport for creating themEncapsulate the scan state (common info for all scan files) as a
RowUtility class for iterating over schema structures and modifying them.
Interface for manipulating Schema elements.
Container for parent StructField information returned by
SchemaIterable.SchemaElement.getParentStructFieldAndPath().Interface for representing a schema element as part of a traversal.
Utility methods for schema related operations such as validating the schema has no duplicate
columns and the names contain only valid characters.
The selection vector for a columnar batch as a boolean
ColumnVector.Delta log action representing a transaction identifier action.
The data type representing
short type values.Action representing a SidecarFile in a top-level V2 checkpoint file.
Represents a snapshot of a Delta table at a specific version.
Indicates how a checksum file should be written for this Snapshot.
Builder for constructing a
Snapshot instance.An implementation of
SnapshotBuilder.Factory class responsible for creating
Snapshot instances.Implementation of
Snapshot.Stores the metrics for an ongoing snapshot construction.
Stores the metrics results for a
SnapshotReportStores the context for a given Snapshot query.
Defines the metadata and metrics for a snapshot construction
MetricsReportA basic POJO implementation of
SnapshotReport for creating themProvides statistics and metadata about a
Snapshot.Exception thrown when the requested start version is not found in the Delta log.
Provides information and utilities for statistics columns given a table schema.
A predicate that tests whether the bounding box of a geometry column intersects a given query
bounding box.
The data type representing
string type values.Represents a subfield of
StructType with additional properties and metadata.A
Row abstraction for a struct type column vector and a specific rowId.Struct type which contains one or more columns.
Represents the Delta Lake table for a given path.
Thrown when trying to create a Delta table at a location where a Delta table already exists.
Utility class for table changes operations.
Represents the table properties.
Base class for table features.
An interface to indicate a feature is legacy, i.e., released before Table Features.
A base class for all table legacy reader-writer features.
A base class for all table legacy writer-only features.
A base class for all non-legacy table reader-writer features.
An interface to indicate a feature applies to readers and writers.
A base class for all non-legacy table writer features.
Contains utility methods related to the Delta table feature support in protocol.
The entry point for loading and creating Delta tables.
Thrown when there is no Delta table at the given location.
A timer class for measuring the duration of operations in nanoseconds
Times an operation that can throw a specific checked exception type.
A timing sample that carries internal state about the Timer's start position.
The timestamp without time zone type represents a local time in microsecond precision, which is
independent of time zone.
A timestamp type, supporting [0001-01-01T00:00:00.000000Z, 9999-12-31T23:59:59.999999Z] where the
left/right-bound is a date and time of the proleptic Gregorian calendar in UTC+00:00.
Utilities for timestamp conversions that avoid overflow issues.
Represents a transaction to mutate a Delta table.
Builder for creating a
Transaction to mutate a Delta table.Contains the result of a successful transaction commit.
Class for building the Protocol, Metadata, and ResolvedClusteringColumns to commit in a
transaction.
Expectations with respect to the given operation:
Create table: both protocol and metadata will be present
Replace table: both protocol and metadata will be present
Update table: metadata and protocol may or may not be present depending on whether there
should be a metadata or protocol update
Stores the metrics for an ongoing transaction.
Stores the metrics results for a
TransactionReportDefines the metadata and metrics for a transaction
MetricsReportA basic POJO implementation of
TransactionReport for creating themRepresents tuple of objects.
Represents a type change for a field, containing the original and new primitive types.
Utility class for checking type widening compatibility according to the Delta protocol.
Thrown when an unknown configuration key is specified.
Exception thrown when Kernel encounters unsupported protocol versions.
Enum representing the type of Delta protocol version.
Base exception thrown when Kernel encounters unsupported table features.
Builds a
Transaction to update an existing Delta table.Various utility methods to help the connectors work with data objects returned by Kernel
A logical variant type.
Indicates that the visibility of a program element (such as a field, method, or class) is
intentionally wider than necessary for testing purposes.