Index

A B C D E F G H I L M N O P R S T U V W
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form

A

AbstractCommitInfo - Interface in io.delta.storage.commit.actions: Interface for objects that represents the base information for a commit.
AbstractDomainMetadata - Interface in io.delta.storage.commit.actions: Interface for Delta domain-metadata actions.
AbstractMetadata - Interface in io.delta.storage.commit.actions: Interface for metadata actions in Delta.
AbstractProtocol - Interface in io.delta.storage.commit.actions: Interface for protocol actions in Delta.
acquire(Path) - Method in class io.delta.storage.internal.PathLock: Acquire a lock for the path before writing.
APP_VERSIONS_PREFIX - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils
AUTH_PREFIX - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils
AUTH_TYPE_KEY - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Auth config key for the token provider type.
AzureLogStore - Class in io.delta.storage: LogStore implementation for Azure.
AzureLogStore(Configuration) - Constructor for class io.delta.storage.AzureLogStore

B

BACKFILL_LISTING_OFFSET - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient: Offset from current commit version for backfill listing optimization.
backfillToVersion(LogStore, Configuration, TableDescriptor, long, Long) - Method in interface io.delta.storage.commit.CommitCoordinatorClient: API to ask the commit coordinator client to backfill all commits up to version and notify the commit coordinator.
backfillToVersion(LogStore, Configuration, TableDescriptor, long, Long) - Method in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient

C

CatalogTrackedInfo - Class in org.apache.spark.sql.delta.coordinatedcommits
CatalogTrackedInfo(Optional<UniformMetadata>) - Constructor for class org.apache.spark.sql.delta.coordinatedcommits.CatalogTrackedInfo
CatalogTrackedInfo(Optional<UniformMetadata>, List<AbstractDomainMetadata>) - Constructor for class org.apache.spark.sql.delta.coordinatedcommits.CatalogTrackedInfo
close() - Method in interface io.delta.storage.commit.uccommitcoordinator.UCClient: Closes any resources used by this client.
close() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient
close() - Method in class io.delta.storage.commit.uccommitcoordinator.UCTokenBasedRestClient
close() - Method in class io.delta.storage.LineCloseableIterator
CloseableIterator<T> - Interface in io.delta.storage: :: DeveloperApi :: An iterator that may contain resources which should be released after use.
ColumnDef(String, String, String, String, boolean, int) - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCClient.ColumnDef
commit(LogStore, Configuration, TableDescriptor, long, Iterator<String>, UpdatedActions) - Method in interface io.delta.storage.commit.CommitCoordinatorClient: API to commit the given set of actions to the table represented by logPath at the given commitVersion.
commit(LogStore, Configuration, TableDescriptor, long, Iterator<String>, UpdatedActions) - Method in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient
commit(LogStore, Configuration, TableDescriptor, long, Iterator<String>, CatalogTrackedInfo, UpdatedActions) - Method in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient
commit(String, URI, TableIdentifier, Optional<Commit>, Optional<Long>, Optional<AbstractMetadata>, Optional<AbstractMetadata>, Optional<AbstractProtocol>, Optional<AbstractProtocol>, List<AbstractDomainMetadata>, Optional<UniformMetadata>) - Method in interface io.delta.storage.commit.uccommitcoordinator.UCClient: Commits new changes to a Delta table using the Unity Catalog as the Commit Coordinator.
commit(String, URI, TableIdentifier, Optional<Commit>, Optional<Long>, Optional<AbstractMetadata>, Optional<AbstractMetadata>, Optional<AbstractProtocol>, Optional<AbstractProtocol>, List<AbstractDomainMetadata>, Optional<UniformMetadata>) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient
commit(String, URI, TableIdentifier, Optional<Commit>, Optional<Long>, Optional<AbstractMetadata>, Optional<AbstractMetadata>, Optional<AbstractProtocol>, Optional<AbstractProtocol>, List<AbstractDomainMetadata>, Optional<UniformMetadata>) - Method in class io.delta.storage.commit.uccommitcoordinator.UCTokenBasedRestClient
Commit - Class in io.delta.storage.commit: Representation of a commit file
Commit(long, FileStatus, long) - Constructor for class io.delta.storage.commit.Commit
CommitCompletionUnknownException - Exception in io.delta.storage.commit.uccommitcoordinator: Raised when UC cannot determine whether the commit at the requested version already landed: the version is at or below the latest commit, but UC retains no row at it (the row was backfilled and cleaned up), so UC cannot tell whether the version holds this caller's own commit or a different writer's.
CommitCompletionUnknownException(String) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.CommitCompletionUnknownException
CommitCoordinatorClient - Interface in io.delta.storage.commit: The CommitCoordinatorClient is responsible for communicating with the commit coordinator and backfilling commits.
commitDirPath(Path) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Returns path to the directory which holds the unbackfilled commits
CommitFailedException - Exception in io.delta.storage.commit: Exception raised by CommitCoordinatorClient.commit(LogStore, Configuration, TableDescriptor, long, Iterator, UpdatedActions)
CommitFailedException(boolean, boolean, String) - Constructor for exception io.delta.storage.commit.CommitFailedException
CommitFailedException(boolean, boolean, String, Throwable) - Constructor for exception io.delta.storage.commit.CommitFailedException
CommitLimitReachedException - Exception in io.delta.storage.commit.uccommitcoordinator: This exception is thrown by the UC client in case UC has reached the maximum number of commits that it is allowed to track (50 by default).
CommitLimitReachedException(String) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.CommitLimitReachedException
CommitReport(long, long, long, long, Optional<Long>, Optional<Long>, Optional<Long>, Optional<UCDeltaModels.FileSizeHistogram>) - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.CommitReport
CommitResponse - Class in io.delta.storage.commit: Response container for CommitCoordinatorClient.commit(LogStore, Configuration, TableDescriptor, long, Iterator, UpdatedActions).
CommitResponse(Commit) - Constructor for class io.delta.storage.commit.CommitResponse
conf - Variable in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient: Configuration map containing settings for the coordinator client.
CoordinatedCommitsUtils - Class in io.delta.storage.commit
createStagingTable(TableIdentifier) - Method in interface io.delta.storage.commit.uccommitcoordinator.UCDeltaClient: Reserves a staging slot for a new Delta table.
createStagingTable(TableIdentifier) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient
createTable(URI, TableIdentifier, UCDeltaModels.TableType, AbstractMetadata, AbstractProtocol, List<AbstractDomainMetadata>, long, Optional<UniformMetadata>) - Method in interface io.delta.storage.commit.uccommitcoordinator.UCDeltaClient: Finalizes a previously staged Delta table, making it visible in the catalog.
createTable(URI, TableIdentifier, UCDeltaModels.TableType, AbstractMetadata, AbstractProtocol, List<AbstractDomainMetadata>, long, Optional<UniformMetadata>) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient
CRED_SCOPED_FS_ENABLED_KEY - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils
CREDENTIAL_VENDING_ENABLED_KEY - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils
CredentialFetchFailedException - Exception in io.delta.storage.commit.uccommitcoordinator.exceptions: Thrown by UCDeltaClient when credential vending exhausts retries.
CredentialFetchFailedException(String, Throwable, UCDeltaModels.TableInfo) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.exceptions.CredentialFetchFailedException

D

DELTA_REST_API_ENABLED_KEY - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils
deltaFile(Path, long) - Static method in class io.delta.storage.internal.FileNameUtils: Returns the delta (json format) path for a given delta file.
DeltaProtocol() - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol
deltaUniformIceberg() - Method in class org.apache.spark.sql.delta.coordinatedcommits.CatalogTrackedInfo
deltaVersion(Path) - Static method in class io.delta.storage.internal.FileNameUtils: Returns the version for the given delta path.

E

EMPTY - Static variable in class org.apache.spark.sql.delta.coordinatedcommits.CatalogTrackedInfo
equals(Object) - Method in class io.delta.storage.commit.Commit
equals(Object) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol
EXTERNAL - Enum constant in enum io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableType
extractAppVersions(Map<String, String>) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Extracts appVersions.* entries from a flat ucConfig map, stripping the prefix.
extractAuthConfig(Map<String, String>) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Extracts authentication configuration from a flat ucConfig map.
extractUCTableId(TableDescriptor) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient
extractUri(Map<String, String>) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Extracts the required uri value from a flat ucConfig map.

F

FileNameUtils - Class in io.delta.storage.internal: Helper for misc functions relating to file names for delta commits.
FileNameUtils() - Constructor for class io.delta.storage.internal.FileNameUtils
FileSizeHistogram(List<Long>, List<Long>, List<Long>, long) - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.FileSizeHistogram
finalizeCreate(String, String, String, String, List<UCClient.ColumnDef>, AbstractProtocol, Map<String, String>, long, List<AbstractDomainMetadata>) - Method in interface io.delta.storage.commit.uccommitcoordinator.UCClient: Promotes a staging table to a real managed table in Unity Catalog.
finalizeCreate(String, String, String, String, List<UCClient.ColumnDef>, AbstractProtocol, Map<String, String>, long, List<AbstractDomainMetadata>) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient
finalizeCreate(String, String, String, String, List<UCClient.ColumnDef>, AbstractProtocol, Map<String, String>, long, List<AbstractDomainMetadata>) - Method in class io.delta.storage.commit.uccommitcoordinator.UCTokenBasedRestClient

G

GCSLogStore - Class in io.delta.storage: The LogStore implementation for GCS, which uses gcs-connector to provide the necessary atomic and durability guarantees: Atomic Visibility: Read/read-after-metadata-update/delete are strongly consistent for GCS. Consistent Listing: GCS guarantees strong consistency for both object and bucket listing operations.
GCSLogStore(Configuration) - Constructor for class io.delta.storage.GCSLogStore
generateUnbackfilledDeltaFilePath(Path, long) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Creates a new unbackfilled delta file path for the given commit version.
getBackfilledDeltaFilePath(Path, Long) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Returns the path to the backfilled delta file for the given commit version.
getBaseConvertedDeltaVersion() - Method in class io.delta.storage.commit.uniform.IcebergMetadata: Returns the Delta version from which this conversion incrementally started, or empty if this was a full (non-incremental) conversion.
getCommit() - Method in class io.delta.storage.commit.CommitResponse
getCommitInfo() - Method in class io.delta.storage.commit.UpdatedActions
getCommits() - Method in class io.delta.storage.commit.GetCommitsResponse
getCommits(TableDescriptor, Long, Long) - Method in interface io.delta.storage.commit.CommitCoordinatorClient: API to get the unbackfilled commits for the table represented by the given logPath.
getCommits(TableDescriptor, Long, Long) - Method in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient
getCommits(String, URI, TableIdentifier, Optional<Long>, Optional<Long>) - Method in interface io.delta.storage.commit.uccommitcoordinator.UCClient: Retrieves the unbackfilled commits for a Delta table within a specified version range.
getCommits(String, URI, TableIdentifier, Optional<Long>, Optional<Long>) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient
getCommits(String, URI, TableIdentifier, Optional<Long>, Optional<Long>) - Method in class io.delta.storage.commit.uccommitcoordinator.UCTokenBasedRestClient
GetCommitsResponse - Class in io.delta.storage.commit: Response container for CommitCoordinatorClient.getCommits(TableDescriptor, Long, Long).
GetCommitsResponse(List<Commit>, long) - Constructor for class io.delta.storage.commit.GetCommitsResponse
getCommitTimestamp() - Method in interface io.delta.storage.commit.actions.AbstractCommitInfo: Get the timestamp of the commit as millis after the epoch.
getCommitTimestamp() - Method in class io.delta.storage.commit.Commit
getCommitVersion() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.FileSizeHistogram
getConfiguration() - Method in interface io.delta.storage.commit.actions.AbstractDomainMetadata: Domain-specific configuration, serialized as JSON.
getConfiguration() - Method in interface io.delta.storage.commit.actions.AbstractMetadata: The table properties defined on the table.
getConflict() - Method in exception io.delta.storage.commit.CommitFailedException
getConvertedDeltaTimestamp() - Method in class io.delta.storage.commit.uniform.IcebergMetadata: Returns the timestamp when the conversion occurred.
getConvertedDeltaVersion() - Method in class io.delta.storage.commit.uniform.IcebergMetadata: Returns the Delta version that was converted to Iceberg.
getCoordinatorConf(AbstractMetadata) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Get the coordinated commits owner configuration from the provided abstract metadata.
getCoordinatorName(AbstractMetadata) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Retrieves the coordinator name from the provided abstract metadata.
getCreatedTime() - Method in interface io.delta.storage.commit.actions.AbstractMetadata: Timestamp for the creation of this metadata.
getDescription() - Method in interface io.delta.storage.commit.actions.AbstractMetadata: User-specified table description.
getDomain() - Method in interface io.delta.storage.commit.actions.AbstractDomainMetadata: Domain identifier (e.g.
getFileCounts() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.FileSizeHistogram
getFileSizeHistogram() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.CommitReport
getFileStatus() - Method in class io.delta.storage.commit.Commit
getFormatOptions() - Method in interface io.delta.storage.commit.actions.AbstractMetadata: The format options
getIcebergMetadata() - Method in class io.delta.storage.commit.uniform.UniformMetadata: Returns the Iceberg metadata if Iceberg conversion is enabled.
getId() - Method in interface io.delta.storage.commit.actions.AbstractMetadata: A unique table identifier.
getLastKnownBackfilledVersion(long, Configuration, LogStore, TableDescriptor) - Method in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient: Find the last known backfilled version by doing a listing of the last UCCommitCoordinatorClient.BACKFILL_LISTING_OFFSET commits.
getLatestTableVersion() - Method in class io.delta.storage.commit.GetCommitsResponse
getLocation() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.StagingTableInfo
getLocation() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableInfo
getLogPath() - Method in class io.delta.storage.commit.TableDescriptor
getMetadata() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableInfo
getMetadataLocation() - Method in class io.delta.storage.commit.uniform.IcebergMetadata: Returns the Iceberg metadata file location.
getMetastoreId() - Method in interface io.delta.storage.commit.uccommitcoordinator.UCClient: Retrieves the metastore ID associated with this Unity Catalog instance.
getMetastoreId() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient
getMetastoreId() - Method in class io.delta.storage.commit.uccommitcoordinator.UCTokenBasedRestClient
getMinReaderVersion() - Method in interface io.delta.storage.commit.actions.AbstractProtocol: The minimum reader version required to read the table.
getMinReaderVersion() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol
getMinWriterVersion() - Method in interface io.delta.storage.commit.actions.AbstractProtocol: The minimum writer version required to read the table.
getMinWriterVersion() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol
getName() - Method in interface io.delta.storage.commit.actions.AbstractMetadata: User-specified table identifier.
getName() - Method in class io.delta.storage.commit.TableIdentifier: Returns the name of the table.
getName() - Method in class io.delta.storage.commit.uccommitcoordinator.UCClient.ColumnDef
getNamespace() - Method in class io.delta.storage.commit.TableIdentifier: Returns the namespace of the table.
getNewMetadata() - Method in class io.delta.storage.commit.UpdatedActions
getNewProtocol() - Method in class io.delta.storage.commit.UpdatedActions
getNumBytesAdded() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.CommitReport
getNumBytesRemoved() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.CommitReport
getNumFilesAdded() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.CommitReport
getNumFilesRemoved() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.CommitReport
getNumRowsInserted() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.CommitReport
getNumRowsRemoved() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.CommitReport
getNumRowsUpdated() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.CommitReport
getOldMetadata() - Method in class io.delta.storage.commit.UpdatedActions
getOldProtocol() - Method in class io.delta.storage.commit.UpdatedActions
getPartitionColumns() - Method in interface io.delta.storage.commit.actions.AbstractMetadata: List of partition columns.
getPosition() - Method in class io.delta.storage.commit.uccommitcoordinator.UCClient.ColumnDef
getProvider() - Method in interface io.delta.storage.commit.actions.AbstractMetadata: The table provider format.
getReaderFeatures() - Method in interface io.delta.storage.commit.actions.AbstractProtocol: The reader features that need to be supported to read the table.
getReaderFeatures() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol
getRequiredProperties() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.StagingTableInfo
getRequiredProtocol() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.StagingTableInfo
getRetryable() - Method in exception io.delta.storage.commit.CommitFailedException
getSchemaString() - Method in interface io.delta.storage.commit.actions.AbstractMetadata: The table schema in string representation.
getSortedBinBoundaries() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.FileSizeHistogram
getStorageProperties() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.StagingTableInfo: Hadoop-style storage options (e.g.
getStorageProperties() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableInfo: Hadoop-style storage options (e.g.
getSuggestedProperties() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.StagingTableInfo
getSuggestedProtocol() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.StagingTableInfo
getTableConf() - Method in class io.delta.storage.commit.TableDescriptor
getTableConf(AbstractMetadata) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Get the coordinated commits table configuration from the provided abstract metadata.
getTableId() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.StagingTableInfo
getTableId() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableInfo: UC's table_uuid; distinct from AbstractMetadata.getId() (the Delta id).
getTableIdentifier() - Method in class io.delta.storage.commit.TableDescriptor
getTableInfoWithoutCredentials() - Method in exception io.delta.storage.commit.uccommitcoordinator.exceptions.CredentialFetchFailedException
getTablePath(Path) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Get the table path from the provided log path.
getTableType() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.StagingTableInfo
getTableType() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableInfo
getTotalBytes() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.FileSizeHistogram
getTypeJson() - Method in class io.delta.storage.commit.uccommitcoordinator.UCClient.ColumnDef
getTypeName() - Method in class io.delta.storage.commit.uccommitcoordinator.UCClient.ColumnDef
getTypeText() - Method in class io.delta.storage.commit.uccommitcoordinator.UCClient.ColumnDef
getUnbackfilledDeltaFile(Path, long, Optional<String>) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Returns the un-backfilled uuid formatted delta (json format) path for a given version.
getUniformMetadata() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableInfo: UniForm conversion metadata, or empty if the table has no UniForm enabled.
getUniformMetadata() - Method in class io.delta.storage.commit.UCDeltaGetCommitsResponse
getVersion() - Method in class io.delta.storage.commit.Commit
getWriterFeatures() - Method in interface io.delta.storage.commit.actions.AbstractProtocol: The writer features that need to be supported to write the table.
getWriterFeatures() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol

H

HadoopFileSystemLogStore - Class in io.delta.storage: Default implementation of LogStore for Hadoop FileSystem implementations.
HadoopFileSystemLogStore(Configuration) - Constructor for class io.delta.storage.HadoopFileSystemLogStore
hasAuthConfig(Map<String, String>) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Returns true if the ucConfig contains any authentication configuration (auth.* keys or legacy token key).
hashCode() - Method in class io.delta.storage.commit.Commit
hashCode() - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol
hasNext() - Method in class io.delta.storage.LineCloseableIterator
HDFSLogStore - Class in io.delta.storage: The LogStore implementation for HDFS, which uses Hadoop FileContext API's to provide the necessary atomic and durability guarantees: Atomic visibility of files: `FileContext.rename` is used write files which is atomic for HDFS. Consistent file listing: HDFS file listing is consistent.
HDFSLogStore(Configuration) - Constructor for class io.delta.storage.HDFSLogStore

I

IcebergMetadata - Class in io.delta.storage.commit.uniform: Metadata for Delta Uniform Iceberg conversion.
IcebergMetadata(String, long, String) - Constructor for class io.delta.storage.commit.uniform.IcebergMetadata: Constructs IcebergMetadata without a base converted delta version (full conversion).
IcebergMetadata(String, long, String, Optional<Long>) - Constructor for class io.delta.storage.commit.uniform.IcebergMetadata: Constructs IcebergMetadata with the specified conversion details.
incorrectLogStoreImplementationException(Throwable) - Static method in class io.delta.storage.internal.LogStoreErrors
initHadoopConf() - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Hadoop configuration that should only be used during initialization of LogStore.
InvalidTargetTableException - Exception in io.delta.storage.commit.uccommitcoordinator: This exception is thrown by the UC client in case a commit attempt tried to add a UUID-based commit to the wrong table.
InvalidTargetTableException(String) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.InvalidTargetTableException
io.delta.storage - package io.delta.storage
io.delta.storage.commit - package io.delta.storage.commit
io.delta.storage.commit.actions - package io.delta.storage.commit.actions
io.delta.storage.commit.uccommitcoordinator - package io.delta.storage.commit.uccommitcoordinator
io.delta.storage.commit.uccommitcoordinator.exceptions - package io.delta.storage.commit.uccommitcoordinator.exceptions
io.delta.storage.commit.uniform - package io.delta.storage.commit.uniform
io.delta.storage.internal - package io.delta.storage.internal
isCoordinatedCommitsToFSConversion(Long, UpdatedActions) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Returns true if the commit is a coordinated commits to filesystem conversion.
isCredentialRenewalEnabled(Map<String, String>) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Returns whether credential renewal is enabled.
isCredentialScopedFsEnabled(Map<String, String>) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Returns whether credential-scoped filesystem access is enabled.
isCredentialVendingEnabled(Map<String, String>) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Returns whether table operations should vend temporary storage credentials.
isDeltaFile(Path) - Static method in class io.delta.storage.internal.FileNameUtils: Returns true if the given path is a delta file, else false.
isDeltaRestApiEnabled(Map<String, String>) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Returns whether the Delta REST API client should be used.
isFatal(Throwable) - Static method in class io.delta.storage.internal.LogStoreErrors: Returns true if the provided Throwable is to be considered fatal, or false if it is to be considered non-fatal
isNonFatal(Throwable) - Static method in class io.delta.storage.internal.LogStoreErrors: Returns true if the provided Throwable is to be considered non-fatal, or false if it is to be considered fatal
isNullable() - Method in class io.delta.storage.commit.uccommitcoordinator.UCClient.ColumnDef
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.AzureLogStore
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.GCSLogStore
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.HDFSLogStore
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.LocalLogStore
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Whether a partial write is visible for the underlying file system of `path`.
isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.S3SingleDriverLogStore
isRemoved() - Method in interface io.delta.storage.commit.actions.AbstractDomainMetadata: true when this entry is a tombstone removing a previously-set domain.

L

LEGACY_TOKEN_KEY - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Legacy top-level token key (without auth. prefix).
LineCloseableIterator - Class in io.delta.storage: Turn a Reader to CloseableIterator which can be read on demand.
LineCloseableIterator(Reader) - Constructor for class io.delta.storage.LineCloseableIterator
listFrom(Path, Configuration) - Method in class io.delta.storage.HadoopFileSystemLogStore
listFrom(Path, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: List the paths in the same directory that are lexicographically greater or equal to (UTF-8 sorting) the given `path`.
listFrom(Path, Configuration) - Method in class io.delta.storage.S3SingleDriverLogStore
loadTable(TableIdentifier) - Method in interface io.delta.storage.commit.uccommitcoordinator.UCDeltaClient: Loads a table's metadata from Unity Catalog.
loadTable(TableIdentifier) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient
LocalLogStore - Class in io.delta.storage: Default LogStore implementation (should be used for testing only!).
LocalLogStore(Configuration) - Constructor for class io.delta.storage.LocalLogStore
logDirPath(Path) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Returns path to the directory which holds the delta log
LogStore - Class in io.delta.storage: :: DeveloperApi ::
LogStore(Configuration) - Constructor for class io.delta.storage.LogStore
LogStoreErrors - Class in io.delta.storage.internal
LogStoreErrors() - Constructor for class io.delta.storage.internal.LogStoreErrors

M

MANAGED - Enum constant in enum io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableType
minReaderVersion(int) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol
minWriterVersion(int) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol

N

next() - Method in class io.delta.storage.LineCloseableIterator
NO_ABSTRACT_FILE_SYSTEM_EXCEPTION_MESSAGE - Static variable in class io.delta.storage.HDFSLogStore
NoSuchTableException - Exception in io.delta.storage.commit.uccommitcoordinator.exceptions: Thrown by UCDeltaClient operations when the catalog reports that the requested table does not exist (HTTP 404).
NoSuchTableException(String) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.exceptions.NoSuchTableException
NoSuchTableException(String, Throwable) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.exceptions.NoSuchTableException

O

org.apache.spark.sql.delta.coordinatedcommits - package org.apache.spark.sql.delta.coordinatedcommits

P

parseBoolean(Map<String, String>, String, boolean) - Static method in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Parses a boolean config value with a default fallback.
PathLock - Class in io.delta.storage.internal: A lock that provides per-file-path `acquire` and `release` semantics.
PathLock() - Constructor for class io.delta.storage.internal.PathLock

R

read(Path, Configuration) - Method in class io.delta.storage.HadoopFileSystemLogStore
read(Path, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Load the given file and return an `Iterator` of lines, with line breaks removed from each line.
readerFeatures(Collection<String>) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol
registerTable(Path, Optional<TableIdentifier>, long, AbstractMetadata, AbstractProtocol) - Method in interface io.delta.storage.commit.CommitCoordinatorClient: API to register the table represented by the given `logPath` at the provided currentTableVersion with the commit coordinator this commit coordinator client represents.
registerTable(Path, Optional<TableIdentifier>, long, AbstractMetadata, AbstractProtocol) - Method in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient: For UC, table registration is a no-op because we already contacted UC during table creation and that already obtained the necessary table config and added it to the metadata (this is for performance reasons and ease of use).
release(Path) - Method in class io.delta.storage.internal.PathLock: Release the lock for the path after writing.
RENEW_CREDENTIAL_ENABLED_KEY - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils
reportMetrics(String, TableIdentifier, UCDeltaModels.CommitReport) - Method in interface io.delta.storage.commit.uccommitcoordinator.UCDeltaClient: Reports post-commit telemetry for a table (file/row counts, byte counts, file-size histogram) to the catalog.
reportMetrics(String, TableIdentifier, UCDeltaModels.CommitReport) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient
resolvePathOnPhysicalStorage(Path, Configuration) - Method in class io.delta.storage.HadoopFileSystemLogStore
resolvePathOnPhysicalStorage(Path, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Resolve the fully qualified path for the given `path`.
runInNewThread(String, boolean, Callable<T>) - Static method in class io.delta.storage.internal.ThreadUtils: Based on Apache Spark's ThreadUtils.runInNewThread Run a piece of code in a new thread and return the result.

S

s3ListFromArray(FileSystem, Path, Path) - Static method in class io.delta.storage.internal.S3LogStoreUtil: Uses the S3ListRequest.v2 interface with the startAfter parameter to only list files which are lexicographically greater than resolvedPath.
S3LogStoreUtil - Class in io.delta.storage.internal: Static utility methods for the S3SingleDriverLogStore.
S3SingleDriverLogStore - Class in io.delta.storage: Single JVM LogStore implementation for S3.
S3SingleDriverLogStore(Configuration) - Constructor for class io.delta.storage.S3SingleDriverLogStore
semanticEquals(CommitCoordinatorClient) - Method in interface io.delta.storage.commit.CommitCoordinatorClient: Determines whether this CommitCoordinatorClient is semantically equal to another CommitCoordinatorClient.
semanticEquals(CommitCoordinatorClient) - Method in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient
sortCommitsByVersion() - Method in class io.delta.storage.commit.GetCommitsResponse: Returns a new response with commits sorted by version.
sortCommitsByVersion() - Method in class io.delta.storage.commit.UCDeltaGetCommitsResponse
StagingTableInfo(UUID, UCDeltaModels.TableType, String, AbstractProtocol, AbstractProtocol, Map<String, String>, Map<String, String>, Map<String, String>) - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.StagingTableInfo
STATIC_AUTH_TYPE - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils: Static token provider type value for legacy token configs.

T

TableDescriptor - Class in io.delta.storage.commit: Container for all the info to uniquely identify the table
TableDescriptor(Path, Optional<TableIdentifier>, Map<String, String>) - Constructor for class io.delta.storage.commit.TableDescriptor
TableIdentifier - Class in io.delta.storage.commit: Identifier for a table.
TableIdentifier(String[], String) - Constructor for class io.delta.storage.commit.TableIdentifier
TableIdentifier(String, String...) - Constructor for class io.delta.storage.commit.TableIdentifier
TableInfo(UUID, UCDeltaModels.TableType, String, AbstractMetadata, Map<String, String>, Optional<UniformMetadata>) - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableInfo
ThreadUtils - Class in io.delta.storage.internal
ThreadUtils() - Constructor for class io.delta.storage.internal.ThreadUtils
transactionDomainMetadata() - Method in class org.apache.spark.sql.delta.coordinatedcommits.CatalogTrackedInfo

U

UC_ATTEMPT_FULL_BACKFILL - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UC_BACKFILL_DOES_NOT_EXIST - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UC_BACKFILL_FAILED - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UC_BACKFILL_FALLBACK_TO_SYNC - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UC_BACKFILL_TO_VERSION - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UC_BACKFILL_VALIDATION_FALLBACK_TO_SYNC - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UC_COMMIT_STATS - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UC_FULL_BACKFILL_ATTEMPT_FAILED - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UC_LAST_KNOWN_BACKFILLED_VERSION_NOT_FOUND - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UC_METASTORE_ID_KEY - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient: Key for identifying Unity Catalog metastore ID in `delta.coordinatedCommits.commitCoordinatorConf{-preview}`.
UC_TABLE_ID_KEY - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient: Key for identifying Unity Catalog table ID in `delta.coordinatedCommits.tableConf{-preview}`.
UC_TABLE_ID_KEY_OLD - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient
ucClient - Variable in class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient: Unity Catalog client instance for interacting with UC services.
UCClient - Interface in io.delta.storage.commit.uccommitcoordinator: Interface for interacting with the Unity Catalog.
UCClient.ColumnDef - Class in io.delta.storage.commit.uccommitcoordinator: Column definition for registering table schema with Unity Catalog.
UCCommitCoordinatorClient - Class in io.delta.storage.commit.uccommitcoordinator: A commit coordinator client that uses unity-catalog as the commit coordinator.
UCCommitCoordinatorClient(Map<String, String>, UCClient) - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorClient
UCCommitCoordinatorException - Exception in io.delta.storage.commit.uccommitcoordinator: Base class for all exceptions thrown by the UC client from coordinated commits-related APIs.
UCCommitCoordinatorException(String) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.UCCommitCoordinatorException
UCConfigUtils - Class in io.delta.storage.commit.uccommitcoordinator: Shared utility methods for parsing flat UC configuration maps (e.g.
UCCoordinatedCommitsUsageLogs - Class in io.delta.storage.commit.uccommitcoordinator: Class containing usage logs emitted by Coordinated Commits.
UCCoordinatedCommitsUsageLogs() - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCCoordinatedCommitsUsageLogs
UCDeltaClient - Interface in io.delta.storage.commit.uccommitcoordinator: Extends UCClient with Delta table lifecycle operations backed by the UC Delta API (load, stage, and create tables).
UCDeltaGetCommitsResponse - Class in io.delta.storage.commit: Extended response for CommitCoordinatorClient.getCommits(io.delta.storage.commit.TableDescriptor, java.lang.Long, java.lang.Long) implementations backed by the Delta REST loadTable API.
UCDeltaGetCommitsResponse(List<Commit>, long, Optional<UniformMetadata>) - Constructor for class io.delta.storage.commit.UCDeltaGetCommitsResponse
UCDeltaModels - Class in io.delta.storage.commit.uccommitcoordinator: Delta-owned models for the UC Delta API.
UCDeltaModels.CommitReport - Class in io.delta.storage.commit.uccommitcoordinator: Per-commit metrics payload reported to Unity Catalog.
UCDeltaModels.DeltaProtocol - Class in io.delta.storage.commit.uccommitcoordinator
UCDeltaModels.FileSizeHistogram - Class in io.delta.storage.commit.uccommitcoordinator: Post-commit file-size distribution snapshot, reported alongside the commit report.
UCDeltaModels.StagingTableInfo - Class in io.delta.storage.commit.uccommitcoordinator
UCDeltaModels.TableInfo - Class in io.delta.storage.commit.uccommitcoordinator: Result of UCDeltaClient.loadTable(io.delta.storage.commit.TableIdentifier) / UCDeltaClient.createTable(java.net.URI, io.delta.storage.commit.TableIdentifier, io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableType, io.delta.storage.commit.actions.AbstractMetadata, io.delta.storage.commit.actions.AbstractProtocol, java.util.List<io.delta.storage.commit.actions.AbstractDomainMetadata>, long, java.util.Optional<io.delta.storage.commit.uniform.UniformMetadata>).
UCDeltaModels.TableType - Enum in io.delta.storage.commit.uccommitcoordinator
UCDeltaTokenBasedRestClient - Class in io.delta.storage.commit.uccommitcoordinator: A REST client implementation of UCDeltaClient that uses the UC Delta API for all table lifecycle and commit coordination operations.
UCDeltaTokenBasedRestClient(Map<String, String>, Supplier<Configuration>) - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient: Creates an instance by parsing all configuration from a flat config map.
UCTokenBasedRestClient - Class in io.delta.storage.commit.uccommitcoordinator: A REST client implementation of UCClient for interacting with Unity Catalog's commit coordination service.
UCTokenBasedRestClient(String, TokenProvider, Map<String, String>) - Constructor for class io.delta.storage.commit.uccommitcoordinator.UCTokenBasedRestClient: Constructs a new UCTokenBasedRestClient with the specified base URI, TokenProvider, and application version information for telemetry.
UniformMetadata - Class in io.delta.storage.commit.uniform: Metadata for Delta Universal Format (UniForm) conversions.
UniformMetadata(IcebergMetadata) - Constructor for class io.delta.storage.commit.uniform.UniformMetadata: Constructs UniformMetadata with Iceberg conversion metadata.
UnsupportedTableFormatException - Exception in io.delta.storage.commit.uccommitcoordinator.exceptions: Thrown when the catalog refuses to serve a non-Delta table.
UnsupportedTableFormatException(String) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.exceptions.UnsupportedTableFormatException
UnsupportedTableFormatException(String, Throwable) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.exceptions.UnsupportedTableFormatException
UpdatedActions - Class in io.delta.storage.commit: A container class to inform the CommitCoordinatorClient about any changes in Protocol/Metadata
UpdatedActions(AbstractCommitInfo, AbstractMetadata, AbstractProtocol, AbstractMetadata, AbstractProtocol) - Constructor for class io.delta.storage.commit.UpdatedActions
UpgradeNotAllowedException - Exception in io.delta.storage.commit.uccommitcoordinator: This exception is thrown by the UC client in case the client attempted an upgrade of a table to a UC managed table but the upgrade is not allowed because the previous commit was a downgrade.
UpgradeNotAllowedException(String) - Constructor for exception io.delta.storage.commit.uccommitcoordinator.UpgradeNotAllowedException
URI_KEY - Static variable in class io.delta.storage.commit.uccommitcoordinator.UCConfigUtils

V

valueOf(String) - Static method in enum io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableType: Returns the enum constant of this type with the specified name.
values() - Static method in enum io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableType: Returns an array containing the constants of this enum type, in the order they are declared.

W

withCommitTimestamp(long) - Method in class io.delta.storage.commit.Commit
withFileStatus(FileStatus) - Method in class io.delta.storage.commit.Commit
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.AzureLogStore
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.GCSLogStore
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.HDFSLogStore
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.LocalLogStore: This write implementation needs to wrap `writeWithRename` with `synchronized` as rename() for RawLocalFileSystem doesn't throw an exception when the target file exists.
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.LogStore: :: DeveloperApi :: Write the given `actions` to the given `path` with or without overwrite as indicated.
write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.S3SingleDriverLogStore
writerFeatures(Collection<String>) - Method in class io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.DeltaProtocol
writeUnbackfilledCommitFile(LogStore, Configuration, String, long, Iterator<String>, String) - Static method in class io.delta.storage.commit.CoordinatedCommitsUtils: Write a UUID-based commit file for the specified version to the table at logPath.

A B C D E F G H I L M N O P R S T U V W
All Classes and Interfaces|All Packages|Constant Field Values|Serialized Form