Package io.delta.storage.commit.uccommitcoordinator

Class UCDeltaTokenBasedRestClient

Object

io.delta.storage.commit.uccommitcoordinator.UCDeltaTokenBasedRestClient

All Implemented Interfaces:

UCClient, UCDeltaClient, AutoCloseable

public class UCDeltaTokenBasedRestClient
extends Object
implements UCDeltaClient

A REST client implementation of UCDeltaClient that uses the UC Delta API for all table lifecycle and commit coordination operations.

This client uses io.unitycatalog.client.delta.api.DeltaTablesApi for Delta-specific table operations (load, create, update) and MetastoresApi for metastore queries.

See Also:

• UCDeltaClient

Nested Class Summary

Nested classes/interfaces inherited from interface io.delta.storage.commit.uccommitcoordinator.UCClient

UCClient.ColumnDef

Constructor Summary

Constructors

Constructor	Description
UCDeltaTokenBasedRestClient(Map<String,String> ucConfig, Supplier<org.apache.hadoop.conf.Configuration> hadoopConfSupplier)	Creates an instance by parsing all configuration from a flat config map.

Method Summary

Modifier and Type	Method	Description
void	close()	Closes any resources used by this client.
void	commit(String tableId, URI tableUri, TableIdentifier tableIdentifier, Optional<Commit> commit, Optional<Long> lastKnownBackfilledVersion, Optional<AbstractMetadata> oldMetadata, Optional<AbstractMetadata> newMetadata, Optional<AbstractProtocol> oldProtocol, Optional<AbstractProtocol> newProtocol, List<AbstractDomainMetadata> transactionDomainMetadata, Optional<UniformMetadata> uniform)	Commits new changes to a Delta table using the Unity Catalog as the Commit Coordinator.
UCDeltaModels.StagingTableInfo	createStagingTable(TableIdentifier tableIdentifier)	Reserves a staging slot for a new Delta table.
UCDeltaModels.TableInfo	createTable(URI tableUri, TableIdentifier tableIdentifier, UCDeltaModels.TableType tableType, AbstractMetadata metadata, AbstractProtocol protocol, List<AbstractDomainMetadata> domainMetadata, long lastCommitTimestampMs, Optional<UniformMetadata> uniformMetadata)	Finalizes a previously staged Delta table, making it visible in the catalog.
void	finalizeCreate(String tableName, String catalogName, String schemaName, String storageLocation, List<UCClient.ColumnDef> columns, AbstractProtocol protocol, Map<String,String> properties, long lastCommitTimestampMs, List<AbstractDomainMetadata> domainMetadata)	Promotes a staging table to a real managed table in Unity Catalog.
GetCommitsResponse	getCommits(String tableId, URI tableUri, TableIdentifier tableIdentifier, Optional<Long> startVersion, Optional<Long> endVersion)	Retrieves the unbackfilled commits for a Delta table within a specified version range.
String	getMetastoreId()	Retrieves the metastore ID associated with this Unity Catalog instance.
UCDeltaModels.TableInfo	loadTable(TableIdentifier tableIdentifier)	Loads a table's metadata from Unity Catalog.
void	reportMetrics(String tableId, TableIdentifier tableIdentifier, UCDeltaModels.CommitReport report)	Reports post-commit telemetry for a table (file/row counts, byte counts, file-size histogram) to the catalog.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

UCDeltaTokenBasedRestClient

public UCDeltaTokenBasedRestClient(Map<String,String> ucConfig,
 Supplier<org.apache.hadoop.conf.Configuration> hadoopConfSupplier)

Creates an instance by parsing all configuration from a flat config map. Recognised keys:

• uri (required) -- the UC server endpoint.
• auth.* / token (legacy) -- authentication parameters.
• appVersions.* -- caller-supplied version entries.
• credentialVending.enabled -- request temporary storage credentials while loading or creating tables (default true).
• renewCredential.enabled -- enable credential renewal (default true).
• credScopedFs.enabled -- enable credential-scoped FS (default true).

Parameters:

ucConfig - the unified configuration map with all keys.

hadoopConfSupplier - called once per request so engine-level changes are picked up; null defaults to () -> new Configuration().

Method Details

getMetastoreId

public String getMetastoreId()
                      throws IOException

Description copied from interface: UCClient

Retrieves the metastore ID associated with this Unity Catalog instance.

Specified by:

getMetastoreId in interface UCClient

Returns:

A String representing the unique identifier of the metastore

Throws:

IOException - if there's an error in retrieving the metastore ID

commit

public void commit(String tableId,
 URI tableUri,
 TableIdentifier tableIdentifier,
 Optional<Commit> commit,
 Optional<Long> lastKnownBackfilledVersion,
 Optional<AbstractMetadata> oldMetadata,
 Optional<AbstractMetadata> newMetadata,
 Optional<AbstractProtocol> oldProtocol,
 Optional<AbstractProtocol> newProtocol,
 List<AbstractDomainMetadata> transactionDomainMetadata,
 Optional<UniformMetadata> uniform)
            throws IOException,
CommitFailedException,
UCCommitCoordinatorException

Description copied from interface: UCClient

Commits new changes to a Delta table using the Unity Catalog as the Commit Coordinator. This method is responsible for committing changes to a Delta table, including new data, metadata updates, and protocol changes. It interacts with the Unity Catalog to ensure proper coordination and consistency of the commit process. Note: At least one of commit or lastKnownBackfilledVersion must be present.

Specified by:

commit in interface UCClient

Parameters:

tableId - The Unity Catalog table ID (UUID) identifying the target table.

tableUri - The URI of the storage location of the table (e.g., s3://bucket/path/to/table, not the _delta_log path).

tableIdentifier - The three-part table identifier (catalog, schema, table name) for the table in Unity Catalog, or null when the caller has no catalog context. Implementations that require this value must reject null; implementations that don't use it may ignore it.

commit - An Optional containing the Commit object with the changes to be committed. If empty, it indicates that no new data is being added in this commit.

lastKnownBackfilledVersion - An Optional containing the last known backfilled version of the table. This value serves as a hint to UC about the most recent version that has been successfully backfilled. UC can use this information to clean up tracking information for backfilled commits up to this version. If not provided (Optional.empty()), UC will rely on its current state without any additional cleanup hints.

oldMetadata - An Optional containing the existing metadata of the table before the commit. If present, UC can validate the current state of the table.

newMetadata - An Optional containing new metadata to be applied to the table. If present, the table's metadata will be updated atomically with the commit.

oldProtocol - An Optional containing the existing protocol of the table before the commit. If present, UC can validate the current protocol state.

newProtocol - An Optional containing a new protocol version to be applied to the table. If present, the table's protocol will be updated atomically with the commit.

transactionDomainMetadata - The raw domain metadata actions from this transaction. A non-removed action sets domain metadata, and a removed action removes the domain metadata.

uniform - An Optional containing UniForm metadata for Delta Universal Format support. If present, this metadata will be used by UC to manage format conversions (e.g., Iceberg, Hudi).

Throws:

IOException - if there's an error during the commit process, such as network issues.

CommitFailedException - if the commit fails due to conflicts or other logical errors.

UCCommitCoordinatorException - if there's an error specific to the Unity Catalog commit coordination process.

getCommits

public GetCommitsResponse getCommits(String tableId,
 URI tableUri,
 TableIdentifier tableIdentifier,
 Optional<Long> startVersion,
 Optional<Long> endVersion)
                              throws IOException,
UCCommitCoordinatorException

Description copied from interface: UCClient

Retrieves the unbackfilled commits for a Delta table within a specified version range.

Specified by:

getCommits in interface UCClient

Parameters:

tableId - The unique identifier of the Delta table.

tableUri - The URI of the storage location of the table. e.g. s3://bucket/path/to/table (and not s3://bucket/path/to/table/_delta_log). If the tableId exists but the tableUri is different from the one previously registered (e.g., if the table as moved), the request will fail.

tableIdentifier - The three-part table identifier (catalog, schema, table name) for the table in Unity Catalog, or null when the caller has no catalog context. Implementations that require this value must reject null; implementations that don't use it may ignore it.

startVersion - An Optional containing the start version of the range of commits to retrieve.

endVersion - An Optional containing the end version of the range of commits to retrieve.

Returns:

A GetCommitsResponse object containing the unbackfilled commits within the specified version range. If all commits are backfilled, the response will contain an empty list. The response also contains the last known backfilled version of the table. If no commits are ratified via UC, the lastKnownBackfilledVersion will be -1.

Throws:

IOException - if there's an error during the commit process, such as network issues.

UCCommitCoordinatorException - if there's an error specific to the Unity Catalog such as the table not being found.

finalizeCreate

public void finalizeCreate(String tableName,
 String catalogName,
 String schemaName,
 String storageLocation,
 List<UCClient.ColumnDef> columns,
 AbstractProtocol protocol,
 Map<String,String> properties,
 long lastCommitTimestampMs,
 List<AbstractDomainMetadata> domainMetadata)
                    throws CommitFailedException

Description copied from interface: UCClient

Promotes a staging table to a real managed table in Unity Catalog. This is the correct API for version 0 (CREATE); for version 1+ (WRITE), use UCClient.commit(java.lang.String, java.net.URI, io.delta.storage.commit.TableIdentifier, java.util.Optional<io.delta.storage.commit.Commit>, java.util.Optional<java.lang.Long>, java.util.Optional<io.delta.storage.commit.actions.AbstractMetadata>, java.util.Optional<io.delta.storage.commit.actions.AbstractMetadata>, java.util.Optional<io.delta.storage.commit.actions.AbstractProtocol>, java.util.Optional<io.delta.storage.commit.actions.AbstractProtocol>, java.util.List<io.delta.storage.commit.actions.AbstractDomainMetadata>, java.util.Optional<io.delta.storage.commit.uniform.UniformMetadata>) instead.

Specified by:

finalizeCreate in interface UCClient

Parameters:

tableName - table name (relative to parent schema)

catalogName - parent catalog name in Unity Catalog

schemaName - parent schema name in Unity Catalog

storageLocation - the storage root URL for the table

columns - column definitions for the table schema

protocol - the table's protocol (min reader/writer versions and features).

properties - properties to persist in UC. For Delta-Commits clients this is the flattened bag (protocol features, metadata config, timestamp, version, clustering); for Delta-Tables clients this is the metadata.configuration.

lastCommitTimestampMs - the Delta-log timestamp of the version-0 commit.

domainMetadata - the version-0 domain-metadata actions (e.g. clustering, row tracking).

Throws:

CommitFailedException - if there is a network or server error during finalization

close

public void close()
           throws IOException

Description copied from interface: UCClient

Closes any resources used by this client. This method should be called to properly release resources such as network connections (e.g., HTTPClient) when the client is no longer needed. Once this method is called, the client should not be used to perform any operations.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface UCClient

Throws:

IOException - if there's an error while closing resources

loadTable

public UCDeltaModels.TableInfo loadTable(TableIdentifier tableIdentifier)
                                  throws IOException

Description copied from interface: UCDeltaClient

Loads a table's metadata from Unity Catalog.

Specified by:

loadTable in interface UCDeltaClient

Parameters:

tableIdentifier - catalog + schema namespace and table name

Returns:

the table's UCDeltaModels.TableInfo, carrying the catalog-supplied storage location and metadata

Throws:

IOException - on network or API errors

createStagingTable

public UCDeltaModels.StagingTableInfo createStagingTable(TableIdentifier tableIdentifier)
                                                  throws IOException

Description copied from interface: UCDeltaClient

Reserves a staging slot for a new Delta table. The returned response contains the table ID, storage location, and protocol/property requirements that the caller must honor when finalizing the table with UCDeltaClient.createTable(java.net.URI, io.delta.storage.commit.TableIdentifier, io.delta.storage.commit.uccommitcoordinator.UCDeltaModels.TableType, io.delta.storage.commit.actions.AbstractMetadata, io.delta.storage.commit.actions.AbstractProtocol, java.util.List<io.delta.storage.commit.actions.AbstractDomainMetadata>, long, java.util.Optional<io.delta.storage.commit.uniform.UniformMetadata>).

Specified by:

createStagingTable in interface UCDeltaClient

Parameters:

tableIdentifier - catalog + schema namespace and table name

Returns:

a UCDeltaModels.StagingTableInfo with the reserved table details

Throws:

IOException - on network or API errors

createTable

public UCDeltaModels.TableInfo createTable(URI tableUri,
 TableIdentifier tableIdentifier,
 UCDeltaModels.TableType tableType,
 AbstractMetadata metadata,
 AbstractProtocol protocol,
 List<AbstractDomainMetadata> domainMetadata,
 long lastCommitTimestampMs,
 Optional<UniformMetadata> uniformMetadata)
                                    throws IOException

Description copied from interface: UCDeltaClient

Finalizes a previously staged Delta table, making it visible in the catalog. Mirrors the shape of UCClient.commit(java.lang.String, java.net.URI, io.delta.storage.commit.TableIdentifier, java.util.Optional<io.delta.storage.commit.Commit>, java.util.Optional<java.lang.Long>, java.util.Optional<io.delta.storage.commit.actions.AbstractMetadata>, java.util.Optional<io.delta.storage.commit.actions.AbstractMetadata>, java.util.Optional<io.delta.storage.commit.actions.AbstractProtocol>, java.util.Optional<io.delta.storage.commit.actions.AbstractProtocol>, java.util.List<io.delta.storage.commit.actions.AbstractDomainMetadata>, java.util.Optional<io.delta.storage.commit.uniform.UniformMetadata>): the catalog-level identity is conveyed by tableId / tableUri / tableIdentifier; the Delta-log shape is conveyed by an AbstractMetadata (description, schemaString, partitionColumns, configuration) and an AbstractProtocol (min reader/writer versions, reader/writer features).

Specified by:

createTable in interface UCDeltaClient

Parameters:

tableUri - the storage location of the staged table

tableIdentifier - catalog + schema namespace and table name

tableType - MANAGED or EXTERNAL

metadata - the Delta metadata to register; only description, schemaString, partitionColumns, and configuration are consumed

protocol - the Delta protocol the table will be created with

domainMetadata - the snapshot's domain-metadata entries (e.g. clustering, row-tracking); pass an empty list when none are set

lastCommitTimestampMs - Delta-log timestamp of the initial commit; UC stores this on the catalog entry

uniformMetadata - UniForm Iceberg metadata generated for the initial snapshot; empty when the table was not created with UniForm enabled

Returns:

the newly created table's UCDeltaModels.TableInfo

Throws:

IOException - on network or API errors

reportMetrics

public void reportMetrics(String tableId,
 TableIdentifier tableIdentifier,
 UCDeltaModels.CommitReport report)
                   throws IOException

Description copied from interface: UCDeltaClient

Reports post-commit telemetry for a table (file/row counts, byte counts, file-size histogram) to the catalog.

Specified by:

reportMetrics in interface UCDeltaClient

Parameters:

tableId - catalog-side table id (UC's `table_uuid`)

tableIdentifier - catalog + schema namespace and table name

report - per-commit metrics payload

Throws:

IOException - on network or API errors