Package io.delta.storage.commit.uccommitcoordinator

Class UCTokenBasedRestClient

Object
io.delta.storage.commit.uccommitcoordinator.UCTokenBasedRestClient
All Implemented Interfaces:
UCClient, AutoCloseable
public class UCTokenBasedRestClient extends Object implements UCClient
A REST client implementation of UCClient for interacting with Unity Catalog's commit coordination service. This client uses the Unity Catalog SDK with TokenProvider-based authentication for managing Delta table commits and metadata.

The client handles the following primary operations:

  • Retrieving metastore information
  • Committing changes to Delta tables
  • Fetching unbackfilled commit histories

All requests are authenticated using a TokenProvider that generates Bearer tokens dynamically. The client uses the Unity Catalog SDK's DeltaCommitsApi and MetastoresApi for API interactions.

Usage example: 


 TokenProvider tokenProvider = ... // Create or configure TokenProvider
 try (UCTokenBasedRestClient client = new UCTokenBasedRestClient(baseUri, tokenProvider, Map.of())) {
     String metastoreId = client.getMetastoreId();
     // Perform operations with the client...
 }
See Also:

  • Constructor Details

    • UCTokenBasedRestClient

      public UCTokenBasedRestClient(String baseUri, io.unitycatalog.client.auth.TokenProvider tokenProvider, Map<String,String> appVersions)
      Constructs a new UCTokenBasedRestClient with the specified base URI, TokenProvider, and application version information for telemetry.
      Parameters:
      baseUri - The base URI of the Unity Catalog server
      tokenProvider - The TokenProvider to use for authentication
      appVersions - A map of application name to version string (e.g. "Delta" -> "4.0.0"). Each entry is registered for User-Agent telemetry. May be empty.

  • Method Details

    • getMetastoreId

      public String getMetastoreId() throws IOException
      Description copied from interface: UCClient
      Retrieves the metastore ID associated with this Unity Catalog instance.
      Specified by:
      getMetastoreId in interface UCClient
      Returns:
      A String representing the unique identifier of the metastore
      Throws:
      IOException - if there's an error in retrieving the metastore ID

    • commit

      public void commit(String tableId, URI tableUri, Optional<Commit> commit, Optional<Long> lastKnownBackfilledVersion, boolean disown, Optional<AbstractMetadata> newMetadata, Optional<AbstractProtocol> newProtocol, Optional<UniformMetadata> uniform) throws IOException, CommitFailedException, UCCommitCoordinatorException
      Description copied from interface: UCClient
      Commits new changes to a Delta table using the Unity Catalog as the Commit Coordinator. This method is responsible for committing changes to a Delta table, including new data, metadata updates, and protocol changes. It interacts with the Unity Catalog to ensure proper coordination and consistency of the commit process. Note: At least one of commit or lastKnownBackfilledVersion must be present.
      Specified by:
      commit in interface UCClient
      Parameters:
      tableId - The unique identifier of the Delta table.
      tableUri - The URI of the storage location of the table. e.g. s3://bucket/path/to/table (and not s3://bucket/path/to/table/_delta_log). If the tableId exists but the tableUri is different from the one previously registered (e.g., if the table as moved), the request will fail.
      commit - An Optional containing the Commit object with the changes to be committed. If empty, it indicates that no new data is being added in this commit.
      lastKnownBackfilledVersion - An Optional containing the last known backfilled version of the table. This value serves as a hint to the UC about the most recent version that has been successfully backfilled. UC can use this information to optimize its internal state management by cleaning up tracking information for backfilled commits up to this version. If not provided (Optional.empty()), UC will rely on its current state without any additional cleanup hints.
      disown - A boolean flag indicating whether to disown the table after commit. If true, the coordinator will release ownership of the table after the commit.
      newMetadata - An Optional containing new metadata to be applied to the table. If present, the table's metadata will be updated atomically with the commit.
      newProtocol - An Optional containing a new protocol version to be applied to the table. If present, the table's protocol will be updated atomically with the commit.
      uniform - An Optional containing UniForm metadata for Delta Universal Format support. If present, this metadata will be used by UC to manage format conversions (e.g., Iceberg, Hudi).
      Throws:
      IOException - if there's an error during the commit process, such as network issues.
      CommitFailedException - if the commit fails due to conflicts or other logical errors.
      UCCommitCoordinatorException - if there's an error specific to the Unity Catalog commit coordination process.

    • getCommits

      public GetCommitsResponse getCommits(String tableId, URI tableUri, Optional<Long> startVersion, Optional<Long> endVersion) throws IOException, UCCommitCoordinatorException
      Description copied from interface: UCClient
      Retrieves the unbackfilled commits for a Delta table within a specified version range.
      Specified by:
      getCommits in interface UCClient
      Parameters:
      tableId - The unique identifier of the Delta table.
      tableUri - The URI of the storage location of the table. e.g. s3://bucket/path/to/table (and not s3://bucket/path/to/table/_delta_log). If the tableId exists but the tableUri is different from the one previously registered (e.g., if the table as moved), the request will fail.
      startVersion - An Optional containing the start version of the range of commits to retrieve.
      endVersion - An Optional containing the end version of the range of commits to retrieve.
      Returns:
      A GetCommitsResponse object containing the unbackfilled commits within the specified version range. If all commits are backfilled, the response will contain an empty list. The response also contains the last known backfilled version of the table. If no commits are ratified via UC, the lastKnownBackfilledVersion will be -1.
      Throws:
      IOException - if there's an error during the commit process, such as network issues.
      UCCommitCoordinatorException - if there's an error specific to the Unity Catalog such as the table not being found.

    • close

      public void close() throws IOException
      Description copied from interface: UCClient
      Closes any resources used by this client. This method should be called to properly release resources such as network connections (e.g., HTTPClient) when the client is no longer needed. Once this method is called, the client should not be used to perform any operations.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface UCClient
      Throws:
      IOException - if there's an error while closing resources

    • finalizeCreate

      public void finalizeCreate(String tableName, String catalogName, String schemaName, String storageLocation, List<UCClient.ColumnDef> columns, Map<String,String> properties) throws CommitFailedException
      Description copied from interface: UCClient
      Promotes a staging table to a real managed table in Unity Catalog. This is the correct API for version 0 (CREATE); for version 1+ (WRITE), use UCClient.commit(java.lang.String, java.net.URI, java.util.Optional<io.delta.storage.commit.Commit>, java.util.Optional<java.lang.Long>, boolean, java.util.Optional<io.delta.storage.commit.actions.AbstractMetadata>, java.util.Optional<io.delta.storage.commit.actions.AbstractProtocol>, java.util.Optional<io.delta.storage.commit.uniform.UniformMetadata>) instead.
      Specified by:
      finalizeCreate in interface UCClient
      Parameters:
      tableName - table name (relative to parent schema)
      catalogName - parent catalog name in Unity Catalog
      schemaName - parent schema name in Unity Catalog
      storageLocation - the storage root URL for the table
      columns - column definitions for the table schema
      properties - properties to persist in UC (protocol features, metadata config, etc.)
      Throws:
      CommitFailedException - if there is a network or server error during finalization