Package io.delta.storage.commit.uccommitcoordinator

Interface UCClient

All Superinterfaces:
AutoCloseable
All Known Implementing Classes:
UCTokenBasedRestClient
public interface UCClient extends AutoCloseable
Interface for interacting with the Unity Catalog. This interface defines the contract for operations related to the Unity Catalog, including retrieving the metastore ID, and adding new commits to Delta tables where UC acts as the Commit Coordinator and similarly retrieving unbackfilled commits. Implementations of this interface should handle the specifics of connecting to and communicating with the Unity Catalog, including any necessary authentication and request handling.

  • Method Details

    • getMetastoreId

      String getMetastoreId() throws IOException
      Retrieves the metastore ID associated with this Unity Catalog instance.
      Returns:
      A String representing the unique identifier of the metastore
      Throws:
      IOException - if there's an error in retrieving the metastore ID

    • commit

      void commit(String tableId, URI tableUri, Optional<Commit> commit, Optional<Long> lastKnownBackfilledVersion, boolean disown, Optional<AbstractMetadata> newMetadata, Optional<AbstractProtocol> newProtocol, Optional<UniformMetadata> uniform) throws IOException, CommitFailedException, UCCommitCoordinatorException
      Commits new changes to a Delta table using the Unity Catalog as the Commit Coordinator. This method is responsible for committing changes to a Delta table, including new data, metadata updates, and protocol changes. It interacts with the Unity Catalog to ensure proper coordination and consistency of the commit process. Note: At least one of commit or lastKnownBackfilledVersion must be present.
      Parameters:
      tableId - The unique identifier of the Delta table.
      tableUri - The URI of the storage location of the table. e.g. s3://bucket/path/to/table (and not s3://bucket/path/to/table/_delta_log). If the tableId exists but the tableUri is different from the one previously registered (e.g., if the table as moved), the request will fail.
      commit - An Optional containing the Commit object with the changes to be committed. If empty, it indicates that no new data is being added in this commit.
      lastKnownBackfilledVersion - An Optional containing the last known backfilled version of the table. This value serves as a hint to the UC about the most recent version that has been successfully backfilled. UC can use this information to optimize its internal state management by cleaning up tracking information for backfilled commits up to this version. If not provided (Optional.empty()), UC will rely on its current state without any additional cleanup hints.
      disown - A boolean flag indicating whether to disown the table after commit. If true, the coordinator will release ownership of the table after the commit.
      newMetadata - An Optional containing new metadata to be applied to the table. If present, the table's metadata will be updated atomically with the commit.
      newProtocol - An Optional containing a new protocol version to be applied to the table. If present, the table's protocol will be updated atomically with the commit.
      uniform - An Optional containing UniForm metadata for Delta Universal Format support. If present, this metadata will be used by UC to manage format conversions (e.g., Iceberg, Hudi).
      Throws:
      IOException - if there's an error during the commit process, such as network issues.
      CommitFailedException - if the commit fails due to conflicts or other logical errors.
      UCCommitCoordinatorException - if there's an error specific to the Unity Catalog commit coordination process.

    • getCommits

      GetCommitsResponse getCommits(String tableId, URI tableUri, Optional<Long> startVersion, Optional<Long> endVersion) throws IOException, UCCommitCoordinatorException
      Retrieves the unbackfilled commits for a Delta table within a specified version range.
      Parameters:
      tableId - The unique identifier of the Delta table.
      tableUri - The URI of the storage location of the table. e.g. s3://bucket/path/to/table (and not s3://bucket/path/to/table/_delta_log). If the tableId exists but the tableUri is different from the one previously registered (e.g., if the table as moved), the request will fail.
      startVersion - An Optional containing the start version of the range of commits to retrieve.
      endVersion - An Optional containing the end version of the range of commits to retrieve.
      Returns:
      A GetCommitsResponse object containing the unbackfilled commits within the specified version range. If all commits are backfilled, the response will contain an empty list. The response also contains the last known backfilled version of the table. If no commits are ratified via UC, the lastKnownBackfilledVersion will be -1.
      Throws:
      IOException - if there's an error during the commit process, such as network issues.
      UCCommitCoordinatorException - if there's an error specific to the Unity Catalog such as the table not being found.

    • finalizeCreate

      void finalizeCreate(String tableName, String catalogName, String schemaName, String storageLocation, List<UCClient.ColumnDef> columns, Map<String,String> properties) throws CommitFailedException
      Promotes a staging table to a real managed table in Unity Catalog. This is the correct API for version 0 (CREATE); for version 1+ (WRITE), use commit(java.lang.String, java.net.URI, java.util.Optional<io.delta.storage.commit.Commit>, java.util.Optional<java.lang.Long>, boolean, java.util.Optional<io.delta.storage.commit.actions.AbstractMetadata>, java.util.Optional<io.delta.storage.commit.actions.AbstractProtocol>, java.util.Optional<io.delta.storage.commit.uniform.UniformMetadata>) instead.
      Parameters:
      tableName - table name (relative to parent schema)
      catalogName - parent catalog name in Unity Catalog
      schemaName - parent schema name in Unity Catalog
      storageLocation - the storage root URL for the table
      columns - column definitions for the table schema
      properties - properties to persist in UC (protocol features, metadata config, etc.)
      Throws:
      CommitFailedException - if there is a network or server error during finalization

    • close

      void close() throws IOException
      Closes any resources used by this client. This method should be called to properly release resources such as network connections (e.g., HTTPClient) when the client is no longer needed. Once this method is called, the client should not be used to perform any operations.
      Specified by:
      close in interface AutoCloseable
      Throws:
      IOException - if there's an error while closing resources