Package io.delta.storage.commit.uccommitcoordinator

Interface UCClient

All Superinterfaces:
AutoCloseable
All Known Subinterfaces:
UCDeltaClient
All Known Implementing Classes:
UCDeltaTokenBasedRestClient, UCTokenBasedRestClient
public interface UCClient extends AutoCloseable
Interface for interacting with the Unity Catalog. This interface defines the contract for operations related to the Unity Catalog, including retrieving the metastore ID, and adding new commits to Delta tables where UC acts as the Commit Coordinator and similarly retrieving unbackfilled commits. Implementations of this interface should handle the specifics of connecting to and communicating with the Unity Catalog, including any necessary authentication and request handling.

  • Method Details

    • getMetastoreId

      String getMetastoreId() throws IOException
      Retrieves the metastore ID associated with this Unity Catalog instance.
      Returns:
      A String representing the unique identifier of the metastore
      Throws:
      IOException - if there's an error in retrieving the metastore ID

    • commit

      void commit(String tableId, URI tableUri, TableIdentifier tableIdentifier, Optional<Commit> commit, Optional<Long> lastKnownBackfilledVersion, Optional<AbstractMetadata> oldMetadata, Optional<AbstractMetadata> newMetadata, Optional<AbstractProtocol> oldProtocol, Optional<AbstractProtocol> newProtocol, List<AbstractDomainMetadata> transactionDomainMetadata, Optional<UniformMetadata> uniform) throws IOException, CommitFailedException, UCCommitCoordinatorException
      Commits new changes to a Delta table using the Unity Catalog as the Commit Coordinator. This method is responsible for committing changes to a Delta table, including new data, metadata updates, and protocol changes. It interacts with the Unity Catalog to ensure proper coordination and consistency of the commit process. Note: At least one of commit or lastKnownBackfilledVersion must be present.
      Parameters:
      tableId - The Unity Catalog table ID (UUID) identifying the target table.
      tableUri - The URI of the storage location of the table (e.g., s3://bucket/path/to/table, not the _delta_log path).
      tableIdentifier - The three-part table identifier (catalog, schema, table name) for the table in Unity Catalog, or null when the caller has no catalog context. Implementations that require this value must reject null; implementations that don't use it may ignore it.
      commit - An Optional containing the Commit object with the changes to be committed. If empty, it indicates that no new data is being added in this commit.
      lastKnownBackfilledVersion - An Optional containing the last known backfilled version of the table. This value serves as a hint to UC about the most recent version that has been successfully backfilled. UC can use this information to clean up tracking information for backfilled commits up to this version. If not provided (Optional.empty()), UC will rely on its current state without any additional cleanup hints.
      oldMetadata - An Optional containing the existing metadata of the table before the commit. If present, UC can validate the current state of the table.
      newMetadata - An Optional containing new metadata to be applied to the table. If present, the table's metadata will be updated atomically with the commit.
      oldProtocol - An Optional containing the existing protocol of the table before the commit. If present, UC can validate the current protocol state.
      newProtocol - An Optional containing a new protocol version to be applied to the table. If present, the table's protocol will be updated atomically with the commit.
      transactionDomainMetadata - The raw domain metadata actions from this transaction. A non-removed action sets domain metadata, and a removed action removes the domain metadata.
      uniform - An Optional containing UniForm metadata for Delta Universal Format support. If present, this metadata will be used by UC to manage format conversions (e.g., Iceberg, Hudi).
      Throws:
      IOException - if there's an error during the commit process, such as network issues.
      CommitFailedException - if the commit fails due to conflicts or other logical errors.
      UCCommitCoordinatorException - if there's an error specific to the Unity Catalog commit coordination process.

    • getCommits

      GetCommitsResponse getCommits(String tableId, URI tableUri, TableIdentifier tableIdentifier, Optional<Long> startVersion, Optional<Long> endVersion) throws IOException, UCCommitCoordinatorException
      Retrieves the unbackfilled commits for a Delta table within a specified version range.
      Parameters:
      tableId - The unique identifier of the Delta table.
      tableUri - The URI of the storage location of the table. e.g. s3://bucket/path/to/table (and not s3://bucket/path/to/table/_delta_log). If the tableId exists but the tableUri is different from the one previously registered (e.g., if the table as moved), the request will fail.
      tableIdentifier - The three-part table identifier (catalog, schema, table name) for the table in Unity Catalog, or null when the caller has no catalog context. Implementations that require this value must reject null; implementations that don't use it may ignore it.
      startVersion - An Optional containing the start version of the range of commits to retrieve.
      endVersion - An Optional containing the end version of the range of commits to retrieve.
      Returns:
      A GetCommitsResponse object containing the unbackfilled commits within the specified version range. If all commits are backfilled, the response will contain an empty list. The response also contains the last known backfilled version of the table. If no commits are ratified via UC, the lastKnownBackfilledVersion will be -1.
      Throws:
      IOException - if there's an error during the commit process, such as network issues.
      UCCommitCoordinatorException - if there's an error specific to the Unity Catalog such as the table not being found.

    • finalizeCreate

      void finalizeCreate(String tableName, String catalogName, String schemaName, String storageLocation, List<UCClient.ColumnDef> columns, AbstractProtocol protocol, Map<String,String> properties, long lastCommitTimestampMs, List<AbstractDomainMetadata> domainMetadata) throws CommitFailedException
      Promotes a staging table to a real managed table in Unity Catalog. This is the correct API for version 0 (CREATE); for version 1+ (WRITE), use commit(java.lang.String, java.net.URI, io.delta.storage.commit.TableIdentifier, java.util.Optional<io.delta.storage.commit.Commit>, java.util.Optional<java.lang.Long>, java.util.Optional<io.delta.storage.commit.actions.AbstractMetadata>, java.util.Optional<io.delta.storage.commit.actions.AbstractMetadata>, java.util.Optional<io.delta.storage.commit.actions.AbstractProtocol>, java.util.Optional<io.delta.storage.commit.actions.AbstractProtocol>, java.util.List<io.delta.storage.commit.actions.AbstractDomainMetadata>, java.util.Optional<io.delta.storage.commit.uniform.UniformMetadata>) instead.
      Parameters:
      tableName - table name (relative to parent schema)
      catalogName - parent catalog name in Unity Catalog
      schemaName - parent schema name in Unity Catalog
      storageLocation - the storage root URL for the table
      columns - column definitions for the table schema
      protocol - the table's protocol (min reader/writer versions and features).
      properties - properties to persist in UC. For Delta-Commits clients this is the flattened bag (protocol features, metadata config, timestamp, version, clustering); for Delta-Tables clients this is the metadata.configuration.
      lastCommitTimestampMs - the Delta-log timestamp of the version-0 commit.
      domainMetadata - the version-0 domain-metadata actions (e.g. clustering, row tracking).
      Throws:
      CommitFailedException - if there is a network or server error during finalization

    • close

      void close() throws IOException
      Closes any resources used by this client. This method should be called to properly release resources such as network connections (e.g., HTTPClient) when the client is no longer needed. Once this method is called, the client should not be used to perform any operations.
      Specified by:
      close in interface AutoCloseable
      Throws:
      IOException - if there's an error while closing resources