Package io.delta.kernel.internal.actions

Class Protocol

Object
io.delta.kernel.internal.actions.Protocol
All Implemented Interfaces:
Serializable
public class Protocol extends Object implements Serializable
See Also:

  • Field Details

    • FULL_SCHEMA

      public static final StructType FULL_SCHEMA

  • Constructor Details

    • Protocol

      public Protocol(int minReaderVersion, int minWriterVersion)

    • Protocol

      public Protocol(int minReaderVersion, int minWriterVersion, Set<String> readerFeatures, Set<String> writerFeatures)

  • Method Details

    • fromRow

      public static Protocol fromRow(Row row)
      Helper method to get the Protocol from the row representation.
      Parameters:
      row - Row representation of the Protocol.
      Returns:
      the Protocol object

    • fromColumnVector

      public static Protocol fromColumnVector(ColumnVector vector, int rowId)

    • getMinReaderVersion

      public int getMinReaderVersion()
      Returns:
      The minimum reader version required for this protocol

    • getMinWriterVersion

      public int getMinWriterVersion()
      Returns:
      The minimum writer version required for this protocol

    • getReaderFeatures

      public Set<String> getReaderFeatures()
      Returns:
      The set of explicitly specified reader features for this protocol. Will be empty if this protocol does not support reader features.

    • getWriterFeatures

      public Set<String> getWriterFeatures()
      Returns:
      The set of explicitly specified writer features for this protocol. Will be empty if this protocol does not support writer features.

    • getReaderAndWriterFeatures

      public Set<String> getReaderAndWriterFeatures()
      Returns:
      The combined set of all reader and writer features for this protocol. Will be empty if this protocol does not support reader or writer features.

    • supportsReaderFeatures

      public boolean supportsReaderFeatures()
      Returns:
      Whether this protocol supports explicitly specifying reader features, which occurs when the minReaderVersion is greater than or equal to 3.

    • supportsWriterFeatures

      public boolean supportsWriterFeatures()
      Returns:
      Whether this protocol supports explicitly specifying writer features, which occurs when the minWriterVersion is greater than or equal to 7.

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toRow

      public Row toRow()
      Encode as a Row object with the schema FULL_SCHEMA. Write any empty `readerFeatures` and `writerFeatures` as null.
      Returns:
      Row object with the schema FULL_SCHEMA

    • getImplicitlySupportedFeatures

      public Set<TableFeature> getImplicitlySupportedFeatures()
      Get the set of features that are implicitly supported by the protocol. Features are implicitly supported if the reader and/or writer version is less than the versions that supports the explicit features specified in `readerFeatures` and `writerFeatures` sets. Examples:

      • (minRV = 1, minWV = 7, readerFeatures=[], writerFeatures=[domainMetadata]) results in []
      • (minRV = 1, minWV = 3) results in [appendOnly, invariants, checkConstraints]
      • (minRV = 3, minWV = 7, readerFeatures=[v2Checkpoint], writerFeatures=[v2Checkpoint]) results in []
      • (minRV = 2, minWV = 6) results in [appendOnly, invariants, checkConstraints, changeDataFeed, generatedColumns, columnMapping, identityColumns]

    • getExplicitlySupportedFeatures

      public Set<TableFeature> getExplicitlySupportedFeatures()
      Get the set of features that are explicitly supported by the protocol. Features are explicitly supported if they are present in the `readerFeatures` and/or `writerFeatures` sets. Examples:

      • (minRV = 1, minWV = 7, writerFeatures=[appendOnly, invariants, checkConstraints]) results in [appendOnly, invariants, checkConstraints]
      • (minRV = 3, minWV = 7, readerFeatures = [columnMapping], writerFeatures=[columnMapping, invariants]) results in [columnMapping, invariants]
      • (minRV = 1, minWV = 2, readerFeatures = [], writerFeatures=[]) results in []
      Throws:
      UnsupportedTableFeatureException - if any table features in the protocol's list of readerFeatures or writerFeatures are unsupported by Kernel

    • getImplicitlyAndExplicitlySupportedFeatures

      public Set<TableFeature> getImplicitlyAndExplicitlySupportedFeatures()
      Get the set of features that are both implicitly and explicitly supported by the protocol. Usually, the protocol has either implicit or explicit features, but not both. This API provides a way to get all enabled features.
      Throws:
      UnsupportedTableFeatureException - if any table features in the protocol's list of readerFeatures or writerFeatures are unsupported by Kernel

    • getImplicitlyAndExplicitlySupportedReaderWriterFeatures

      public Set<TableFeature> getImplicitlyAndExplicitlySupportedReaderWriterFeatures()
      Get the set of reader writer features that are both implicitly and explicitly supported by the protocol. Usually, the protocol has either implicit or explicit features, but not both. This API provides a way to get all enabled reader writer features. It doesn't return any writer only features.

    • withFeatures

      public Protocol withFeatures(Iterable<TableFeature> newFeatures)
      Create a new Protocol object with the given TableFeature supported.

    • withFeature

      public Protocol withFeature(TableFeature feature)
      Get a new Protocol object that has `feature` supported. Writer-only features will be added to `writerFeatures` field, and reader-writer features will be added to `readerFeatures` and `writerFeatures` fields.

      If `feature` is already implicitly supported in the current protocol's legacy reader or writer protocol version, the new protocol will not modify the original protocol version, i.e., the feature will not be explicitly added to the protocol's `readerFeatures` or `writerFeatures`. This is to avoid unnecessary protocol upgrade for feature that it already supports.

      Examples:

      • current protocol (2, 5) and new feature to add 'invariants` result in (2, 5) as this protocol already supports 'invariants' implicitly.
      • current protocol is (1, 7, writerFeature='rowTracking,domainMetadata' and the new feature to add is 'appendOnly' results in (1, 7, writerFeature='rowTracking,domainMetadata,appendOnly')
      • current protocol is (1, 7, writerFeature='rowTracking,domainMetadata' and the new feature to add is 'columnMapping' results in throwing UnsupportedOperationException as 'columnMapping' requires higher reader version (2) than the current protocol's reader version (1).

    • canUpgradeTo

      public boolean canUpgradeTo(Protocol to)
      Determine whether this protocol can be safely upgraded to a new protocol `to`. This means all features supported by this protocol are supported by `to`.

      Examples regarding feature status:

      • `[appendOnly]` to `[appendOnly]` results in allowed.
      • `[appendOnly, changeDataFeed]` to `[appendOnly]` results in not allowed.

    • normalized

      public Protocol normalized()
      Protocol normalization is the process of converting a table features protocol to the weakest possible form. This primarily refers to converting a table features protocol to a legacy protocol. A Table Features protocol can be represented with the legacy representation only when the features set of the former exactly matches a legacy protocol.

      Normalization can also decrease the reader version of a table features protocol when it is higher than necessary.

      For example:

      • (1, 7, AppendOnly, Invariants, CheckConstraints) results in (1, 3)
      • (3, 7, RowTracking) results in (1, 7, RowTracking)

    • denormalized

      public Protocol denormalized()
      Protocol denormalization is the process of converting a legacy protocol to the equivalent table features protocol. This is the inverse of protocol normalization. It can be used to allow operations on legacy protocols that yield results which cannot be represented anymore by a legacy protocol. For example
      • (1, 3) results in (1, 7, readerFeatures=[], writerFeatures=[appendOnly, invariants, checkConstraints])
      • (2, 5) results in (2, 7, readerFeatures=[], writerFeatures=[appendOnly, invariants, checkConstraints, changeDataFeed, generatedColumns, columnMapping])

    • denormalizedNormalized

      public Protocol denormalizedNormalized()
      Helper method that applies both denormalization and normalization. This can be used to normalize invalid legacy protocols such as (2, 3), (1, 5). A legacy protocol is invalid when the version numbers are higher than required to support the implied feature set.

    • merge

      public Protocol merge(Protocol... others)
      Merge this protocol with multiple `protocols` to have the highest reader and writer versions plus all explicitly and implicitly supported features.

    • supportsFeature

      public boolean supportsFeature(TableFeature feature)
      Check if the protocol supports the given table feature