object DeltaOperations
Exhaustive list of operations that can be performed on a Delta table. These operations are
tracked as the first line in delta logs, and power DESCRIBE HISTORY for Delta tables.
- Alphabetic
- By Inheritance
- DeltaOperations
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Type Members
- case class AddColumns(colsToAdd: Seq[QualifiedColTypeWithPositionForLog]) extends Operation with Product with Serializable
Recorded when columns are added.
- case class AddConstraint(constraintName: String, expr: String) extends Operation with Product with Serializable
- case class ChangeColumn(columnPath: Seq[String], columnName: String, newColumn: StructField, colPosition: Option[String]) extends Operation with Product with Serializable
Recorded when columns are changed.
- case class ChangeColumns(columns: Seq[ChangeColumn]) extends Operation with Product with Serializable
Recorded when columns are changed in bulk.
- case class Clone(source: String, sourceVersion: Long) extends Operation with Product with Serializable
- case class ClusterBy(oldClusteringColumns: String, newClusteringColumns: String) extends Operation with Product with Serializable
Recorded when clustering columns are changed on clustered tables.
- case class ComputeStats(predicate: Seq[Expression]) extends OperationWithPredicates with Product with Serializable
Recorded when recomputing stats on the table.
- case class Convert(numFiles: Long, partitionBy: Seq[String], collectStats: Boolean, catalogTable: Option[String], sourceFormat: Option[String]) extends Operation with Product with Serializable
Recorded when converting a table into a Delta table.
- case class CreateTable(metadata: Metadata, isManaged: Boolean, asSelect: Boolean = false, clusterBy: Option[Seq[String]] = None) extends Operation with Product with Serializable
Recorded when the table is created.
- case class Delete(predicate: Seq[Expression]) extends OperationWithPredicates with Product with Serializable
Recorded while deleting certain partitions.
- case class DropColumns(colsToDrop: Seq[Seq[String]]) extends Operation with Product with Serializable
- case class DropConstraint(constraintName: String, expr: Option[String]) extends Operation with Product with Serializable
- case class DropTableFeature(featureName: String, truncateHistory: Boolean) extends Operation with Product with Serializable
Recorded when dropping a table feature.
- case class Merge(predicate: Option[Expression], updatePredicate: Option[String], deletePredicate: Option[String], insertPredicate: Option[String], matchedPredicates: Seq[MergePredicate], notMatchedPredicates: Seq[MergePredicate], notMatchedBySourcePredicates: Seq[MergePredicate]) extends OperationWithPredicates with Product with Serializable
- case class MergePredicate(predicate: Option[String], actionType: String) extends Product with Serializable
Represents the predicates and action type (insert, update, delete) for a Merge clause
- sealed abstract class Operation extends AnyRef
An operation that can be performed on a Delta table.
- abstract class OperationWithPredicates extends Operation
- case class Optimize(predicate: Seq[Expression], zOrderBy: Seq[String] = Seq.empty, auto: Boolean = false, clusterBy: Option[Seq[String]] = None, isFull: Boolean = false) extends OptimizeOrReorg with Product with Serializable
Recorded when optimizing the table.
- sealed abstract class OptimizeOrReorg extends OperationWithPredicates
- case class QualifiedColTypeWithPositionForLog(columnPath: Seq[String], column: StructField, colPosition: Option[String]) extends Product with Serializable
Qualified column type with position.
Qualified column type with position. We define a copy of the type here to avoid depending on the parser output classes in our logging.
- case class RemoveColumnMapping(userMetadata: Option[String] = None) extends Operation with Product with Serializable
- case class RenameColumn(oldColumnPath: Seq[String], newColumnPath: Seq[String]) extends Operation with Product with Serializable
- case class Reorg(predicate: Seq[Expression], applyPurge: Boolean = true) extends OptimizeOrReorg with Product with Serializable
Recorded when running REORG on the table.
- case class ReplaceColumns(columns: Seq[StructField]) extends Operation with Product with Serializable
Recorded when columns are replaced.
- case class ReplaceTable(metadata: Metadata, isManaged: Boolean, orCreate: Boolean, asSelect: Boolean = false, userMetadata: Option[String] = None, clusterBy: Option[Seq[String]] = None) extends Operation with Product with Serializable
Recorded when the table is replaced.
- case class Restore(version: Option[Long], timestamp: Option[String]) extends Operation with Product with Serializable
- case class RowTrackingBackfill(batchId: Int = 0) extends Operation with Product with Serializable
Recorded when we backfill a Delta table's existing AddFiles with row tracking data.
- case class SetTableProperties(properties: Map[String, String]) extends Operation with Product with Serializable
- case class StreamingUpdate(outputMode: OutputMode, queryId: String, epochId: Long, userMetadata: Option[String] = None) extends Operation with Product with Serializable
Recorded during streaming inserts.
- case class TestOperation(operationName: String = "TEST", isInPlaceFileMetadataUpdate: Option[Boolean] = None) extends Operation with Product with Serializable
Dummy operation only for testing with arbitrary operation names
- case class Truncate() extends Operation with Product with Serializable
Recorded when truncating the table.
- case class UnsetTableProperties(propKeys: Seq[String], ifExists: Boolean) extends Operation with Product with Serializable
Recorded when the table properties are unset.
- case class Update(predicate: Option[Expression]) extends OperationWithPredicates with Product with Serializable
Recorded when an update operation is committed to the table.
- case class UpdateColumnMetadata(operationName: String, columns: Seq[(Seq[String], StructField)]) extends Operation with Product with Serializable
- case class UpdateSchema(oldSchema: StructType, newSchema: StructType) extends Operation with Product with Serializable
- case class UpgradeProtocol(newProtocol: Protocol) extends Operation with Product with Serializable
- case class UpgradeUniformProperties(properties: Map[String, String]) extends Operation with Product with Serializable
recorded when upgrading a table set uniform properties by REORG TABLE ...
recorded when upgrading a table set uniform properties by REORG TABLE ... UPGRADE UNIFORM
- case class VacuumEnd(status: String) extends Operation with Product with Serializable
- status
- whether the vacuum operation was successful; either "COMPLETED" or "FAILED"
- case class VacuumStart(retentionCheckEnabled: Boolean, specifiedRetentionMillis: Option[Long], defaultRetentionMillis: Long) extends Operation with Product with Serializable
- retentionCheckEnabled
- whether retention check was enabled for this run of vacuum.
- specifiedRetentionMillis
- specified retention interval
- defaultRetentionMillis
- default retention period for the table
- case class Write(mode: SaveMode, partitionBy: Option[Seq[String]] = None, predicate: Option[String] = None, userMetadata: Option[String] = None) extends Operation with Product with Serializable
Recorded during batch inserts.
Recorded during batch inserts. Predicates can be provided for overwrites.
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- val AUTO_COMPACTION_PARAMETER_KEY: String
parameter key to indicate whether it's an Auto Compaction
- val CLUSTERING_IS_FULL_KEY: String
parameter key to indicate the operation for
OPTIMIZE tbl FULL - val CLUSTERING_PARAMETER_KEY: String
parameter key to indicate clustering columns
- val OPTIMIZE_OPERATION_NAME: String
operation name for OPTIMIZE command
- val OP_CLONE: String
Recorded when cloning a Delta table into a new location.
- val OP_DROP_COLUMN: String
Recorded when columns are dropped.
- val OP_MERGE: String
Recorded when a merge operation is committed to the table.
Recorded when a merge operation is committed to the table.
updatePredicate,deletePredicate, andinsertPredicateare DEPRECATED. Only usepredicate,matchedPredicates,notMatchedPredicatesandnotMatchedBySourcePredicatesto record the merge. - val OP_RENAME_COLUMN: String
Recorded when column is renamed
- val OP_RESTORE: String
Recorded when restoring a Delta table to an older version.
- val OP_SET_TBLPROPERTIES: String
Recorded when the table properties are set.
- val REORG_OPERATION_NAME: String
operation name for REORG command
- val ROW_TRACKING_BACKFILL_OPERATION_NAME: String
operation name for ROW TRACKING BACKFILL command
- val ZORDER_PARAMETER_KEY: String
parameter key to indicate which columns to z-order by
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def predicatesToString(predicates: Seq[Expression]): Seq[String]
Helper method to convert a sequence of command predicates in the form of an Expressions to a sequence of Strings so be stored in the commit info.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- object AddDeletionVectorsTombstones extends Operation
Recorded when dropping deletion vectors.
Recorded when dropping deletion vectors. Deletion Vector tombstones directly reference deletion vector files within the retention period. This is to protect them from deletion against oblivious writers when vacuuming.
- object EmptyCommit extends Operation
A commit without any actions.
A commit without any actions. Could be used to force creation of new checkpoints.
- object ManualUpdate extends Operation
- object Merge extends Serializable
- object MergePredicate extends Serializable
- object VacuumEnd extends Serializable
- object VacuumStart extends Serializable
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)