Packages

c

org.apache.spark.sql.delta.commands

DeltaOptimizeContext

case class DeltaOptimizeContext(isPurge: Boolean = false, minFileSize: Option[Long] = None, maxDeletedRowsRatio: Option[Double] = None) extends Product with Serializable

Stored all runtime context information that can control the execution of optimize.

isPurge

Whether the rewriting task is only for purging soft-deleted data instead of for compaction. If isPurge is true, only files with DVs will be selected for compaction.

minFileSize

Files which are smaller than this threshold will be selected for compaction. If not specified, DeltaSQLConf.DELTA_OPTIMIZE_MIN_FILE_SIZE will be used. This parameter must be set to 0 when isPurge is true.

maxDeletedRowsRatio

Files with a ratio of soft-deleted rows to the total rows larger than this threshold will be rewritten by the OPTIMIZE command. If not specified, DeltaSQLConf.DELTA_OPTIMIZE_MAX_DELETED_ROWS_RATIO will be used. This parameter must be set to 0 when isPurge is true.

Linear Supertypes
Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DeltaOptimizeContext
  2. Serializable
  3. Product
  4. Equals
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new DeltaOptimizeContext(isPurge: Boolean = false, minFileSize: Option[Long] = None, maxDeletedRowsRatio: Option[Double] = None)

    isPurge

    Whether the rewriting task is only for purging soft-deleted data instead of for compaction. If isPurge is true, only files with DVs will be selected for compaction.

    minFileSize

    Files which are smaller than this threshold will be selected for compaction. If not specified, DeltaSQLConf.DELTA_OPTIMIZE_MIN_FILE_SIZE will be used. This parameter must be set to 0 when isPurge is true.

    maxDeletedRowsRatio

    Files with a ratio of soft-deleted rows to the total rows larger than this threshold will be rewritten by the OPTIMIZE command. If not specified, DeltaSQLConf.DELTA_OPTIMIZE_MAX_DELETED_ROWS_RATIO will be used. This parameter must be set to 0 when isPurge is true.

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  8. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  9. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  10. val isPurge: Boolean
  11. val maxDeletedRowsRatio: Option[Double]
  12. val minFileSize: Option[Long]
  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. def productElementNames: Iterator[String]
    Definition Classes
    Product
  17. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  18. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  19. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  20. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped