class DataFiltersBuilder extends AnyRef
Builds the data filters for data skipping.
- Alphabetic
- By Inheritance
- DataFiltersBuilder
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new DataFiltersBuilder(spark: SparkSession, dataSkippingType: DeltaDataSkippingType)
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def apply(dataFilter: Expression): Option[DataSkippingPredicate]
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- val dataSkippingType: DeltaDataSkippingType
- Attributes
- protected
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def rewriteDataFiltersAsPartitionLike(clusteringColumns: Seq[String], expr: Expression): Option[DataSkippingPredicate]
Rewrites the given expression as a partition-like expression if possible: 1.
Rewrites the given expression as a partition-like expression if possible: 1. Rewrite the attribute references in the expression to reference the collected min stats on the attribute reference's column. 2. Construct an expression that returns true if any of the referenced columns are not partition-like on a given file. The rewritten expression is a union of the above expressions: a file is read if it's either not partition-like on any of the columns or if the rewritten expression evaluates to true.
- clusteringColumns
The columns that are used for clustering.
- expr
The data filtering expression to rewrite.
- returns
If the expression is safe to rewrite, return the rewritten expression. Otherwise, return None.
- val spark: SparkSession
- Attributes
- protected
- val statsProvider: StatsProvider
- Attributes
- protected
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- object SkippingEligibleExpression extends GenericSkippingEligibleExpression
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)