case class PreprocessTableWithDVsStrategy(session: SparkSession) extends SparkStrategy with PreprocessTableWithDVs with Product with Serializable
Strategy to process tables with DVs and add the skip row column and filters.
This strategy will apply all transformations needed to tables with DVs and delegate to FileSourceStrategy to create the final plan. The DV filter will be the bottom-most filter in the plan and so it will be pushed down to the FileSourceScanExec at the beginning of the filter list.
Linear Supertypes
Ordering
- Alphabetic
- By Inheritance
Inherited
- PreprocessTableWithDVsStrategy
- Serializable
- Product
- Equals
- PreprocessTableWithDVs
- SubqueryTransformerHelper
- SparkStrategy
- GenericStrategy
- Logging
- AnyRef
- Any
- Hide All
- Show All
Visibility
- Public
- Protected
Instance Constructors
- new PreprocessTableWithDVsStrategy(session: SparkSession)
Type Members
- implicit class LogStringContext extends AnyRef
- Definition Classes
- Logging
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def apply(plan: LogicalPlan): Seq[SparkPlan]
- Definition Classes
- PreprocessTableWithDVsStrategy → GenericStrategy
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def initializeLogIfNecessary(isInterpreter: Boolean): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isSubqueryRoot(plan: LogicalPlan): Boolean
Is the give plan a subquery root.
Is the give plan a subquery root.
- Definition Classes
- SubqueryTransformerHelper
- def isTraceEnabled(): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def log: Logger
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logName: String
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def planLater(plan: LogicalPlan): SparkPlan
- Attributes
- protected
- Definition Classes
- SparkStrategy → GenericStrategy
- def preprocessTablesWithDVs(plan: LogicalPlan): LogicalPlan
- Definition Classes
- PreprocessTableWithDVs
- def productElementNames: Iterator[String]
- Definition Classes
- Product
- val session: SparkSession
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def transformWithSubqueries(plan: LogicalPlan)(rule: PartialFunction[LogicalPlan, LogicalPlan]): LogicalPlan
Transform all nodes matched by the rule in the query plan rooted at given
plan.Transform all nodes matched by the rule in the query plan rooted at given
plan. It traverses the tree starting from the leaves, whenever a SubqueryExpression expression is encountered, given rule is applied to the subquery planplanin SubqueryExpression starting from theplanroot until leaves.This is slightly different behavior compared to QueryPlan.transformUpWithSubqueries or QueryPlan.transformDownWithSubqueries
It requires that the given plan already gone through OptimizeSubqueries and the root node denoting a subquery is removed and optimized appropriately.
- Definition Classes
- SubqueryTransformerHelper
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- def withLogContext(context: Map[String, String])(body: => Unit): Unit
- Attributes
- protected
- Definition Classes
- Logging
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)