object NormalizePlan extends PredicateHelper
- Alphabetic
- By Inheritance
- NormalizePlan
- PredicateHelper
- Logging
- AliasHelper
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Type Members
- implicit class LogStringContext extends AnyRef
- Definition Classes
- Logging
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def apply(plan: LogicalPlan): LogicalPlan
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def buildBalancedPredicate(expressions: Seq[Expression], op: (Expression, Expression) => Expression): Expression
Builds a balanced output predicate in bottom up approach, by applying binary operator op pair by pair on input predicates exprs recursively.
Builds a balanced output predicate in bottom up approach, by applying binary operator op pair by pair on input predicates exprs recursively. Example: exprs = [a, b, c, d], op = And, returns (a And b) And (c And d) exprs = [a, b, c, d, e, f], op = And, returns ((a And b) And (c And d)) And (e And f)
- Attributes
- protected
- Definition Classes
- PredicateHelper
- def canEvaluate(expr: Expression, plan: LogicalPlan): Boolean
Returns true if
exprcan be evaluated using only the output ofplan.Returns true if
exprcan be evaluated using only the output ofplan. This method can be used to determine when it is acceptable to move expression evaluation within a query plan.For example consider a join between two relations R(a, b) and S(c, d).
-
canEvaluate(EqualTo(a,b), R)returnstrue-canEvaluate(EqualTo(a,c), R)returnsfalse-canEvaluate(Literal(1), R)returnstrueas literals CAN be evaluated on any plan- Attributes
- protected
- Definition Classes
- PredicateHelper
- def canEvaluateWithinJoin(expr: Expression): Boolean
Returns true iff
exprcould be evaluated as a condition within join.Returns true iff
exprcould be evaluated as a condition within join.- Attributes
- protected
- Definition Classes
- PredicateHelper
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def extractPredicatesWithinOutputSet(condition: Expression, outputSet: AttributeSet): Option[Expression]
Returns a filter that its reference is a subset of
outputSetand it contains the maximum constraints fromcondition.Returns a filter that its reference is a subset of
outputSetand it contains the maximum constraints fromcondition. This is used for predicate pushdown. When there is no such filter,Noneis returned.- Attributes
- protected
- Definition Classes
- PredicateHelper
- def findExpressionAndTrackLineageDown(exp: Expression, plan: LogicalPlan): Option[(Expression, LogicalPlan)]
Find the origin of where the input references of expression exp were scanned in the tree of plan, and if they originate from a single leaf node.
Find the origin of where the input references of expression exp were scanned in the tree of plan, and if they originate from a single leaf node. Returns optional tuple with Expression, undoing any projections and aliasing that has been done along the way from plan to origin, and the origin LeafNode plan from which all the exp
- Definition Classes
- PredicateHelper
- def getAliasMap(exprs: Seq[NamedExpression]): AttributeMap[Alias]
- Attributes
- protected
- Definition Classes
- AliasHelper
- def getAliasMap(plan: Aggregate): AttributeMap[Alias]
- Attributes
- protected
- Definition Classes
- AliasHelper
- def getAliasMap(plan: Project): AttributeMap[Alias]
- Attributes
- protected
- Definition Classes
- AliasHelper
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def initializeLogIfNecessary(isInterpreter: Boolean): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isLikelySelective(e: Expression): Boolean
Returns whether an expression is likely to be selective
Returns whether an expression is likely to be selective
- Definition Classes
- PredicateHelper
- def isNullIntolerant(expr: Expression): Boolean
- Attributes
- protected
- Definition Classes
- PredicateHelper
- def isTraceEnabled(): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def log: Logger
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logName: String
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def normalizeExprIds(plan: LogicalPlan): LogicalPlan
Since attribute references are given globally unique ids during analysis, we must normalize them to check if two different queries are identical.
- def normalizeExpressions(plan: LogicalPlan): LogicalPlan
Normalizes expressions in a plan, that either produces non-deterministic results or will be different between fixed-point and single-pass analyzer, due to the nature of bottom-up resolution.
Normalizes expressions in a plan, that either produces non-deterministic results or will be different between fixed-point and single-pass analyzer, due to the nature of bottom-up resolution. Before normalization, pre-process the plan by replacing all RuntimeReplaceable nodes with their replacements.
- def normalizePlan(plan: LogicalPlan): LogicalPlan
Normalizes plans: - Filter the filter conditions that appear in a plan.
Normalizes plans: - Filter the filter conditions that appear in a plan. For instance, ((expr 1 && expr 2) && expr 3), (expr 1 && expr 2 && expr 3), (expr 3 && (expr 1 && expr 2) etc., will all now be equivalent. - Sample the seed will replaced by 0L. - Join conditions will be resorted by hashCode. - CTERelationDef ids will be rewritten using a monitonically increasing counter from 0. - CTERelationRef ids will be remapped based on the new CTERelationDef IDs. This is possible, because WithCTE returns cteDefs as first children, and the defs will be traversed before the refs.
- def normalizeRuntimeReplaceable(plan: LogicalPlan): LogicalPlan
Normalize RuntimeReplaceable nodes by replacing them with their replacement expressions.
Normalize RuntimeReplaceable nodes by replacing them with their replacement expressions. This is necessary because fixed-point analyzer may produce non-deterministic results when resolving original expressions. For example, in a query like:
SELECT assert_true(1)Before resolution, we have UnresolvedFunction whose child is Literal(1). This child will first be converted to Cast(Literal(1), BooleanType) by type coercion. Because in this case Cast doesn't require timezone, the expression will be implicitly resolved. Because the child of initially unresolved function is resolved, the function can be converted to AssertTrue, which is of type InheritAnalysisRules. However, because the only child of InheritAnalysisRules is the replacement expression, the original expression will be lost and timezone will never be applied. This causes inconsistencies, because fixed-point semantic is to ALWAYS apply timezone, regardless of whether the Cast actually needs it.
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def outputWithNullability(output: Seq[Attribute], nonNullAttrExprIds: Seq[ExprId]): Seq[Attribute]
- Attributes
- protected
- Definition Classes
- PredicateHelper
- def replaceAlias(expr: Expression, aliasMap: AttributeMap[Alias]): Expression
Replace all attributes, that reference an alias, with the aliased expression
Replace all attributes, that reference an alias, with the aliased expression
- Attributes
- protected
- Definition Classes
- AliasHelper
- def replaceAliasButKeepName(expr: NamedExpression, aliasMap: AttributeMap[Alias]): NamedExpression
Replace all attributes, that reference an alias, with the aliased expression, but keep the name of the outermost attribute.
Replace all attributes, that reference an alias, with the aliased expression, but keep the name of the outermost attribute.
- Attributes
- protected
- Definition Classes
- AliasHelper
- def splitConjunctivePredicates(condition: Expression): Seq[Expression]
- Attributes
- protected
- Definition Classes
- PredicateHelper
- def splitDisjunctivePredicates(condition: Expression): Seq[Expression]
- Attributes
- protected
- Definition Classes
- PredicateHelper
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def trimAliases(e: Expression): Expression
- Attributes
- protected
- Definition Classes
- AliasHelper
- def trimNonTopLevelAliases[T <: Expression](e: T): T
- Attributes
- protected
- Definition Classes
- AliasHelper
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- def withLogContext(context: Map[String, String])(body: => Unit): Unit
- Attributes
- protected
- Definition Classes
- Logging
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)