org.apache.spark.sql.catalyst.analysis.resolver
ExpressionResolver
Companion object ExpressionResolver
class ExpressionResolver extends TreeNodeResolver[Expression, Expression] with ProducesUnresolvedSubtree with ResolvesExpressionChildren
The ExpressionResolver is used by the Resolver during the analysis to resolve expressions.
The functions here generally traverse unresolved Expression nodes recursively, constructing and returning the resolved Expression nodes bottom-up. This is the primary entry point for implementing expression analysis, wherein the resolve method accepts a fully unresolved Expression and returns a fully resolved Expression in response with all data types and attribute reference ID assigned for valid requests. This resolver also takes responsibility to detect any errors in the initial SQL query or DataFrame and return appropriate error messages including precise parse locations wherever possible.
- Alphabetic
- By Inheritance
- ExpressionResolver
- ProducesUnresolvedSubtree
- ResolvesExpressionChildren
- TreeNodeResolver
- QueryErrorsBase
- DataTypeErrorsBase
- SQLConfHelper
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new ExpressionResolver(resolver: Resolver, functionResolution: FunctionResolution, planLogger: PlanLogger)
- resolver
Resolver is passed from the parent to resolve other operators which are nested in expressions.
- functionResolution
FunctionResolution to resolve function expressions.
- planLogger
PlanLogger to log expression tree resolution events.
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def conf: SQLConf
The active config object within the current scope.
The active config object within the current scope. See SQLConf.get for more information.
- Definition Classes
- SQLConfHelper
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def getExpressionIdAssigner: ExpressionIdAssigner
- def getExpressionResolutionContextStack: ArrayDeque[ExpressionResolutionContext]
Get the expression resolution context stack.
- def getNameScopes: NameScopeStack
Get NameScopeStack bound to the used Resolver.
- def getParentOperator: Option[LogicalPlan]
Get the most recent operator (bottommost) from the
parentOperatorsstack. - def getQueryContext(context: QueryContext): Array[QueryContext]
- Definition Classes
- DataTypeErrorsBase
- def getSummary(sqlContext: QueryContext): String
- Definition Classes
- DataTypeErrorsBase
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def ordinalNumber(i: Int): String
- Definition Classes
- QueryErrorsBase
- def quoteByDefault(elem: String): String
- Attributes
- protected
- Definition Classes
- DataTypeErrorsBase
- def resolve(unresolvedExpression: Expression): Expression
This method is an expression analysis entry point.
This method is an expression analysis entry point. The method first checks if the expression has already been resolved (necessary because of partially-unresolved subtrees, see ProducesUnresolvedSubtree). If not already resolved, method takes an unresolved Expression and chooses the right
resolve*method using pattern matching on theunresolvedExpressiontype. This pattern matching enumerates all the expression node types that are supported by the single-pass analysis. When developers introduce a new Expression type to the Catalyst, they should implement a correspondingresolve*method in the ExpressionResolver and add it to this pattern match list.resolve will be called recursively during the expression tree traversal eventually producing a fully resolved expression subtree or a descriptive error message.
resolve can recursively call
resolverto resolve nested operators (e.g. scalar subqueries):SELECT * FROM VALUES (1), (2) WHERE col1 IN (SELECT 1);
In this case
INis an expression andSELECT 1is a nested operator tree for which the ExpressionResolver would invoke the Resolver.- Definition Classes
- ExpressionResolver → TreeNodeResolver
- def resolveExpressionGenericallyWithTypeCoercion(expression: Expression): Expression
Resolves Expression by resolving its children and applying generic type coercion transformations to the resulting expression.
Resolves Expression by resolving its children and applying generic type coercion transformations to the resulting expression. This resolution method is used for nodes that require type coercion on top of resolveExpressionGenerically.
- def resolveExpressionTreeInOperator(unresolvedExpression: Expression, parentOperator: LogicalPlan): Expression
Resolve
unresolvedExpressionwhich is a child ofparentOperator.Resolve
unresolvedExpressionwhich is a child ofparentOperator. This is the main entry point into the ExpressionResolver for operators. - def resolveLimitExpression(unresolvedLimitExpr: Expression, unresolvedLimit: LogicalPlan): Expression
Resolve the limit expression from either a LocalLimit or a GlobalLimit operator.
- def resolveProjectList(unresolvedProjectList: Seq[NamedExpression], operator: LogicalPlan): ResolvedProjectList
The Project list can contain different unresolved expressions before the resolution, which will be resolved using generic resolve.
The Project list can contain different unresolved expressions before the resolution, which will be resolved using generic resolve. However, UnresolvedStar is a special case, because it is expanded into a sequence of NamedExpressions. Because of that this method returns a sequence and doesn't conform to generic resolve interface - it's called directly from the Resolver during Project resolution.
The output sequence can be larger than the input sequence due to UnresolvedStar expansion.
- returns
The list of resolved expressions along with flags indicating whether the resolved project list contains aggregate expressions or attributes (encapsulated in ResolvedProjectList) which are used during the further resolution of the tree. The following query:
SELECT COUNT(col1), 2 FROM VALUES(1);
would have a project list with two expressions:
COUNT(col1)and2. After the resolution it would return the following result: ResolvedProjectList( expressions = [count(col1) as count(col1), 2 AS 2], hasAggregateExpressions = true, // because it containscount(col1)in the project list hasAttributes = false // because it doesn't contain any AttributeReferences in the // project list (only under the aggregate expression, please check // AggregateExpressionResolver for more details).
- def resolveStar(unresolvedStar: UnresolvedStar): Seq[NamedExpression]
UnresolvedStar resolution relies on the NameScope's ability to get the attributes by a multipart name (UnresolvedStar's
targetfield):UnresolvedStar resolution relies on the NameScope's ability to get the attributes by a multipart name (UnresolvedStar's
targetfield):- Star target is defined:
SELECT t.* FROM VALUES (1) AS t; -> Project [col1#19]
- Star target is not defined:
SELECT * FROM (SELECT 1 as col1), (SELECT 2 as col2); -> Project [col1#19, col2#20]
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toDSOption(option: String): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLConf(conf: String): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLConfVal(conf: String): String
- Definition Classes
- QueryErrorsBase
- def toSQLExpr(e: Expression): String
- Definition Classes
- QueryErrorsBase
- def toSQLId(parts: Seq[String]): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLId(parts: String): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLStmt(text: String): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLType(t: AbstractDataType): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLType(text: String): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLValue(v: Any, t: DataType): String
- Definition Classes
- QueryErrorsBase
- def toSQLValue(value: Double): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLValue(value: Float): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLValue(value: Long): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLValue(value: Int): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLValue(value: Short): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLValue(value: UTF8String): String
- Definition Classes
- DataTypeErrorsBase
- def toSQLValue(value: String): String
- Definition Classes
- DataTypeErrorsBase
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- def withResolvedChildren[ExpressionType <: Expression](unresolvedExpression: ExpressionType, resolveChild: (Expression) => Expression): ExpressionType
Resolves generic Expression children and returns its copy with children resolved.
Resolves generic Expression children and returns its copy with children resolved.
- Attributes
- protected
- Definition Classes
- ResolvesExpressionChildren
- def withResolvedSubtree(expression: Expression, expressionResolver: (Expression) => Expression)(body: => Expression): Expression
Helper method used to resolve a subtree that is generated as part of the resolution of some node.
Helper method used to resolve a subtree that is generated as part of the resolution of some node. Method ensures that the downwards traversal never visits previously resolved nodes by tracking the limits of the traversal with a tag. Invokes a resolver callback to resolve children, but DOES NOT resolve the root of the subtree.
- Attributes
- protected
- Definition Classes
- ProducesUnresolvedSubtree
- def withSQLConf[T](pairs: (String, String)*)(f: => T): T
Sets all SQL configurations specified in
pairs, callsf, and then restores all SQL configurations.Sets all SQL configurations specified in
pairs, callsf, and then restores all SQL configurations.- Attributes
- protected
- Definition Classes
- SQLConfHelper
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)