This rule is the second phase to resolve lateral column alias.

Resolve lateral column alias, which references the alias defined previously in the SELECT list. Plan-wise, it handles two types of operators: Project and Aggregate. - in Project, pushing down the referenced lateral alias into a newly created Project, resolve the attributes referencing these aliases - in Aggregate, inserting the Project node above and falling back to the resolution of Project.

The whole process is generally divided into two phases: 1) recognize resolved lateral alias, wrap the attributes referencing them with LateralColumnAliasReference 2) when the whole operator is resolved, or contains Window but have all other resolved, For Project, it unwrap LateralColumnAliasReference, further resolves the attributes and push down the referenced lateral aliases. For Aggregate, it goes through the whole aggregation list, extracts the aggregation expressions and grouping expressions to keep them in this Aggregate node, and add a Project above with the original output. It doesn't do anything on LateralColumnAliasReference, but completely leave it to the Project in the future turns of this rule.

** Example for Project: Before rewrite: Project [age AS a, 'a + 1] +- Child

After phase 1: Project [age AS a, lca(a) + 1] +- Child

After phase 2: Project [a, a + 1] +- Project [child output, age AS a] +- Child

** Example for Aggregate: Before rewrite: Aggregate [dept#14] [dept#14 AS a#12, 'a + 1, avg(salary#16) AS b#13, 'b + avg(bonus#17)] +- Child [dept#14,name#15,salary#16,bonus#17]

After phase 1: Aggregate [dept#14] [dept#14 AS a#12, lca(a) + 1, avg(salary#16) AS b#13, lca(b) + avg(bonus#17)] +- Child [dept#14,name#15,salary#16,bonus#17]

After phase 2: Project [dept#14 AS a#12, lca(a) + 1, avg(salary)#26 AS b#13, lca(b) + avg(bonus)#27] +- Aggregate [dept#14] [avg(salary#16) AS avg(salary)#26, avg(bonus#17) AS avg(bonus)#27,dept#14] +- Child [dept#14,name#15,salary#16,bonus#17]

Now the problem falls back to the lateral alias resolution in Project. After future rounds of this rule: Project [a#12, a#12 + 1, b#13, b#13 + avg(bonus)#27] +- Project [dept#14 AS a#12, avg(salary)#26 AS b#13] +- Aggregate [dept#14] [avg(salary#16) AS avg(salary)#26, avg(bonus#17) AS avg(bonus)#27, dept#14] +- Child [dept#14,name#15,salary#16,bonus#17]

** Example for Window: Query: select dept as d, sum(salary) as s, avg(s) over (partition by s order by d) as avg from employee group by dept

After phase 1: 'Aggregate [dept#17], [dept#17 AS d#15, sum(salary#19) AS s#16L, avg(lca(s#16L)) windowspecdefinition(lca(s#16L), lca(d#15) ASC NULLS FIRST, specifiedwindowframe(..)) AS avg#25] +- Relation spark_catalog.default.employee[dept#17,name#18,salary#19,bonus#20,properties#21] It is similar to a regular Aggregate. All expressions in it are resolved, but itself is unresolved due to the Window expression. The rule allows appliction on this case.

After phase 2: 'Project [dept#17 AS d#15, sum(salary)#26L AS s#16L, avg(lca(s#16L)) windowspecdefinition(lca(s#16L), lca(d#15) ASC NULLS FIRST, specifiedwindowframe(..)) AS avg#25] +- Aggregate [dept#17], [dept#17, sum(salary#19) AS sum(salary)#26L] +- Relation spark_catalog.default.employee[dept#17,name#18,salary#19,bonus#20,properties#21] Same as Aggregate, it extracts grouping expressions and aggregate functions. Window expressions are completely lifted up to upper Project, free from the current Aggregate.

Then this rule will apply on the Project, adding another Project below. Till this phase, all lateral column alias references have been resolved and removed. Finally, rule ExtractWindowExpressions will apply on the top Project with window expressions. It is guaranteed that ResolveLateralColumnAliasReference is applied before ExtractWindowExpressions.