org.apache.flink.table.planner.plan.optimize.program

Class FlinkDynamicPartitionPruningProgram

java.lang.Object

org.apache.flink.table.planner.plan.optimize.program.FlinkDynamicPartitionPruningProgram

public class FlinkDynamicPartitionPruningProgram
extends Object

Planner program that tries to do partition prune in the execution phase, which can translate a BatchPhysicalTableSourceScan to a BatchPhysicalDynamicFilteringTableSourceScan whose source is a partition source. The OptimizerConfigOptions.TABLE_OPTIMIZER_DYNAMIC_FILTERING_ENABLED need to be true.

Suppose we have the original physical plan:

 LogicalProject(...)
 HashJoin(joinType=[InnerJoin], where=[=(fact_partition_key, dim_key)], select=[xxx])
  * :- TableSourceScan(table=[[fact]], fields=[xxx, fact_partition_key],) # Is a partition table.
  * +- Exchange(distribution=[broadcast])
  *    +- Calc(select=[xxx], where=[<(xxx, xxx)]) # Need have an arbitrary filter condition.
  *       +- TableSourceScan(table=[[dim, filter=[]]], fields=[xxx, dim_key])
 

This physical plan will be rewritten to:

 HashJoin(joinType=[InnerJoin], where=[=(fact_partition_key, dim_key)], select=[xxx])
 :- DynamicFilteringTableSourceScan(table=[[fact]], fields=[xxx, fact_partition_key]) # Is a partition table.
 :  +- DynamicFilteringDataCollector(fields=[dim_key])
 :     +- Calc(select=[xxx], where=[<(xxx, xxx)])
 :        +- TableSourceScan(table=[[dim, filter=[]]], fields=[xxx, dim_key])
 +- Exchange(distribution=[broadcast])
    +- Calc(select=[xxx], where=[<(xxx, xxx)]) # Need have an arbitrary filter condition.
       +- TableSourceScan(table=[[dim, filter=[]]], fields=[xxx, dim_key])
 

Note: We use a FlinkOptimizeProgram instead of a RelRule here because the HepPlanner doesn't support matching a partially determined pattern or dynamically replacing the inputs of matched nodes. Once we improve the HepPlanner, then class can be converted to RelRule.

Constructor Summary

Constructors

Constructor and Description
FlinkDynamicPartitionPruningProgram()

Method Summary

Modifier and Type	Method and Description
org.apache.calcite.rel.RelNode	optimize(org.apache.calcite.rel.RelNode root, BatchOptimizeContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FlinkDynamicPartitionPruningProgram

public FlinkDynamicPartitionPruningProgram()

Method Detail

optimize

public org.apache.calcite.rel.RelNode optimize(org.apache.calcite.rel.RelNode root,
                                               BatchOptimizeContext context)