Package org.apache.beam.sdk.extensions.sql.impl.rel

Class BeamSortRel

  • All Implemented Interfaces:
    java.lang.Cloneable, BeamRelNode, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptNode, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.hint.Hintable, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode
    public class BeamSortRel
extends org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort
implements BeamRelNode
    BeamRelNode to replace a Sort node.

    Since Beam does not fully support global sort, it uses Top to implement the Sort algebra. The following types of ORDER BY are supported: 

    
 SELECT * FROM t ORDER BY id DESC LIMIT 10;
 SELECT * FROM t ORDER BY id DESC LIMIT 10 OFFSET 5;

    but an ORDER BY without a LIMIT is NOT supported. For example, the following will throw an exception: 

    
 SELECT * FROM t ORDER BY id DESC;

    Constraints

    • Due to the constraints of Top, the result of a ORDER BY LIMIT must fit into the memory of a single machine.
    • Since WINDOW (HOP, TUMBLE, SESSION, etc.) is always associated with `GroupBy`, it does not make much sense to use ORDER BY with WINDOW.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  BeamSortRel.BeamSqlRowComparator  

      • Nested classes/interfaces inherited from interface org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode

        org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode.Context

    • Field Summary

      • Fields inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort

        collation, fetch, hints, offset

      • Fields inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.SingleRel

        input

      • Fields inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.AbstractRelNode

        digest, id, rowType, traitSet

    • Constructor Summary

      Constructors 
      Constructor Description
      BeamSortRel​(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptCluster cluster, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelTraitSet traits, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode child, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelCollation collation, @Nullable org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode offset, @Nullable org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode fetch)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      BeamCostModel beamComputeSelfCost​(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptPlanner planner, BeamRelMetadataQuery mq)
      This method is called by org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.NonCumulativeCostImpl.
      org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollectionList<org.apache.beam.sdk.values.Row>,​org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.Row>> buildPTransform()  
      org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort copy​(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelTraitSet traitSet, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode newInput, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelCollation newCollation, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode offset, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode fetch)  
      NodeStats estimateNodeStats​(BeamRelMetadataQuery mq)
      This method is called by org.apache.beam.sdk.extensions.sql.impl.planner.RelMdNodeStats.
      int getCount()  
      boolean isLimitOnly()  

      • Methods inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort

        accept, computeSelfCost, copy, copy, explainTerms, getCollation, getHints, getSortExps, isEnforcer

      • Methods inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.SingleRel

        childrenAccept, deriveRowType, estimateRowCount, getInput, getInputs, replaceInput

      • Methods inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.AbstractRelNode

        accept, collectVariablesSet, collectVariablesUsed, deepEquals, deepHashCode, equals, explain, getCluster, getConvention, getCorrelVariable, getDescription, getDigest, getExpectedInputRowType, getId, getInput, getRelDigest, getRelTypeName, getRowType, getTable, getTraitSet, getVariablesSet, hashCode, isValid, metadata, onRegister, recomputeDigest, register, sole, toString

      • Methods inherited from class java.lang.Object

        clone, finalize, getClass, notify, notifyAll, wait, wait, wait

      • Methods inherited from interface org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.hint.Hintable

        attachHints, withHints

      • Methods inherited from interface org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode

        accept, accept, childrenAccept, collectVariablesSet, collectVariablesUsed, computeSelfCost, copy, deepEquals, deepHashCode, estimateRowCount, explain, explain, fieldIsNullable, getConvention, getCorrelVariable, getDigest, getExpectedInputRowType, getInput, getInputs, getRelDigest, getRelTypeName, getRowType, getTable, getVariablesSet, isEnforcer, isValid, metadata, onRegister, recomputeDigest, register, replaceInput, stripped

      • Methods inherited from interface org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptNode

        getCluster, getDescription, getId, getTraitSet

    • Constructor Detail

      • BeamSortRel

        public BeamSortRel​(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptCluster cluster,
                   org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelTraitSet traits,
                   org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode child,
                   org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelCollation collation,
                   @Nullable org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode offset,
                   @Nullable org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode fetch)

    • Method Detail

      • estimateNodeStats

        public NodeStats estimateNodeStats​(BeamRelMetadataQuery mq)
        Description copied from interface: BeamRelNode
        This method is called by org.apache.beam.sdk.extensions.sql.impl.planner.RelMdNodeStats. This is currently only used in SQLTransform Path (and not JDBC path). When a RelNode wants to calculate its BeamCost or estimate its NodeStats, it may need NodeStat of its inputs. However, it should not call this directly (because maybe its inputs are not physical yet). It should call org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils#getNodeStats( org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.metadata.RelMetadataQuery) instead.
        Specified by:
        estimateNodeStats in interface BeamRelNode

      • beamComputeSelfCost

        public BeamCostModel beamComputeSelfCost​(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptPlanner planner,
                                         BeamRelMetadataQuery mq)
        Description copied from interface: BeamRelNode
        This method is called by org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.NonCumulativeCostImpl. This is currently only used in SQLTransform Path (and not JDBC path). This is needed when Calcite Query Planner wants to get the cost of a plan. Instead of calling this directly for a node, if we needed that it should be obtained by calling mq.getNonCumulativeCost. This way RelMetadataQuery will call this method instead of ComputeSelfCost if the handler is set correctly (see org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner#convertToBeamRel(String))
        Specified by:
        beamComputeSelfCost in interface BeamRelNode

      • isLimitOnly

        public boolean isLimitOnly()

      • getCount

        public int getCount()

      • buildPTransform

        public org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollectionList<org.apache.beam.sdk.values.Row>,​org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.Row>> buildPTransform()
        Specified by:
        buildPTransform in interface BeamRelNode

      • copy

        public org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort copy​(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelTraitSet traitSet,
                                                                                    org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode newInput,
                                                                                    org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelCollation newCollation,
                                                                                    org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode offset,
                                                                                    org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode fetch)
        Specified by:
        copy in class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort