Package org.apache.beam.runners.spark

Class SparkRunner.Evaluator

java.lang.Object
org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults
org.apache.beam.runners.spark.SparkRunner.Evaluator
All Implemented Interfaces:
org.apache.beam.sdk.Pipeline.PipelineVisitor
Direct Known Subclasses:
SparkNativePipelineVisitor
Enclosing class:
SparkRunner
public static class SparkRunner.Evaluator extends org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults
Evaluator on the pipeline.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.beam.sdk.Pipeline.PipelineVisitor

    org.apache.beam.sdk.Pipeline.PipelineVisitor.CompositeBehavior, org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected final EvaluationContext
    ctxt
     
    protected final SparkPipelineTranslator
    translator
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    Evaluator(SparkPipelineTranslator translator, EvaluationContext ctxt)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.beam.sdk.Pipeline.PipelineVisitor.CompositeBehavior
    enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)
     
    protected org.apache.beam.sdk.values.PCollection.IsBounded
    isBoundedCollection(Collection<org.apache.beam.sdk.values.PCollection<?>> pValues)
     
    protected boolean
    shouldDefer(org.apache.beam.sdk.runners.TransformHierarchy.Node node)
     
    protected <TransformT extends org.apache.beam.sdk.transforms.PTransform<? super org.apache.beam.sdk.values.PInput, org.apache.beam.sdk.values.POutput>>
    TransformEvaluator<TransformT>
    translate(org.apache.beam.sdk.runners.TransformHierarchy.Node node, TransformT transform)
    Determine if this Node belongs to a Bounded branch of the pipeline, or Unbounded, and translate with the proper translator.
    void
    visitPrimitiveTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)
     

    Methods inherited from class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

    enterPipeline, getPipeline, leaveCompositeTransform, leavePipeline, visitValue

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

  • Method Details

    • enterCompositeTransform

      public org.apache.beam.sdk.Pipeline.PipelineVisitor.CompositeBehavior enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)
      Specified by:
      enterCompositeTransform in interface org.apache.beam.sdk.Pipeline.PipelineVisitor
      Overrides:
      enterCompositeTransform in class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

    • shouldDefer

      protected boolean shouldDefer(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

    • visitPrimitiveTransform

      public void visitPrimitiveTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)
      Specified by:
      visitPrimitiveTransform in interface org.apache.beam.sdk.Pipeline.PipelineVisitor
      Overrides:
      visitPrimitiveTransform in class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

    • translate

      protected <TransformT extends org.apache.beam.sdk.transforms.PTransform<? super org.apache.beam.sdk.values.PInput, org.apache.beam.sdk.values.POutput>> TransformEvaluator<TransformT> translate(org.apache.beam.sdk.runners.TransformHierarchy.Node node, TransformT transform)
      Determine if this Node belongs to a Bounded branch of the pipeline, or Unbounded, and translate with the proper translator.

    • isBoundedCollection

      protected org.apache.beam.sdk.values.PCollection.IsBounded isBoundedCollection(Collection<org.apache.beam.sdk.values.PCollection<?>> pValues)