Package org.apache.beam.runners.spark

Class SparkNativePipelineVisitor

java.lang.Object

org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

org.apache.beam.runners.spark.SparkRunner.Evaluator

org.apache.beam.runners.spark.SparkNativePipelineVisitor

All Implemented Interfaces:

org.apache.beam.sdk.Pipeline.PipelineVisitor

public class SparkNativePipelineVisitor
extends SparkRunner.Evaluator

Pipeline visitor for translating a Beam pipeline into equivalent Spark operations. Used for debugging purposes using SparkRunnerDebugger.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.Pipeline.PipelineVisitor

org.apache.beam.sdk.Pipeline.PipelineVisitor.CompositeBehavior, org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

Field Summary

Fields inherited from class org.apache.beam.runners.spark.SparkRunner.Evaluator

ctxt, translator

Method Summary

Modifier and Type	Method	Description
org.apache.beam.sdk.Pipeline.PipelineVisitor.CompositeBehavior	enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Methods inherited from class org.apache.beam.runners.spark.SparkRunner.Evaluator

isBoundedCollection, shouldDefer, translate, visitPrimitiveTransform

Methods inherited from class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

enterPipeline, getPipeline, leaveCompositeTransform, leavePipeline, visitValue

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

enterCompositeTransform

public org.apache.beam.sdk.Pipeline.PipelineVisitor.CompositeBehavior enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Specified by:

enterCompositeTransform in interface org.apache.beam.sdk.Pipeline.PipelineVisitor

Overrides:

enterCompositeTransform in class SparkRunner.Evaluator