Package org.apache.beam.runners.spark.translation.streaming

Internal utilities to translate Beam pipelines to Spark streaming.

Class Summary

Class	Description
Checkpoint	Checkpoint data to make it available in future pipeline runs.
Checkpoint.CheckpointDir	Checkpoint dir tree.
CreateStreamingSparkView<ElemT,ViewT>	Spark streaming overrides for various view (side input) transforms.
CreateStreamingSparkView.CreateSparkPCollectionView<ElemT,ViewT>	Creates a primitive PCollectionView.
CreateStreamingSparkView.Factory<ElemT,ViewT>
ParDoStateUpdateFn<KeyT,ValueT,InputT extends org.apache.beam.sdk.values.KV<KeyT,ValueT>,OutputT>	A function to handle stateful processing in Apache Beam's SparkRunner.
ParDoStateUpdateFn.SparkTimerInternalsIterator	An iterator implementation that processes timers from SparkTimerInternals.
SparkRunnerStreamingContextFactory	A JavaStreamingContext factory for resilience.
StatefulStreamingParDoEvaluator<KeyT,ValueT,OutputT>	A specialized evaluator for ParDo operations in Spark Streaming context that is invoked when stateful streaming is detected in the DoFn.
StreamingTransformTranslator	Supports translation between a Beam transform, and Spark's operations on DStreams.
StreamingTransformTranslator.SparkTransformsRegistrar	Registers classes specialized by the Spark runner.
StreamingTransformTranslator.Translator	Translator matches Beam transformation with the appropriate evaluator.
TestDStream<T>
UnboundedDataset<T>	DStream holder Can also crate a DStream from a supplied queue of values, but mainly for testing.