Package org.apache.beam.runners.spark.translation.streaming

Class StatefulStreamingParDoEvaluator<KeyT,​ValueT,​OutputT>

  • All Implemented Interfaces:
    java.io.Serializable, TransformEvaluator<org.apache.beam.sdk.transforms.ParDo.MultiOutput<org.apache.beam.sdk.values.KV<KeyT,​ValueT>,​OutputT>>
    public class StatefulStreamingParDoEvaluator<KeyT,​ValueT,​OutputT>
extends java.lang.Object
implements TransformEvaluator<org.apache.beam.sdk.transforms.ParDo.MultiOutput<org.apache.beam.sdk.values.KV<KeyT,​ValueT>,​OutputT>>
    A specialized evaluator for ParDo operations in Spark Streaming context that is invoked when stateful streaming is detected in the DoFn.

    This class is used by StreamingTransformTranslator's ParDo evaluator to handle stateful streaming operations. When a DoFn contains stateful processing logic, the translation process routes the execution through this evaluator instead of the standard ParDo evaluator.

    The evaluator manages state handling and ensures proper processing semantics for streaming stateful operations in the Spark runner context.

    Important: This evaluator includes validation logic that rejects DoFn implementations containing @Timer annotations, as timer functionality is not currently supported in the Spark streaming context.

    See Also:
    Serialized Form