Class AbstractInOutIterator<K,InputT,OutputT>
- java.lang.Object
-
- org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.UnmodifiableIterator<T>
-
- org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.AbstractIterator<scala.Tuple2<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.values.WindowedValue<?>>>
-
- org.apache.beam.runners.spark.translation.AbstractInOutIterator<K,InputT,OutputT>
-
- Type Parameters:
K- The key type for the processing contextInputT- The input element type to be processedOutputT- The output element type after processing
- All Implemented Interfaces:
java.util.Iterator<scala.Tuple2<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.values.WindowedValue<?>>>
public abstract class AbstractInOutIterator<K,InputT,OutputT> extends org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.AbstractIterator<scala.Tuple2<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.values.WindowedValue<?>>>Abstract base class for iterators that process Spark input data and produce corresponding output values. This class serves as a common base for both bounded and unbounded processing strategies in the Spark runner.The class extends Guava's
AbstractIteratorand provides common functionality for iterating through input elements, processing them using a DoFnRunner, and producing output elements as tuples ofTupleTagandWindowedValuepairs.
-
-
Field Summary
Fields Modifier and Type Field Description protected SparkProcessContext<K,InputT,OutputT>ctx
-
Constructor Summary
Constructors Modifier Constructor Description protectedAbstractInOutIterator(SparkProcessContext<K,InputT,OutputT> ctx)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidfireTimer(org.apache.beam.runners.core.TimerInternals.TimerData timer)Fires a timer using the DoFnRunner from the context and performs cleanup afterwards.-
Methods inherited from class org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.AbstractIterator
computeNext, endOfData, hasNext, next, peek
-
Methods inherited from class org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.UnmodifiableIterator
remove
-
-
-
-
Field Detail
-
ctx
protected final SparkProcessContext<K,InputT,OutputT> ctx
-
-
Constructor Detail
-
AbstractInOutIterator
protected AbstractInOutIterator(SparkProcessContext<K,InputT,OutputT> ctx)
-
-
Method Detail
-
fireTimer
public void fireTimer(org.apache.beam.runners.core.TimerInternals.TimerData timer)
Fires a timer using the DoFnRunner from the context and performs cleanup afterwards.After firing the timer, if the timer data iterator is an instance of
ParDoStateUpdateFn.SparkTimerInternalsIterator, the fired timer will be deleted as part of cleanup to prevent re-firing of the same timer.- Parameters:
timer- The timer data containing information about the timer to fire- Throws:
java.lang.IllegalArgumentException- If the timer namespace is not aStateNamespaces.WindowNamespace
-
-