Package org.apache.beam.runners.jet.processors

Class StatefulParDoP<OutputT>

  • All Implemented Interfaces:
    com.hazelcast.jet.core.Processor
    public class StatefulParDoP<OutputT>
extends java.lang.Object
    Jet Processor implementation for Beam's stateful ParDo primitive.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  StatefulParDoP.Supplier<OutputT>
      Jet Processor supplier that will provide instances of StatefulParDoP.

      • Nested classes/interfaces inherited from interface com.hazelcast.jet.core.Processor

        com.hazelcast.jet.core.Processor.Context

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      boolean complete()  
      boolean completeEdge​(int ordinal)  
      protected void finishRunnerBundle​(org.apache.beam.runners.core.DoFnRunner<InputT,​OutputT> runner)  
      protected org.apache.beam.runners.core.DoFnRunner<org.apache.beam.sdk.values.KV<?,​?>,​OutputT> getDoFnRunner​(org.apache.beam.sdk.options.PipelineOptions pipelineOptions, org.apache.beam.sdk.transforms.DoFn<org.apache.beam.sdk.values.KV<?,​?>,​OutputT> doFn, org.apache.beam.runners.core.SideInputReader sideInputReader, org.apache.beam.runners.jet.processors.AbstractParDoP.JetOutputManager outputManager, org.apache.beam.sdk.values.TupleTag<OutputT> mainOutputTag, java.util.List<org.apache.beam.sdk.values.TupleTag<?>> additionalOutputTags, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.values.KV<?,​?>> inputValueCoder, java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,​org.apache.beam.sdk.coders.Coder<?>> outputValueCoders, org.apache.beam.sdk.values.WindowingStrategy<?,​?> windowingStrategy, org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation, java.util.Map<java.lang.String,​org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping)  
      void init​(com.hazelcast.jet.core.Outbox outbox, com.hazelcast.jet.core.Processor.Context context)  
      boolean isCooperative()  
      void process​(int ordinal, com.hazelcast.jet.core.Inbox inbox)  
      protected void processElementWithRunner​(org.apache.beam.runners.core.DoFnRunner<org.apache.beam.sdk.values.KV<?,​?>,​OutputT> runner, org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<?,​?>> windowedValue)  
      protected void startRunnerBundle​(org.apache.beam.runners.core.DoFnRunner<org.apache.beam.sdk.values.KV<?,​?>,​OutputT> runner)  
      boolean tryProcess()  
      boolean tryProcessWatermark​(com.hazelcast.jet.core.Watermark watermark)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface com.hazelcast.jet.core.Processor

        finishSnapshotRestore, restoreFromSnapshot, saveToSnapshot, snapshotCommitFinish, snapshotCommitPrepare

    • Method Detail

      • getDoFnRunner

        protected org.apache.beam.runners.core.DoFnRunner<org.apache.beam.sdk.values.KV<?,​?>,​OutputT> getDoFnRunner​(org.apache.beam.sdk.options.PipelineOptions pipelineOptions,
                                                                                                                        org.apache.beam.sdk.transforms.DoFn<org.apache.beam.sdk.values.KV<?,​?>,​OutputT> doFn,
                                                                                                                        org.apache.beam.runners.core.SideInputReader sideInputReader,
                                                                                                                        org.apache.beam.runners.jet.processors.AbstractParDoP.JetOutputManager outputManager,
                                                                                                                        org.apache.beam.sdk.values.TupleTag<OutputT> mainOutputTag,
                                                                                                                        java.util.List<org.apache.beam.sdk.values.TupleTag<?>> additionalOutputTags,
                                                                                                                        org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.values.KV<?,​?>> inputValueCoder,
                                                                                                                        java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,​org.apache.beam.sdk.coders.Coder<?>> outputValueCoders,
                                                                                                                        org.apache.beam.sdk.values.WindowingStrategy<?,​?> windowingStrategy,
                                                                                                                        org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation,
                                                                                                                        java.util.Map<java.lang.String,​org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping)

      • startRunnerBundle

        protected void startRunnerBundle​(org.apache.beam.runners.core.DoFnRunner<org.apache.beam.sdk.values.KV<?,​?>,​OutputT> runner)

      • processElementWithRunner

        protected void processElementWithRunner​(org.apache.beam.runners.core.DoFnRunner<org.apache.beam.sdk.values.KV<?,​?>,​OutputT> runner,
                                        org.apache.beam.sdk.values.WindowedValue<org.apache.beam.sdk.values.KV<?,​?>> windowedValue)

      • tryProcessWatermark

        public boolean tryProcessWatermark​(@Nonnull
                                   com.hazelcast.jet.core.Watermark watermark)
        Specified by:
        tryProcessWatermark in interface com.hazelcast.jet.core.Processor

      • complete

        public boolean complete()
        Specified by:
        complete in interface com.hazelcast.jet.core.Processor

      • init

        public void init​(@Nonnull
                 com.hazelcast.jet.core.Outbox outbox,
                 @Nonnull
                 com.hazelcast.jet.core.Processor.Context context)
        Specified by:
        init in interface com.hazelcast.jet.core.Processor

      • isCooperative

        public boolean isCooperative()
        Specified by:
        isCooperative in interface com.hazelcast.jet.core.Processor

      • close

        public void close()
        Specified by:
        close in interface com.hazelcast.jet.core.Processor

      • process

        public void process​(int ordinal,
                    @Nonnull
                    com.hazelcast.jet.core.Inbox inbox)
        Specified by:
        process in interface com.hazelcast.jet.core.Processor

      • finishRunnerBundle

        protected void finishRunnerBundle​(org.apache.beam.runners.core.DoFnRunner<InputT,​OutputT> runner)

      • tryProcess

        public boolean tryProcess()
        Specified by:
        tryProcess in interface com.hazelcast.jet.core.Processor

      • completeEdge

        public boolean completeEdge​(int ordinal)
        Specified by:
        completeEdge in interface com.hazelcast.jet.core.Processor