Package org.apache.flink.streaming.api.transformations

Class StubTransformation<T>

  • @Internal
public class StubTransformation<T>
extends org.apache.flink.api.dag.Transformation<T>
    A stub transformation that is later connected to an existing Transformation. This implementation is useful when the full DAG of transformations cannot be constructed at once. For example, an SQL query may not be fully defined at the time of its creation and later refined (e.g., connecting side outputs).

    Usage:

    To create a stub which will later connect to an upstream operator, use create(TypeInformation, Predicate): 

    
 TypeInformation<T> typeInfo = ...; // expected output type of the upstream transformation
 StubTransformation<T> inputStub = StubTransformation.create(typeInfo,
     t -> t.getName().equals("decoupled-transformation"));
 Transformation<T> output = new PartitionTransformation<>(inputStub, new ShufflePartitioner<T>());
 ...

    The upstream part that may be created before or after the stub: 

    
 Transformation<T> input = new OneInputTransformation<>(typeInfo, "decoupled-transformation", ...);
 ... // independent subtopology

    While translating the Transformation, the stub will find one or more upstream transformations based on the predicate provided in the StubTransformation and implicitly connect upstreams to the downstream transformations of the StubTransformation which effectively means that it replaces itself with the union (all) of the upstream transformations.

    • Field Summary

      • Fields inherited from class org.apache.flink.api.dag.Transformation

        bufferTimeout, description, id, name, outputType, typeUsed, UPPER_BOUND_MAX_PARALLELISM

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      static <T> StubTransformation<T> create​(org.apache.flink.api.common.typeinfo.TypeInformation<T> typeInformation, Predicate<org.apache.flink.api.dag.Transformation<?>> upstreamFinder)
      Creates a new StubTransformation with the given type information and predicate to find the upstream transformation.
      static <T> StubTransformation<T> create​(org.apache.flink.api.common.typeinfo.TypeInformation<T> typeInformation, Predicate<org.apache.flink.api.dag.Transformation<?>> upstreamFinder, Function<org.apache.flink.api.dag.Transformation<?>,​org.apache.flink.api.dag.Transformation<?>> inputAdjuster)
      Creates a new StubTransformation with the given type information and predicate to find the upstream transformation and additionally allows the upstream transformation to be adjusted before connecting it to the downstream transformations.
      Function<org.apache.flink.api.dag.Transformation<?>,​org.apache.flink.api.dag.Transformation<?>> getInputAdjuster()  
      List<org.apache.flink.api.dag.Transformation<?>> getInputs()  
      protected List<org.apache.flink.api.dag.Transformation<?>> getTransitivePredecessorsInternal()  
      Predicate<org.apache.flink.api.dag.Transformation<?>> getUpstreamFinder()  

      • Methods inherited from class org.apache.flink.api.dag.Transformation

        declareManagedMemoryUseCaseAtOperatorScope, declareManagedMemoryUseCaseAtSlotScope, enableAsyncState, equals, getAttribute, getBufferTimeout, getCoLocationGroupKey, getDescription, getId, getManagedMemoryOperatorScopeUseCaseWeights, getManagedMemorySlotScopeUseCases, getMaxParallelism, getMinResources, getName, getNewNodeId, getOutputType, getParallelism, getPreferredResources, getSlotSharingGroup, getTransitivePredecessors, getUid, getUserProvidedNodeHash, hashCode, isParallelismConfigured, setAttribute, setBufferTimeout, setCoLocationGroupKey, setDescription, setMaxParallelism, setName, setOutputType, setParallelism, setParallelism, setResources, setSlotSharingGroup, setSlotSharingGroup, setUid, setUidHash, toString, updateManagedMemoryStateBackendUseCase

    • Method Detail

      • create

        public static <T> StubTransformation<T> create​(org.apache.flink.api.common.typeinfo.TypeInformation<T> typeInformation,
                                               Predicate<org.apache.flink.api.dag.Transformation<?>> upstreamFinder)
        Creates a new StubTransformation with the given type information and predicate to find the upstream transformation.

      • create

        public static <T> StubTransformation<T> create​(org.apache.flink.api.common.typeinfo.TypeInformation<T> typeInformation,
                                               Predicate<org.apache.flink.api.dag.Transformation<?>> upstreamFinder,
                                               Function<org.apache.flink.api.dag.Transformation<?>,​org.apache.flink.api.dag.Transformation<?>> inputAdjuster)
        Creates a new StubTransformation with the given type information and predicate to find the upstream transformation and additionally allows the upstream transformation to be adjusted before connecting it to the downstream transformations. A common use case for the adjuster is to attach the downstream transformations to the side-output of the upstream transformation.

      • getInputAdjuster

        public Function<org.apache.flink.api.dag.Transformation<?>,​org.apache.flink.api.dag.Transformation<?>> getInputAdjuster()

      • getUpstreamFinder

        public Predicate<org.apache.flink.api.dag.Transformation<?>> getUpstreamFinder()

      • getTransitivePredecessorsInternal

        protected List<org.apache.flink.api.dag.Transformation<?>> getTransitivePredecessorsInternal()
        Specified by:
        getTransitivePredecessorsInternal in class org.apache.flink.api.dag.Transformation<T>

      • getInputs

        public List<org.apache.flink.api.dag.Transformation<?>> getInputs()
        Specified by:
        getInputs in class org.apache.flink.api.dag.Transformation<T>