org.apache.flink.runtime.source.coordinator

Class SourceCoordinatorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

java.lang.Object

org.apache.flink.runtime.source.coordinator.SourceCoordinatorContext<SplitT>

Type Parameters:

SplitT - the type of the splits.

All Implemented Interfaces:

AutoCloseable, org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT>, org.apache.flink.api.connector.source.SupportsIntermediateNoMoreSplits

@Internal
public class SourceCoordinatorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>
extends Object
implements org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT>, org.apache.flink.api.connector.source.SupportsIntermediateNoMoreSplits, AutoCloseable

A context class for the OperatorCoordinator. Compared with SplitEnumeratorContext this class allows interaction with state and sending OperatorEvent to the SourceOperator while SplitEnumeratorContext only allows sending SourceEvent.

The context serves a few purposes:

• Information provider - The context provides necessary information to the enumerator for it to know what is the status of the source readers and their split assignments. These information allows the split enumerator to do the coordination.
• Action taker - The context also provides a few actions that the enumerator can take to carry out the coordination. So far there are two actions: 1) assign splits to the source readers. and 2) sens a custom SourceEvents to the source readers.
• Thread model enforcement - The context ensures that all the manipulations to the coordinator state are handled by the same thread.

Constructor Summary

Constructors

Constructor and Description
SourceCoordinatorContext(SourceCoordinatorProvider.CoordinatorExecutorThreadFactory coordinatorThreadFactory, int numWorkerThreads, OperatorCoordinator.Context operatorCoordinatorContext, org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer, boolean supportsConcurrentExecutionAttempts)

Method Summary

Modifier and Type	Method and Description
void	assignSplits(org.apache.flink.api.connector.source.SplitsAssignment<SplitT> assignment)
<T> void	callAsync(Callable<T> callable, java.util.function.BiConsumer<T,Throwable> handler)
<T> void	callAsync(Callable<T> callable, java.util.function.BiConsumer<T,Throwable> handler, long initialDelay, long period)
void	close()
int	currentParallelism()
org.apache.flink.util.TernaryBoolean	isBacklog() Returns whether the Source is processing backlog data.
org.apache.flink.metrics.groups.SplitEnumeratorMetricGroup	metricGroup()
Map<Integer,org.apache.flink.api.connector.source.ReaderInfo>	registeredReaders()
Map<Integer,Map<Integer,org.apache.flink.api.connector.source.ReaderInfo>>	registeredReadersOfAttempts()
void	runInCoordinatorThread(Runnable runnable) If the runnable throws an Exception, the corresponding job is failed.
void	sendEventToSourceReader(int subtaskId, int attemptNumber, org.apache.flink.api.connector.source.SourceEvent event)
void	sendEventToSourceReader(int subtaskId, org.apache.flink.api.connector.source.SourceEvent event)
void	setIsProcessingBacklog(boolean isProcessingBacklog)
void	signalIntermediateNoMoreSplits(int subtask)
void	signalNoMoreSplits(int subtask)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.connector.source.SplitEnumeratorContext

assignSplit

Constructor Detail

SourceCoordinatorContext

public SourceCoordinatorContext(SourceCoordinatorProvider.CoordinatorExecutorThreadFactory coordinatorThreadFactory,
                                int numWorkerThreads,
                                OperatorCoordinator.Context operatorCoordinatorContext,
                                org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer,
                                boolean supportsConcurrentExecutionAttempts)

Method Detail

metricGroup

public org.apache.flink.metrics.groups.SplitEnumeratorMetricGroup metricGroup()

Specified by:

metricGroup in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

sendEventToSourceReader

public void sendEventToSourceReader(int subtaskId,
                                    org.apache.flink.api.connector.source.SourceEvent event)

Specified by:

sendEventToSourceReader in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

sendEventToSourceReader

public void sendEventToSourceReader(int subtaskId,
                                    int attemptNumber,
                                    org.apache.flink.api.connector.source.SourceEvent event)

Specified by:

sendEventToSourceReader in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

currentParallelism

public int currentParallelism()

Specified by:

currentParallelism in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

registeredReaders

public Map<Integer,org.apache.flink.api.connector.source.ReaderInfo> registeredReaders()

Specified by:

registeredReaders in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

registeredReadersOfAttempts

public Map<Integer,Map<Integer,org.apache.flink.api.connector.source.ReaderInfo>> registeredReadersOfAttempts()

Specified by:

registeredReadersOfAttempts in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

assignSplits

public void assignSplits(org.apache.flink.api.connector.source.SplitsAssignment<SplitT> assignment)

Specified by:

assignSplits in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

signalNoMoreSplits

public void signalNoMoreSplits(int subtask)

Specified by:

signalNoMoreSplits in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

signalIntermediateNoMoreSplits

public void signalIntermediateNoMoreSplits(int subtask)

Specified by:

signalIntermediateNoMoreSplits in interface org.apache.flink.api.connector.source.SupportsIntermediateNoMoreSplits

callAsync

public <T> void callAsync(Callable<T> callable,
                          java.util.function.BiConsumer<T,Throwable> handler,
                          long initialDelay,
                          long period)

Specified by:

callAsync in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

callAsync

public <T> void callAsync(Callable<T> callable,
                          java.util.function.BiConsumer<T,Throwable> handler)

Specified by:

callAsync in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

runInCoordinatorThread

public void runInCoordinatorThread(Runnable runnable)

If the runnable throws an Exception, the corresponding job is failed.

Specified by:

runInCoordinatorThread in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

close

public void close()
           throws InterruptedException

Specified by:

close in interface AutoCloseable

Throws:

InterruptedException

setIsProcessingBacklog

public void setIsProcessingBacklog(boolean isProcessingBacklog)

Specified by:

setIsProcessingBacklog in interface org.apache.flink.api.connector.source.SplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

isBacklog

public org.apache.flink.util.TernaryBoolean isBacklog()

Returns whether the Source is processing backlog data. UNDEFINED is returned if it is not set by the setIsProcessingBacklog(boolean) method.