org.apache.flink.connector.base.source.hybrid

Class HybridSourceSplitEnumerator

java.lang.Object

org.apache.flink.connector.base.source.hybrid.HybridSourceSplitEnumerator

All Implemented Interfaces:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

public class HybridSourceSplitEnumerator
extends Object
implements org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

Wraps the actual split enumerators and facilitates source switching. Enumerators are created lazily when source switch occurs to support runtime position conversion.

This enumerator delegates to the current underlying split enumerator and transitions to the next source once all readers have indicated via SourceReaderFinishedEvent that all input was consumed.

Switching between enumerators occurs by creating the new enumerator via Source.createEnumerator(SplitEnumeratorContext). The start position can be fixed at pipeline construction time through the source or supplied at switch time through a converter function by using the end state of the previous enumerator.

During subtask recovery, splits that have been assigned since the last checkpoint will be added back by the source coordinator. These splits may originate from a previous enumerator that is no longer active. In that case HybridSourceSplitEnumerator will suspend forwarding to the current enumerator and replay the returned splits by activating the previous readers. After returned splits were processed, delegation to the current underlying enumerator resumes.

Constructor Summary

Constructors

Constructor and Description
HybridSourceSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<HybridSourceSplit> context, List<org.apache.flink.connector.base.source.hybrid.HybridSource.SourceListEntry> sources, int initialSourceIndex, HybridSourceEnumeratorState restoredEnumeratorState)

Method Summary

Modifier and Type	Method and Description
void	addReader(int subtaskId)
void	addSplitsBack(List<HybridSourceSplit> splits, int subtaskId)
void	close()
void	handleSourceEvent(int subtaskId, org.apache.flink.api.connector.source.SourceEvent sourceEvent)
void	handleSplitRequest(int subtaskId, String requesterHostname)
void	notifyCheckpointAborted(long checkpointId)
void	notifyCheckpointComplete(long checkpointId)
HybridSourceEnumeratorState	snapshotState(long checkpointId)
void	start()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HybridSourceSplitEnumerator

public HybridSourceSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<HybridSourceSplit> context,
                                   List<org.apache.flink.connector.base.source.hybrid.HybridSource.SourceListEntry> sources,
                                   int initialSourceIndex,
                                   HybridSourceEnumeratorState restoredEnumeratorState)

Method Detail

start

public void start()

Specified by:

start in interface org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

handleSplitRequest

public void handleSplitRequest(int subtaskId,
                               String requesterHostname)

Specified by:

handleSplitRequest in interface org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

addSplitsBack

public void addSplitsBack(List<HybridSourceSplit> splits,
                          int subtaskId)

Specified by:

addSplitsBack in interface org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

addReader

public void addReader(int subtaskId)

Specified by:

addReader in interface org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

snapshotState

public HybridSourceEnumeratorState snapshotState(long checkpointId)
                                          throws Exception

Specified by:

snapshotState in interface org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

Throws:

Exception

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

Specified by:

notifyCheckpointComplete in interface org.apache.flink.api.common.state.CheckpointListener

Specified by:

notifyCheckpointComplete in interface org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

Throws:

Exception

notifyCheckpointAborted

public void notifyCheckpointAborted(long checkpointId)
                             throws Exception

Specified by:

notifyCheckpointAborted in interface org.apache.flink.api.common.state.CheckpointListener

Throws:

Exception

handleSourceEvent

public void handleSourceEvent(int subtaskId,
                              org.apache.flink.api.connector.source.SourceEvent sourceEvent)

Specified by:

handleSourceEvent in interface org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface org.apache.flink.api.connector.source.SplitEnumerator<HybridSourceSplit,HybridSourceEnumeratorState>

Throws:

IOException