org.apache.flink.runtime.source.coordinator

Class SplitAssignmentTracker<SplitT extends org.apache.flink.api.connector.source.SourceSplit>

java.lang.Object

org.apache.flink.runtime.source.coordinator.SplitAssignmentTracker<SplitT>

@Internal
public class SplitAssignmentTracker<SplitT extends org.apache.flink.api.connector.source.SourceSplit>
extends Object

A class that is responsible for tracking the past split assignments made by SplitEnumerator.

Constructor Summary

Constructors

Constructor and Description
SplitAssignmentTracker()

Method Summary

Modifier and Type	Method and Description
List<SplitT>	getAndRemoveUncheckpointedAssignment(int subtaskId, long restoredCheckpointId) This method is invoked when a source reader fails over.
void	onCheckpoint(long checkpointId) Behavior of SplitAssignmentTracker on checkpoint.
void	onCheckpointComplete(long checkpointId) when a checkpoint has been successfully made, this method is invoked to clean up the assignment history before this successful checkpoint.
void	recordSplitAssignment(org.apache.flink.api.connector.source.SplitsAssignment<SplitT> splitsAssignment) Record a new split assignment.
void	restoreState(org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer, byte[] assignmentData) Restore the state of the SplitAssignmentTracker.
byte[]	snapshotState(org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer) Take a snapshot of the split assignments.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SplitAssignmentTracker

public SplitAssignmentTracker()

Method Detail

onCheckpoint

public void onCheckpoint(long checkpointId)
                  throws Exception

Behavior of SplitAssignmentTracker on checkpoint. Tracker will mark uncheckpointed assignment as checkpointed with current checkpoint ID.

Parameters:

checkpointId - the id of the ongoing checkpoint

Throws:

Exception

snapshotState

public byte[] snapshotState(org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer)
                     throws Exception

Take a snapshot of the split assignments.

Throws:

Exception

restoreState

public void restoreState(org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer,
                         byte[] assignmentData)
                  throws Exception

Restore the state of the SplitAssignmentTracker.

Parameters:

splitSerializer - The serializer of the splits.

assignmentData - The state of the SplitAssignmentTracker.

Throws:

Exception - when the state deserialization fails.

onCheckpointComplete

public void onCheckpointComplete(long checkpointId)

when a checkpoint has been successfully made, this method is invoked to clean up the assignment history before this successful checkpoint.

Parameters:

checkpointId - the id of the successful checkpoint.

recordSplitAssignment

public void recordSplitAssignment(org.apache.flink.api.connector.source.SplitsAssignment<SplitT> splitsAssignment)

Record a new split assignment.

Parameters:

splitsAssignment - the new split assignment.

getAndRemoveUncheckpointedAssignment

public List<SplitT> getAndRemoveUncheckpointedAssignment(int subtaskId,
                                                         long restoredCheckpointId)

This method is invoked when a source reader fails over. In this case, the source reader will restore its split assignment to the last successful checkpoint. Any split assignment to that source reader after the last successful checkpoint will be lost on the source reader side as if those splits were never assigned. To handle this case, the coordinator needs to find those splits and return them back to the SplitEnumerator for re-assignment.

Parameters:

subtaskId - the subtask id of the reader that failed over.

restoredCheckpointId - the ID of the checkpoint that the reader was restored to.

Returns:

A list of splits that needs to be added back to the SplitEnumerator.