org.apache.flink.connector.file.src.impl

Class StaticFileSplitEnumerator

java.lang.Object

org.apache.flink.connector.file.src.impl.StaticFileSplitEnumerator

All Implemented Interfaces:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.connector.source.SplitEnumerator<FileSourceSplit,PendingSplitsCheckpoint<FileSourceSplit>>, org.apache.flink.api.connector.source.SupportsBatchSnapshot

@Internal
public class StaticFileSplitEnumerator
extends Object
implements org.apache.flink.api.connector.source.SplitEnumerator<FileSourceSplit,PendingSplitsCheckpoint<FileSourceSplit>>, org.apache.flink.api.connector.source.SupportsBatchSnapshot

A SplitEnumerator implementation for bounded / batch FileSource input.

This enumerator takes all files that are present in the configured input directories and assigns them to the readers. Once all files are processed, the source is finished.

The implementation of this class is rather thin. The actual logic for creating the set of FileSourceSplits to process, and the logic to decide which reader gets what split, are in FileEnumerator and in FileSplitAssigner, respectively.

Constructor Summary

Constructors

Constructor and Description
StaticFileSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<FileSourceSplit> context, FileSplitAssigner splitAssigner)

Method Summary

Modifier and Type	Method and Description
void	addReader(int subtaskId)
void	addSplitsBack(List<FileSourceSplit> splits, int subtaskId)
void	close()
void	handleSourceEvent(int subtaskId, org.apache.flink.api.connector.source.SourceEvent sourceEvent)
void	handleSplitRequest(int subtask, String hostname)
PendingSplitsCheckpoint<FileSourceSplit>	snapshotState(long checkpointId)
void	start()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.connector.source.SplitEnumerator

notifyCheckpointComplete

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

Constructor Detail

StaticFileSplitEnumerator

public StaticFileSplitEnumerator(org.apache.flink.api.connector.source.SplitEnumeratorContext<FileSourceSplit> context,
                                 FileSplitAssigner splitAssigner)

Method Detail

start

public void start()

Specified by:

start in interface org.apache.flink.api.connector.source.SplitEnumerator<FileSourceSplit,PendingSplitsCheckpoint<FileSourceSplit>>

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface org.apache.flink.api.connector.source.SplitEnumerator<FileSourceSplit,PendingSplitsCheckpoint<FileSourceSplit>>

Throws:

IOException

addReader

public void addReader(int subtaskId)

Specified by:

addReader in interface org.apache.flink.api.connector.source.SplitEnumerator<FileSourceSplit,PendingSplitsCheckpoint<FileSourceSplit>>

handleSplitRequest

public void handleSplitRequest(int subtask,
                               @Nullable
                               String hostname)

Specified by:

handleSplitRequest in interface org.apache.flink.api.connector.source.SplitEnumerator<FileSourceSplit,PendingSplitsCheckpoint<FileSourceSplit>>

handleSourceEvent

public void handleSourceEvent(int subtaskId,
                              org.apache.flink.api.connector.source.SourceEvent sourceEvent)

Specified by:

handleSourceEvent in interface org.apache.flink.api.connector.source.SplitEnumerator<FileSourceSplit,PendingSplitsCheckpoint<FileSourceSplit>>

addSplitsBack

public void addSplitsBack(List<FileSourceSplit> splits,
                          int subtaskId)

Specified by:

addSplitsBack in interface org.apache.flink.api.connector.source.SplitEnumerator<FileSourceSplit,PendingSplitsCheckpoint<FileSourceSplit>>

snapshotState

public PendingSplitsCheckpoint<FileSourceSplit> snapshotState(long checkpointId)

Specified by:

snapshotState in interface org.apache.flink.api.connector.source.SplitEnumerator<FileSourceSplit,PendingSplitsCheckpoint<FileSourceSplit>>