org.apache.flink.connector.file.src.assigners

Interface FileSplitAssigner

All Known Implementing Classes:

LocalityAwareSplitAssigner, SimpleSplitAssigner

@PublicEvolving
public interface FileSplitAssigner

The FileSplitAssigner is responsible for deciding what split should be processed next by which node. It determines split processing order and locality.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	FileSplitAssigner.Provider Factory for the FileSplitAssigner, to allow the FileSplitAssigner to be eagerly initialized and to not be serializable.

Method Summary

Modifier and Type	Method and Description
void	addSplits(Collection<FileSourceSplit> splits) Adds a set of splits to this assigner.
Optional<FileSourceSplit>	getNext(String hostname) Gets the next split.
Collection<FileSourceSplit>	remainingSplits() Gets the remaining splits that this assigner has pending.

Method Detail

getNext

Optional<FileSourceSplit> getNext(@Nullable
                                  String hostname)

Gets the next split.

When this method returns an empty Optional, then the set of splits is assumed to be done and the source will finish once the readers finished their current splits.

addSplits

void addSplits(Collection<FileSourceSplit> splits)

Adds a set of splits to this assigner. This happens for example when some split processing failed and the splits need to be re-added, or when new splits got discovered.

remainingSplits

Collection<FileSourceSplit> remainingSplits()

Gets the remaining splits that this assigner has pending.