Package org.apache.flink.connector.file.src.assigners

Class LocalityAwareSplitAssigner

java.lang.Object

org.apache.flink.connector.file.src.assigners.LocalityAwareSplitAssigner

All Implemented Interfaces:

FileSplitAssigner

@PublicEvolving
public class LocalityAwareSplitAssigner
extends Object
implements FileSplitAssigner

A FileSplitAssigner that assigns to each host preferably splits that are local, before assigning splits that are not local.

Implementation Note: This class is an adjusted copy of the previous API's class LocatableInputSplitAssigner and reproduces the same assignment logic, for compatibility. The logic has not been changed or optimized.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.connector.file.src.assigners.FileSplitAssigner

FileSplitAssigner.Provider

Constructor Summary

Constructors

Constructor	Description
LocalityAwareSplitAssigner(Collection<FileSourceSplit> splits)

Method Summary

Modifier and Type	Method	Description
void	addSplits(Collection<FileSourceSplit> splits)	Adds a set of splits to this assigner.
Optional<FileSourceSplit>	getNext(String host)	Gets the next split.
Collection<FileSourceSplit>	remainingSplits()	Gets the remaining splits that this assigner has pending.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LocalityAwareSplitAssigner

public LocalityAwareSplitAssigner(Collection<FileSourceSplit> splits)

Method Detail

getNext

public Optional<FileSourceSplit> getNext(@Nullable
                                         String host)

Description copied from interface: FileSplitAssigner

Gets the next split.

When this method returns an empty Optional, then the set of splits is assumed to be done and the source will finish once the readers finished their current splits.

Specified by:

getNext in interface FileSplitAssigner

addSplits

public void addSplits(Collection<FileSourceSplit> splits)

Description copied from interface: FileSplitAssigner

Adds a set of splits to this assigner. This happens for example when some split processing failed and the splits need to be re-added, or when new splits got discovered.

Specified by:

addSplits in interface FileSplitAssigner

remainingSplits

public Collection<FileSourceSplit> remainingSplits()

Description copied from interface: FileSplitAssigner

Gets the remaining splits that this assigner has pending.

Specified by:

remainingSplits in interface FileSplitAssigner