org.apache.flink.table.runtime.operators.sort

Class AbstractBinaryExternalMerger<Entry>

java.lang.Object

org.apache.flink.table.runtime.operators.sort.AbstractBinaryExternalMerger<Entry>

Type Parameters:

Entry - Type of Entry to Merge sort.

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

BinaryExternalMerger, BinaryKVExternalMerger

public abstract class AbstractBinaryExternalMerger<Entry>
extends Object
implements Closeable

Spilled files Merger of BinaryExternalSorter. It merges maxFanIn spilled files at most once.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.flink.runtime.io.disk.iomanager.IOManager	ioManager
protected int	pageSize

Constructor Summary

Constructors

Constructor and Description
AbstractBinaryExternalMerger(org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, int pageSize, int maxFanIn, SpillChannelManager channelManager, boolean compressionEnabled, org.apache.flink.runtime.io.compression.BlockCompressionFactory compressionCodecFactory, int compressionBlockSize)

Method Summary

Modifier and Type	Method and Description
protected abstract org.apache.flink.util.MutableObjectIterator<Entry>	channelReaderInputViewIterator(org.apache.flink.runtime.io.disk.iomanager.AbstractChannelReaderInputView inView)
void	close()
BinaryMergeIterator<Entry>	getMergingIterator(List<ChannelWithMeta> channelIDs, List<org.apache.flink.runtime.io.disk.iomanager.FileIOChannel> openChannels) Returns an iterator that iterates over the merged result from all given channels.
List<ChannelWithMeta>	mergeChannelList(List<ChannelWithMeta> channelIDs) Merges the given sorted runs to a smaller number of sorted runs.
protected abstract Comparator<Entry>	mergeComparator()
protected abstract List<Entry>	mergeReusedEntries(int size)
protected abstract void	writeMergingOutput(org.apache.flink.util.MutableObjectIterator<Entry> mergeIterator, org.apache.flink.runtime.memory.AbstractPagedOutputView output) read the merged stream and write the data back.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

pageSize

protected final int pageSize

ioManager

protected final org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager

Constructor Detail

AbstractBinaryExternalMerger

public AbstractBinaryExternalMerger(org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
                                    int pageSize,
                                    int maxFanIn,
                                    SpillChannelManager channelManager,
                                    boolean compressionEnabled,
                                    org.apache.flink.runtime.io.compression.BlockCompressionFactory compressionCodecFactory,
                                    int compressionBlockSize)

Method Detail

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

getMergingIterator

public BinaryMergeIterator<Entry> getMergingIterator(List<ChannelWithMeta> channelIDs,
                                                     List<org.apache.flink.runtime.io.disk.iomanager.FileIOChannel> openChannels)
                                              throws IOException

Returns an iterator that iterates over the merged result from all given channels.

Parameters:

channelIDs - The channels that are to be merged and returned.

Returns:

An iterator over the merged records of the input channels.

Throws:

IOException - Thrown, if the readers encounter an I/O problem.

mergeChannelList

public List<ChannelWithMeta> mergeChannelList(List<ChannelWithMeta> channelIDs)
                                       throws IOException

Merges the given sorted runs to a smaller number of sorted runs.

Parameters:

channelIDs - The IDs of the sorted runs that need to be merged.

Returns:

A list of the IDs of the merged channels.

Throws:

IOException - Thrown, if the readers or writers encountered an I/O problem.

channelReaderInputViewIterator

protected abstract org.apache.flink.util.MutableObjectIterator<Entry> channelReaderInputViewIterator(org.apache.flink.runtime.io.disk.iomanager.AbstractChannelReaderInputView inView)

Returns:

entry iterator reading from inView.

mergeComparator

protected abstract Comparator<Entry> mergeComparator()

Returns:

merging comparator used in merging.

mergeReusedEntries

protected abstract List<Entry> mergeReusedEntries(int size)

Returns:

reused entry object used in merging.

writeMergingOutput

protected abstract void writeMergingOutput(org.apache.flink.util.MutableObjectIterator<Entry> mergeIterator,
                                           org.apache.flink.runtime.memory.AbstractPagedOutputView output)
                                    throws IOException

read the merged stream and write the data back.

Throws:

IOException