org.apache.flink.runtime.io.network.partition.hybrid

Class HsMemoryDataSpiller

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.HsMemoryDataSpiller

All Implemented Interfaces:

AutoCloseable

public class HsMemoryDataSpiller
extends Object
implements AutoCloseable

This component is responsible for asynchronously writing in-memory data to disk. Each spilling operation will write the disk file sequentially.

Constructor Summary

Constructors

Constructor and Description
HsMemoryDataSpiller(Path dataFilePath)

Method Summary

Modifier and Type	Method and Description
void	close() Close this HsMemoryDataSpiller.
CompletableFuture<List<HsFileDataIndex.SpilledBuffer>>	spillAsync(List<BufferWithIdentity> bufferToSpill) Spilling buffers to disk asynchronously.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HsMemoryDataSpiller

public HsMemoryDataSpiller(Path dataFilePath)
                    throws IOException

Throws:

IOException

Method Detail

spillAsync

public CompletableFuture<List<HsFileDataIndex.SpilledBuffer>> spillAsync(List<BufferWithIdentity> bufferToSpill)

Spilling buffers to disk asynchronously.

Parameters:

bufferToSpill - buffers need to be spilled, must ensure that it is sorted by (subpartitionId, bufferIndex).

Returns:

the completable future contains spilled buffers information.

close

public void close()

Close this HsMemoryDataSpiller. It means spiller will no longer accept new spilling operation and wait for all previous spilling operation done blocking.

Specified by:

close in interface AutoCloseable