org.apache.flink.runtime.io.network.partition.hybrid

Class HsFileDataIndexImpl

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.HsFileDataIndexImpl

All Implemented Interfaces:

HsFileDataIndex

@ThreadSafe
public class HsFileDataIndexImpl
extends Object
implements HsFileDataIndex

Default implementation of HsFileDataIndex.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HsFileDataIndexImpl.HsFileDataIndexRegionHelper The implementation of FileDataIndexRegionHelper to writing a region to the file or reading a region from the file.
static class	HsFileDataIndexImpl.InternalRegion A HsFileDataIndexImpl.InternalRegion is an implementation of FileDataIndexRegionHelper.Region.

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.network.partition.hybrid.HsFileDataIndex

HsFileDataIndex.ReadableRegion, HsFileDataIndex.SpilledBuffer

Constructor Summary

Constructors

Constructor and Description
HsFileDataIndexImpl(int numSubpartitions, Path indexFilePath, int regionGroupSizeInBytes, long numRetainedInMemoryRegionsMax)

Method Summary

Modifier and Type	Method and Description
void	addBuffers(List<HsFileDataIndex.SpilledBuffer> spilledBuffers) Add buffers to the index.
void	close() Close this file data index.
Optional<HsFileDataIndex.ReadableRegion>	getReadableRegion(int subpartitionId, int bufferIndex, int consumingOffset) Get a HsFileDataIndex.ReadableRegion for the given subpartition that starts with the given buffer index, if existed.
void	markBufferReleased(int subpartitionId, int bufferIndex) Mark a buffer as RELEASED.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HsFileDataIndexImpl

public HsFileDataIndexImpl(int numSubpartitions,
                           Path indexFilePath,
                           int regionGroupSizeInBytes,
                           long numRetainedInMemoryRegionsMax)

Method Detail

close

public void close()

Description copied from interface: HsFileDataIndex

Close this file data index.

Specified by:

close in interface HsFileDataIndex

getReadableRegion

public Optional<HsFileDataIndex.ReadableRegion> getReadableRegion(int subpartitionId,
                                                                  int bufferIndex,
                                                                  int consumingOffset)

Description copied from interface: HsFileDataIndex

Get a HsFileDataIndex.ReadableRegion for the given subpartition that starts with the given buffer index, if existed.

Note: Depending on the implementation, this method does not guarantee to always return the longest possible readable region.

Specified by:

getReadableRegion in interface HsFileDataIndex

Parameters:

subpartitionId - that the readable region belongs to

bufferIndex - that the readable region starts with

consumingOffset - of the downstream

Returns:

a HsFileDataIndex.ReadableRegion for the given subpartition that starts with the given buffer index, if exist; otherwise, Optional.empty().

addBuffers

public void addBuffers(List<HsFileDataIndex.SpilledBuffer> spilledBuffers)

Description copied from interface: HsFileDataIndex

Add buffers to the index.

Attention: this method only called by spilling thread.

Specified by:

addBuffers in interface HsFileDataIndex

Parameters:

spilledBuffers - to be added. The buffers in the list are expected in the same order as in the spilled file.

markBufferReleased

public void markBufferReleased(int subpartitionId,
                               int bufferIndex)

Description copied from interface: HsFileDataIndex

Mark a buffer as RELEASED.

Specified by:

markBufferReleased in interface HsFileDataIndex

Parameters:

subpartitionId - that the buffer belongs to

bufferIndex - of the buffer within the subpartition