Package org.apache.flink.runtime.io.network.partition

Class HashBasedDataBuffer

java.lang.Object

org.apache.flink.runtime.io.network.partition.HashBasedDataBuffer

All Implemented Interfaces:

DataBuffer

public class HashBasedDataBuffer
extends Object
implements DataBuffer

* A DataBuffer implementation which sorts all appended records only by subpartition index. Records of the same subpartition keep the appended order.

Different from the SortBasedDataBuffer, in this DataBuffer implementation, memory segment boundary serves as the nature data boundary of different subpartitions, which means that one memory segment can never contain data from different subpartitions.

Constructor Summary

Constructors

Constructor	Description
HashBasedDataBuffer(LinkedList<org.apache.flink.core.memory.MemorySegment> freeSegments, BufferRecycler bufferRecycler, int numSubpartitions, int bufferSize, int numGuaranteedBuffers, int[] customReadOrder)

Method Summary

Modifier and Type	Method	Description
boolean	append(ByteBuffer source, int targetSubpartition, Buffer.DataType dataType)	Partial data of the target record can be written if this HashBasedDataBuffer is full.
void	finish()	Finishes this DataBuffer which means no record can be appended anymore.
BufferWithSubpartition	getNextBuffer(org.apache.flink.core.memory.MemorySegment transitBuffer)	Copies data in this DataBuffer to the target MemorySegment in subpartition index order and returns BufferWithSubpartition which contains the copied data and the corresponding subpartition index.
boolean	hasRemaining()	Returns true if not all data appended to this DataBuffer is consumed.
boolean	isFinished()	Whether this DataBuffer is finished or not.
boolean	isReleased()	Whether this DataBuffer is released or not.
long	numTotalBytes()	Returns the total number of bytes written to this DataBuffer.
long	numTotalRecords()	Returns the total number of records written to this DataBuffer.
void	release()	Releases this DataBuffer which releases all resources.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HashBasedDataBuffer

public HashBasedDataBuffer(LinkedList<org.apache.flink.core.memory.MemorySegment> freeSegments,
                           BufferRecycler bufferRecycler,
                           int numSubpartitions,
                           int bufferSize,
                           int numGuaranteedBuffers,
                           @Nullable
                           int[] customReadOrder)

Method Detail

append

public boolean append(ByteBuffer source,
                      int targetSubpartition,
                      Buffer.DataType dataType)
               throws IOException

Partial data of the target record can be written if this HashBasedDataBuffer is full. The remaining data of the target record will be written to the next data region (a new data buffer or this data buffer after reset).

Specified by:

append in interface DataBuffer

Throws:

IOException

getNextBuffer

public BufferWithSubpartition getNextBuffer(org.apache.flink.core.memory.MemorySegment transitBuffer)

Description copied from interface: DataBuffer

Copies data in this DataBuffer to the target MemorySegment in subpartition index order and returns BufferWithSubpartition which contains the copied data and the corresponding subpartition index.

Specified by:

getNextBuffer in interface DataBuffer

numTotalRecords

public long numTotalRecords()

Description copied from interface: DataBuffer

Returns the total number of records written to this DataBuffer.

Specified by:

numTotalRecords in interface DataBuffer

numTotalBytes

public long numTotalBytes()

Description copied from interface: DataBuffer

Returns the total number of bytes written to this DataBuffer.

Specified by:

numTotalBytes in interface DataBuffer

hasRemaining

public boolean hasRemaining()

Description copied from interface: DataBuffer

Returns true if not all data appended to this DataBuffer is consumed.

Specified by:

hasRemaining in interface DataBuffer

finish

public void finish()

Description copied from interface: DataBuffer

Finishes this DataBuffer which means no record can be appended anymore.

Specified by:

finish in interface DataBuffer

isFinished

public boolean isFinished()

Description copied from interface: DataBuffer

Whether this DataBuffer is finished or not.

Specified by:

isFinished in interface DataBuffer

release

public void release()

Description copied from interface: DataBuffer

Releases this DataBuffer which releases all resources.

Specified by:

release in interface DataBuffer

isReleased

public boolean isReleased()

Description copied from interface: DataBuffer

Whether this DataBuffer is released or not.

Specified by:

isReleased in interface DataBuffer