org.apache.flink.table.runtime.hashtable

类 LongHybridHashTable

java.lang.Object

org.apache.flink.table.runtime.hashtable.BaseHybridHashTable

org.apache.flink.table.runtime.hashtable.LongHybridHashTable

所有已实现的接口:

org.apache.flink.core.memory.MemorySegmentSource, MemorySegmentPool

public abstract class LongHybridHashTable
extends BaseHybridHashTable

Special optimized hashTable with key long.

See LongHashPartition. TODO add min max long filter and bloomFilter to spilled partition.

字段概要

从类继承的字段 org.apache.flink.table.runtime.hashtable.BaseHybridHashTable

buildRowCount, buildSpillRetBufferNumbers, buildSpillReturnBuffers, closed, compressionBlockSize, compressionCodecFactory, compressionEnabled, currentEnumerator, currentRecursionDepth, currentSpilledBuildSide, currentSpilledProbeSide, initPartitionFanOut, internalPool, ioManager, LOG, MAX_NUM_PARTITIONS, MAX_RECURSION_DEPTH, numSpillFiles, segmentSize, segmentSizeBits, segmentSizeMask, spillInBytes, totalNumBuffers, tryDistinctBuildRow

构造器概要

构造器

构造器和说明
LongHybridHashTable(Object owner, boolean compressionEnabled, int compressionBlockSize, BinaryRowDataSerializer buildSideSerializer, BinaryRowDataSerializer probeSideSerializer, org.apache.flink.runtime.memory.MemoryManager memManager, long reservedMemorySize, org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, int avgRecordLen, long buildRowCount)

方法概要

限定符和类型	方法和说明
protected void	clearPartitions()
void	close() Closes the hash table.
int	compressionBlockSize()
org.apache.flink.runtime.io.compression.BlockCompressionFactory	compressionCodecFactory()
boolean	compressionEnabled()
void	endBuild()
void	free()
RowIterator<org.apache.flink.table.data.binary.BinaryRowData>	get(long probeKey) This method is only used for operator fusion codegen to get build row from hash table.
abstract long	getBuildLongKey(org.apache.flink.table.data.RowData row) For code gen get build side long key.
LongHashPartition.MatchIterator	getBuildSideIterator()
org.apache.flink.table.data.RowData	getCurrentProbeRow()
List<LongHashPartition>	getPartitionsPendingForSMJ()
abstract long	getProbeLongKey(org.apache.flink.table.data.RowData row) For code gen get probe side long key.
RowIterator	getSpilledPartitionBuildSideIter(LongHashPartition p)
ProbeIterator	getSpilledPartitionProbeSideIter(LongHashPartition p)
void	insertIntoProbeBuffer(org.apache.flink.table.data.RowData probeRecord) If the probe row corresponding partition has been spilled to disk, just call this method spill probe row to disk.
boolean	nextMatching()
abstract org.apache.flink.table.data.binary.BinaryRowData	probeToBinary(org.apache.flink.table.data.RowData row) For code gen probe side to BinaryRowData.
void	putBuildRow(org.apache.flink.table.data.binary.BinaryRowData row)
int	spillPartition()
boolean	tryProbe(org.apache.flink.table.data.RowData record)

从类继承的方法 org.apache.flink.table.runtime.hashtable.BaseHybridHashTable

createInputView, ensureNumBuffersReturned, freeCurrent, freePages, getNextBuffer, getNextBuffers, getNotNullNextBuffer, getNumSpillFiles, getSpillInBytes, getUsedMemoryInBytes, hash, maxInitBufferOfBucketArea, maxNumPartition, nextSegment, pageSize, readAllBuffers, releaseMemoryCacheForSMJ, remainBuffers, returnAll, returnPage

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

LongHybridHashTable

public LongHybridHashTable(Object owner,
                           boolean compressionEnabled,
                           int compressionBlockSize,
                           BinaryRowDataSerializer buildSideSerializer,
                           BinaryRowDataSerializer probeSideSerializer,
                           org.apache.flink.runtime.memory.MemoryManager memManager,
                           long reservedMemorySize,
                           org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
                           int avgRecordLen,
                           long buildRowCount)

方法详细资料

putBuildRow

public void putBuildRow(org.apache.flink.table.data.binary.BinaryRowData row)
                 throws IOException

抛出:

IOException

endBuild

public void endBuild()
              throws IOException

抛出:

IOException

get

@Nullable
public final RowIterator<org.apache.flink.table.data.binary.BinaryRowData> get(long probeKey)
                                                                                  throws IOException

This method is only used for operator fusion codegen to get build row from hash table. If the build partition has spilled to disk, return null directly which requires the join operator also spill probe row to disk.

抛出:

IOException

insertIntoProbeBuffer

public final void insertIntoProbeBuffer(org.apache.flink.table.data.RowData probeRecord)
                                 throws IOException

If the probe row corresponding partition has been spilled to disk, just call this method spill probe row to disk.

Note: This must be called only after get(long) method.

抛出:

IOException

tryProbe

public boolean tryProbe(org.apache.flink.table.data.RowData record)
                 throws IOException

抛出:

IOException

nextMatching

public boolean nextMatching()
                     throws IOException

抛出:

IOException

getCurrentProbeRow

public org.apache.flink.table.data.RowData getCurrentProbeRow()

getBuildSideIterator

public LongHashPartition.MatchIterator getBuildSideIterator()

close

public void close()

从类复制的说明: BaseHybridHashTable

Closes the hash table. This effectively releases all internal structures and closes all open files and removes them. The call to this method is valid both as a cleanup after the complete inputs were properly processed, and as an cancellation call, which cleans up all resources that are currently held by the hash join.

覆盖:

close 在类中 BaseHybridHashTable

free

public void free()

覆盖:

free 在类中 BaseHybridHashTable

getBuildLongKey

public abstract long getBuildLongKey(org.apache.flink.table.data.RowData row)

For code gen get build side long key.

getProbeLongKey

public abstract long getProbeLongKey(org.apache.flink.table.data.RowData row)

For code gen get probe side long key.

probeToBinary

public abstract org.apache.flink.table.data.binary.BinaryRowData probeToBinary(org.apache.flink.table.data.RowData row)

For code gen probe side to BinaryRowData.

spillPartition

public int spillPartition()
                   throws IOException

指定者:

spillPartition 在类中 BaseHybridHashTable

抛出:

IOException

getPartitionsPendingForSMJ

public List<LongHashPartition> getPartitionsPendingForSMJ()

getSpilledPartitionBuildSideIter

public RowIterator getSpilledPartitionBuildSideIter(LongHashPartition p)
                                             throws IOException

抛出:

IOException

getSpilledPartitionProbeSideIter

public ProbeIterator getSpilledPartitionProbeSideIter(LongHashPartition p)
                                               throws IOException

抛出:

IOException

clearPartitions

protected void clearPartitions()

指定者:

clearPartitions 在类中 BaseHybridHashTable

compressionEnabled

public boolean compressionEnabled()

compressionCodecFactory

public org.apache.flink.runtime.io.compression.BlockCompressionFactory compressionCodecFactory()

compressionBlockSize

public int compressionBlockSize()