org.apache.flink.table.runtime.util.collections.binary

Class AbstractBytesMultiMap<K>

java.lang.Object

org.apache.flink.table.runtime.util.collections.binary.BytesMap<K,Iterator<org.apache.flink.table.data.RowData>>

org.apache.flink.table.runtime.util.collections.binary.AbstractBytesMultiMap<K>

Direct Known Subclasses:

BytesMultiMap, WindowBytesMultiMap

public abstract class AbstractBytesMultiMap<K>
extends BytesMap<K,Iterator<org.apache.flink.table.data.RowData>>

A binary map in the structure like Map<K, List<V>>, where there are multiple values under a single key, and they are all bytes based. It can be used for performing aggregations where the accumulator of aggregations are unfixed-width. The memory is divided into three areas:

Bucket area: pointer + hashcode

   |---- 4 Bytes (pointer to key entry) ----|
   |----- 4 Bytes (key hashcode) ----------|
 

Key area: a key entry contains key data, pointer to the tail value, and the head value entry.

   |--- 4 + len(K) Bytes  (key data) ------|
   |--- 4 Bytes (pointer to tail value) ---|
   |--- 4 Bytes (pointer to next value) ---|
   |--- 4 + len(V) Bytes (value data) -----|
 

Value area: a value entry contains a pointer to the next value and the value data. Pointer is -1 if this is the last entry.

   |--- 4 Bytes (pointer to next value) ---|
   |--- 4 + len(V) Bytes (value data) -----|
 

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.table.runtime.util.collections.binary.BytesMap

BytesMap.LookupInfo<K,V>

Field Summary

Fields

Modifier and Type	Field and Description
protected PagedTypeSerializer<K>	keySerializer Used to serialize map key into RecordArea's MemorySegments.

Fields inherited from class org.apache.flink.table.runtime.util.collections.binary.BytesMap

BUCKET_SIZE, BUCKET_SIZE_BITS, bucketSegments, ELEMENT_POINT_LENGTH, END_OF_LIST, growthThreshold, INIT_BUCKET_MEMORY_IN_BYTES, lastBucketPosition, LOAD_FACTOR, log2NumBuckets, memoryPool, numBucketsMask, numBucketsMask2, numBucketsPerSegment, numBucketsPerSegmentBits, numBucketsPerSegmentMask, numElements, numSpillFiles, RECORD_EXTRA_LENGTH, recordArea, reservedNumBuffers, reusedKey, reusedValue, segmentSize, spillInBytes, STEP_INCREMENT

Constructor Summary

Constructors

Constructor and Description
AbstractBytesMultiMap(Object owner, org.apache.flink.runtime.memory.MemoryManager memoryManager, long memorySize, PagedTypeSerializer<K> keySerializer, int valueArity)
AbstractBytesMultiMap(Object owner, org.apache.flink.runtime.memory.MemoryManager memoryManager, long memorySize, PagedTypeSerializer<K> keySerializer, org.apache.flink.table.types.logical.LogicalType[] valueTypes)

Method Summary

Modifier and Type	Method and Description
void	append(BytesMap.LookupInfo<K,Iterator<org.apache.flink.table.data.RowData>> lookupInfo, org.apache.flink.table.data.binary.BinaryRowData value) Append an value into the hash map's record area.
void	free() release the map's record and bucket area's memory segments.
void	free(boolean reservedFixedMemory)
KeyValueIterator<K,Iterator<org.apache.flink.table.data.RowData>>	getEntryIterator(boolean requiresCopy)
long	getNumKeys() Returns the number of keys in this map.
void	reset() reset the map's record and bucket area's memory segments for reusing.

Methods inherited from class org.apache.flink.table.runtime.util.collections.binary.BytesMap

calcSecondHashCode, getNumElements, getNumSpillFiles, getSpillInBytes, growAndRehash, initBucketSegments, lookup, resetBucketSegments, returnSegments

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

keySerializer

protected final PagedTypeSerializer<K> keySerializer

Used to serialize map key into RecordArea's MemorySegments.

Constructor Detail

AbstractBytesMultiMap

public AbstractBytesMultiMap(Object owner,
                             org.apache.flink.runtime.memory.MemoryManager memoryManager,
                             long memorySize,
                             PagedTypeSerializer<K> keySerializer,
                             org.apache.flink.table.types.logical.LogicalType[] valueTypes)

AbstractBytesMultiMap

public AbstractBytesMultiMap(Object owner,
                             org.apache.flink.runtime.memory.MemoryManager memoryManager,
                             long memorySize,
                             PagedTypeSerializer<K> keySerializer,
                             int valueArity)

Method Detail

getNumKeys

public long getNumKeys()

Description copied from class: BytesMap

Returns the number of keys in this map.

Specified by:

getNumKeys in class BytesMap<K,Iterator<org.apache.flink.table.data.RowData>>

append

public void append(BytesMap.LookupInfo<K,Iterator<org.apache.flink.table.data.RowData>> lookupInfo,
                   org.apache.flink.table.data.binary.BinaryRowData value)
            throws IOException

Append an value into the hash map's record area.

Throws:

IOException

getEntryIterator

public KeyValueIterator<K,Iterator<org.apache.flink.table.data.RowData>> getEntryIterator(boolean requiresCopy)

free

public void free()

release the map's record and bucket area's memory segments.

free

public void free(boolean reservedFixedMemory)

Overrides:

free in class BytesMap<K,Iterator<org.apache.flink.table.data.RowData>>

Parameters:

reservedFixedMemory - reserved fixed memory or not.

reset

public void reset()

reset the map's record and bucket area's memory segments for reusing.

Overrides:

reset in class BytesMap<K,Iterator<org.apache.flink.table.data.RowData>>