org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier

Interface TierConsumerAgent

All Known Implementing Classes:

DiskTierConsumerAgent, MemoryTierConsumerAgent, RemoteTierConsumerAgent

public interface TierConsumerAgent

The TierConsumerAgent is the consumer agent of each tier in tiered store, which could read data from responding tier.

Method Summary

Modifier and Type	Method and Description
void	close() Close the consumer agent.
Optional<Buffer>	getNextBuffer(TieredStoragePartitionId partitionId, TieredStorageSubpartitionId subpartitionId, int segmentId) Get buffer from the consumer agent.
int	peekNextBufferSubpartitionId(TieredStoragePartitionId partitionId, ResultSubpartitionIndexSet indexSet) Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available or the subpartition index does not belong to the specified indexSet.
void	registerAvailabilityNotifier(AvailabilityNotifier notifier) Register the notifier to notify the availability of a subpartition.
void	setup(TieredStorageMemoryManager memoryManager) The consumer agent may request buffers from the memory manager.
void	start() Start the consumer agent.
void	updateTierShuffleDescriptor(TieredStoragePartitionId partitionId, TieredStorageInputChannelId inputChannelId, TieredStorageSubpartitionId subpartitionId, TierShuffleDescriptor tierShuffleDescriptor) Update the TierShuffleDescriptor for the consumer agent.

Method Detail

setup

void setup(TieredStorageMemoryManager memoryManager)

The consumer agent may request buffers from the memory manager. Therefore, the TieredStorageMemoryManager should be integrated into the tier consumer agent. Since the buffer pool is initialized after the creation of the client, the memory manager need to be assigned after the buffer pool becomes available.

start

void start()

Start the consumer agent.

peekNextBufferSubpartitionId

int peekNextBufferSubpartitionId(TieredStoragePartitionId partitionId,
                                 ResultSubpartitionIndexSet indexSet)
                          throws IOException

Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available or the subpartition index does not belong to the specified indexSet.

Parameters:

partitionId - The index of the partition which the returned subpartition should belong to.

indexSet - The indexes of the subpartitions expected.

Throws:

IOException

getNextBuffer

Optional<Buffer> getNextBuffer(TieredStoragePartitionId partitionId,
                               TieredStorageSubpartitionId subpartitionId,
                               int segmentId)
                        throws IOException

Get buffer from the consumer agent.

Parameters:

partitionId - the id of partition.

subpartitionId - the id of subpartition.

segmentId - the id of segment.

Returns:

buffer.

Throws:

IOException

registerAvailabilityNotifier

void registerAvailabilityNotifier(AvailabilityNotifier notifier)

Register the notifier to notify the availability of a subpartition.

Parameters:

notifier - to notify availability.

updateTierShuffleDescriptor

void updateTierShuffleDescriptor(TieredStoragePartitionId partitionId,
                                 TieredStorageInputChannelId inputChannelId,
                                 TieredStorageSubpartitionId subpartitionId,
                                 TierShuffleDescriptor tierShuffleDescriptor)

Update the TierShuffleDescriptor for the consumer agent.

close

void close()
    throws IOException

Close the consumer agent.

Throws:

IOException