org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.remote

Class RemoteTierConsumerAgent

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.remote.RemoteTierConsumerAgent

All Implemented Interfaces:

AvailabilityNotifier, TierConsumerAgent

public class RemoteTierConsumerAgent
extends Object
implements TierConsumerAgent, AvailabilityNotifier

The data client is used to fetch data from remote tier.

Constructor Summary

Constructors

Constructor and Description
RemoteTierConsumerAgent(List<TieredStorageConsumerSpec> tieredStorageConsumerSpecs, RemoteStorageScanner remoteStorageScanner, PartitionFileReader partitionFileReader, int bufferSizeBytes)

Method Summary

Modifier and Type	Method and Description
void	close() Close the consumer agent.
Optional<Buffer>	getNextBuffer(TieredStoragePartitionId partitionId, TieredStorageSubpartitionId subpartitionId, int segmentId) Get buffer from the consumer agent.
void	notifyAvailable(TieredStoragePartitionId partitionId, TieredStorageInputChannelId inputChannelId) Notify that the data for the specific partition and input channel is available in tiered storage.
void	notifyAvailable(TieredStoragePartitionId partitionId, TieredStorageSubpartitionId subpartitionId) Notify that the data from the specific partition and subpartition is available in tiered storage.
int	peekNextBufferSubpartitionId(TieredStoragePartitionId partitionId, ResultSubpartitionIndexSet indexSet) Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available or the subpartition index does not belong to the specified indexSet.
void	registerAvailabilityNotifier(AvailabilityNotifier notifier) Register the notifier to notify the availability of a subpartition.
void	start() Start the consumer agent.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RemoteTierConsumerAgent

public RemoteTierConsumerAgent(List<TieredStorageConsumerSpec> tieredStorageConsumerSpecs,
                               RemoteStorageScanner remoteStorageScanner,
                               PartitionFileReader partitionFileReader,
                               int bufferSizeBytes)

Method Detail

start

public void start()

Description copied from interface: TierConsumerAgent

Start the consumer agent.

Specified by:

start in interface TierConsumerAgent

peekNextBufferSubpartitionId

public int peekNextBufferSubpartitionId(TieredStoragePartitionId partitionId,
                                        ResultSubpartitionIndexSet indexSet)
                                 throws IOException

Description copied from interface: TierConsumerAgent

Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available or the subpartition index does not belong to the specified indexSet.

Specified by:

peekNextBufferSubpartitionId in interface TierConsumerAgent

Parameters:

partitionId - The index of the partition which the returned subpartition should belong to.

indexSet - The indexes of the subpartitions expected.

Throws:

IOException

getNextBuffer

public Optional<Buffer> getNextBuffer(TieredStoragePartitionId partitionId,
                                      TieredStorageSubpartitionId subpartitionId,
                                      int segmentId)

Description copied from interface: TierConsumerAgent

Get buffer from the consumer agent.

Specified by:

getNextBuffer in interface TierConsumerAgent

Parameters:

partitionId - the id of partition.

subpartitionId - the id of subpartition.

segmentId - the id of segment.

Returns:

buffer.

registerAvailabilityNotifier

public void registerAvailabilityNotifier(AvailabilityNotifier notifier)

Description copied from interface: TierConsumerAgent

Register the notifier to notify the availability of a subpartition.

Specified by:

registerAvailabilityNotifier in interface TierConsumerAgent

Parameters:

notifier - to notify availability.

close

public void close()
           throws IOException

Description copied from interface: TierConsumerAgent

Close the consumer agent.

Specified by:

close in interface TierConsumerAgent

Throws:

IOException

notifyAvailable

public void notifyAvailable(TieredStoragePartitionId partitionId,
                            TieredStorageSubpartitionId subpartitionId)

Description copied from interface: AvailabilityNotifier

Notify that the data from the specific partition and subpartition is available in tiered storage.

Specified by:

notifyAvailable in interface AvailabilityNotifier

Parameters:

partitionId - the partition id.

subpartitionId - the subpartition id.

notifyAvailable

public void notifyAvailable(TieredStoragePartitionId partitionId,
                            TieredStorageInputChannelId inputChannelId)

Description copied from interface: AvailabilityNotifier

Notify that the data for the specific partition and input channel is available in tiered storage.

Specified by:

notifyAvailable in interface AvailabilityNotifier

Parameters:

partitionId - the partition id.

inputChannelId - the input channel id.