Package org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.memory

Class MemoryTierProducerAgent

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.memory.MemoryTierProducerAgent

All Implemented Interfaces:

AutoCloseable, NettyServiceProducer, TierProducerAgent

public class MemoryTierProducerAgent
extends Object
implements TierProducerAgent, NettyServiceProducer

The memory tier implementation of TierProducerAgent.

Constructor Summary

Constructors

Constructor	Description
MemoryTierProducerAgent(TieredStoragePartitionId partitionId, int numSubpartitions, int bufferSizeBytes, int segmentSizeBytes, int subpartitionMaxQueuedBuffers, boolean isBroadcastOnly, TieredStorageMemoryManager memoryManager, TieredStorageNettyService nettyService, TieredStorageResourceRegistry resourceRegistry, BufferCompressor bufferCompressor)

Method Summary

Modifier and Type	Method	Description
void	close()	Close the agent.
void	connectionBroken(NettyConnectionId connectionId)	NettyConnectionWriter related to a connection id will be notified when the netty connection is broken.
void	connectionEstablished(TieredStorageSubpartitionId subpartitionId, NettyConnectionWriter nettyConnectionWriter)	NettyConnectionWriter will be created when a netty connection is established for a subpartition.
boolean	tryStartNewSegment(TieredStorageSubpartitionId subpartitionId, int segmentId, int minNumBuffers)	Try to start a new segment in the Tier.
boolean	tryWrite(TieredStorageSubpartitionId subpartitionId, Buffer finishedBuffer, Object bufferOwner, int numRemainingConsecutiveBuffers)	Writes the finished Buffer to the consumer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MemoryTierProducerAgent

public MemoryTierProducerAgent(TieredStoragePartitionId partitionId,
                               int numSubpartitions,
                               int bufferSizeBytes,
                               int segmentSizeBytes,
                               int subpartitionMaxQueuedBuffers,
                               boolean isBroadcastOnly,
                               TieredStorageMemoryManager memoryManager,
                               TieredStorageNettyService nettyService,
                               TieredStorageResourceRegistry resourceRegistry,
                               BufferCompressor bufferCompressor)

Method Detail

tryStartNewSegment

public boolean tryStartNewSegment(TieredStorageSubpartitionId subpartitionId,
                                  int segmentId,
                                  int minNumBuffers)

Description copied from interface: TierProducerAgent

Try to start a new segment in the Tier.

Specified by:

tryStartNewSegment in interface TierProducerAgent

Parameters:

subpartitionId - subpartition id that the new segment belongs to

segmentId - id of the new segment

minNumBuffers - the minimum number of buffers that are expected to be written to the newly created segment.

Returns:

true if the segment can be started, false otherwise.

tryWrite

public boolean tryWrite(TieredStorageSubpartitionId subpartitionId,
                        Buffer finishedBuffer,
                        Object bufferOwner,
                        int numRemainingConsecutiveBuffers)

Description copied from interface: TierProducerAgent

Writes the finished Buffer to the consumer.

Note that the method is successfully executed (without throwing any exception), the buffer should be released by the caller, otherwise the tier should be responsible to recycle the buffer.

Specified by:

tryWrite in interface TierProducerAgent

Parameters:

subpartitionId - the subpartition id that the buffer is writing to

finishedBuffer - the writing buffer

bufferOwner - the current owner of this writing buffer

numRemainingConsecutiveBuffers - number of buffers that would be passed in the following invocations and should be written to the same segment as this one

Returns:

return true if the buffer is written successfully, return false if the current segment can not store this buffer(and the following consecutive ones) and the current segment is finished. When returning false, the agent should try start a new segment before writing the buffer.

connectionEstablished

public void connectionEstablished(TieredStorageSubpartitionId subpartitionId,
                                  NettyConnectionWriter nettyConnectionWriter)

Description copied from interface: NettyServiceProducer

NettyConnectionWriter will be created when a netty connection is established for a subpartition.

Specified by:

connectionEstablished in interface NettyServiceProducer

Parameters:

subpartitionId - subpartition id indicates the id of subpartition.

nettyConnectionWriter - writer is used to write buffers to netty connection.

connectionBroken

public void connectionBroken(NettyConnectionId connectionId)

Description copied from interface: NettyServiceProducer

NettyConnectionWriter related to a connection id will be notified when the netty connection is broken.

Specified by:

connectionBroken in interface NettyServiceProducer

Parameters:

connectionId - connection id is the id of connection.

close

public void close()

Description copied from interface: TierProducerAgent

Close the agent.

Note this only releases resources directly hold by the agent, which excludes resources managed by the resource registry.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface TierProducerAgent