public class LocalBufferPool extends Object implements BufferPool
Buffer instances from the NetworkBufferPool.
Buffer requests are mediated to the network buffer pool to ensure deadlock free operation of the network stack by limiting the number of buffers per local buffer pool. It also implements the default mechanism for buffer recycling, which ensures that every buffer is ultimately returned to the network buffer pool.
The size of this pool can be dynamically changed at runtime (setNumBuffers(int). It
will then lazily return the required number of buffers to the NetworkBufferPool to match
its new size.
New buffers can be requested only when numberOfRequestedMemorySegments <
currentPoolSize + maxOverdraftBuffersPerGate. In other words, all buffers exceeding the
currentPoolSize will be dynamically regarded as overdraft buffers.
Availability is defined as returning a non-overdraft segment on a subsequent requestBuffer()/ requestBufferBuilder() and heaving a non-blocking requestBufferBuilderBlocking(int). In particular,
availableMemorySegments.
maxBuffersPerChannel.
To ensure this contract, the implementation eagerly fetches additional memory segments from
NetworkBufferPool as long as it hasn't reached maxNumberOfMemorySegments or one
subpartition reached the quota.
AvailabilityProvider.AvailabilityHelperBufferRecycler.DummyBufferRecyclerAVAILABLE| 限定符和类型 | 方法和说明 |
|---|---|
boolean |
addBufferListener(BufferListener listener)
Adds a buffer availability listener to the buffer provider.
|
int |
bestEffortGetNumOfUsedBuffers()
Returns the number of used buffers of this buffer pool.
|
CompletableFuture<?> |
getAvailableFuture() |
int |
getEstimatedNumberOfRequestedMemorySegments()
Estimates the number of requested buffers.
|
int |
getMaxNumberOfMemorySegments()
Returns the maximum number of memory segments this buffer pool should use.
|
int |
getMaxOverdraftBuffersPerGate()
Returns the max overdraft buffer size of per gate.
|
int |
getNumberOfAvailableMemorySegments()
Returns the number memory segments, which are currently held by this buffer pool.
|
int |
getNumberOfRequestedMemorySegments() |
int |
getNumberOfRequiredMemorySegments()
Returns the number of guaranteed (minimum number of) memory segments of this buffer pool.
|
int |
getNumBuffers()
Returns the current size of this buffer pool.
|
boolean |
isDestroyed()
Checks whether this buffer pool has been destroyed.
|
void |
lazyDestroy()
Destroy is called after the produce or consume phase of a task finishes.
|
void |
recycle(org.apache.flink.core.memory.MemorySegment segment)
Recycles the
MemorySegment to its original BufferPool instance. |
Buffer |
requestBuffer()
Returns a
Buffer instance from the buffer provider, if one is available. |
BufferBuilder |
requestBufferBuilder()
Returns a
BufferBuilder instance from the buffer provider. |
BufferBuilder |
requestBufferBuilder(int targetChannel)
Returns a
BufferBuilder instance from the buffer provider. |
BufferBuilder |
requestBufferBuilderBlocking()
Returns a
BufferBuilder instance from the buffer provider. |
BufferBuilder |
requestBufferBuilderBlocking(int targetChannel)
Returns a
BufferBuilder instance from the buffer provider. |
org.apache.flink.core.memory.MemorySegment |
requestMemorySegment()
Returns a
MemorySegment instance from the buffer provider. |
org.apache.flink.core.memory.MemorySegment |
requestMemorySegmentBlocking()
Returns a
MemorySegment instance from the buffer provider. |
void |
reserveSegments(int numberOfSegmentsToReserve)
Reserves the target number of segments to this pool.
|
void |
setMaxOverdraftBuffersPerGate(int maxOverdraftBuffersPerGate)
Sets the max overdraft buffer size of per gate.
|
void |
setNumBuffers(int numBuffers)
Sets the current size of this buffer pool.
|
String |
toString() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, waitand, isApproximatelyAvailable, isAvailable, orpublic void reserveSegments(int numberOfSegmentsToReserve)
throws IOException
BufferPoolreserveSegments 在接口中 BufferPoolIOExceptionpublic boolean isDestroyed()
BufferPoolisDestroyed 在接口中 BufferPoolisDestroyed 在接口中 BufferProviderpublic int getNumberOfRequiredMemorySegments()
BufferPoolgetNumberOfRequiredMemorySegments 在接口中 BufferPoolpublic int getMaxNumberOfMemorySegments()
BufferPoolgetMaxNumberOfMemorySegments 在接口中 BufferPoolpublic int getEstimatedNumberOfRequestedMemorySegments()
getMaxNumberOfMemorySegments() for bounded pools. For
unbounded pools it returns an approximation based upon getNumberOfRequiredMemorySegments()@VisibleForTesting public int getNumberOfRequestedMemorySegments()
public int getNumberOfAvailableMemorySegments()
BufferPoolgetNumberOfAvailableMemorySegments 在接口中 BufferPoolpublic int getNumBuffers()
BufferPoolThe size of the buffer pool can change dynamically at runtime.
getNumBuffers 在接口中 BufferPoolpublic int bestEffortGetNumOfUsedBuffers()
BufferPoolbestEffortGetNumOfUsedBuffers 在接口中 BufferPoolpublic Buffer requestBuffer()
BufferProviderBuffer instance from the buffer provider, if one is available.requestBuffer 在接口中 BufferProvidernull if no buffer is available or the buffer provider has been destroyed.public BufferBuilder requestBufferBuilder()
BufferProviderBufferBuilder instance from the buffer provider. This equals to BufferProvider.requestBufferBuilder(int) with unknown target channel.requestBufferBuilder 在接口中 BufferProvidernull if no buffer is available or the buffer provider has been destroyed.public BufferBuilder requestBufferBuilder(int targetChannel)
BufferProviderBufferBuilder instance from the buffer provider.requestBufferBuilder 在接口中 BufferProvidertargetChannel - to which the request will be accounted to.null if no buffer is available or the buffer provider has been destroyed.public BufferBuilder requestBufferBuilderBlocking() throws InterruptedException
BufferProviderBufferBuilder instance from the buffer provider. This equals to BufferProvider.requestBufferBuilderBlocking(int) with unknown target channel.
If there is no buffer available, the call will block until one becomes available again or the buffer provider has been destroyed.
requestBufferBuilderBlocking 在接口中 BufferProviderInterruptedExceptionpublic org.apache.flink.core.memory.MemorySegment requestMemorySegmentBlocking()
throws InterruptedException
BufferProviderMemorySegment instance from the buffer provider.
If there is no memory segment available, the call will block until one becomes available again or the buffer provider has been destroyed.
requestMemorySegmentBlocking 在接口中 BufferProviderInterruptedExceptionpublic BufferBuilder requestBufferBuilderBlocking(int targetChannel) throws InterruptedException
BufferProviderBufferBuilder instance from the buffer provider.
If there is no buffer available, the call will block until one becomes available again or the buffer provider has been destroyed.
requestBufferBuilderBlocking 在接口中 BufferProvidertargetChannel - to which the request will be accounted to.InterruptedExceptionpublic org.apache.flink.core.memory.MemorySegment requestMemorySegment()
BufferProviderMemorySegment instance from the buffer provider.requestMemorySegment 在接口中 BufferProvidernull if no memory segment is available or the buffer provider has been
destroyed.public void recycle(org.apache.flink.core.memory.MemorySegment segment)
BufferRecyclerMemorySegment to its original BufferPool instance.recycle 在接口中 BufferRecyclersegment - The memory segment to be recycled.public void lazyDestroy()
lazyDestroy 在接口中 BufferPoolpublic boolean addBufferListener(BufferListener listener)
BufferProviderThe operation fails with return value false, when there is a buffer available
or the buffer provider has been destroyed.
addBufferListener 在接口中 BufferProviderpublic void setNumBuffers(int numBuffers)
BufferPoolThe size needs to be greater or equal to the guaranteed number of memory segments.
setNumBuffers 在接口中 BufferPoolpublic void setMaxOverdraftBuffersPerGate(int maxOverdraftBuffersPerGate)
BufferPoolsetMaxOverdraftBuffersPerGate 在接口中 BufferPoolpublic int getMaxOverdraftBuffersPerGate()
BufferPoolgetMaxOverdraftBuffersPerGate 在接口中 BufferPoolpublic CompletableFuture<?> getAvailableFuture()
getAvailableFuture 在接口中 AvailabilityProviderCopyright © 2014–2024 The Apache Software Foundation. All rights reserved.