public class UnionResultSubpartitionView extends Object implements ResultSubpartitionView, BufferAvailabilityListener
ResultSubpartitionViews. This class provides
the following guarantees to the output buffers.
ResultSubpartitionView.AvailabilityWithBacklog| 构造器和说明 |
|---|
UnionResultSubpartitionView(BufferAvailabilityListener availabilityListener,
int numTotalViews) |
| 限定符和类型 | 方法和说明 |
|---|---|
void |
acknowledgeAllDataProcessed() |
ResultSubpartitionView.AvailabilityWithBacklog |
getAvailabilityAndBacklog(boolean isCreditAvailable)
Get the availability and backlog of the view.
|
Throwable |
getFailureCause()
ResultSubpartitionView can decide whether the failure cause should be reported to
consumer as failure (primary failure) or ProducerFailedException (secondary failure). |
ResultSubpartition.BufferAndBacklog |
getNextBuffer()
Returns the next
Buffer instance of this queue iterator. |
int |
getNumberOfQueuedBuffers() |
boolean |
isReleased() |
void |
notifyDataAvailable() |
void |
notifyDataAvailable(ResultSubpartitionView view)
Called whenever there might be new data available.
|
void |
notifyNewBufferSize(int newBufferSize) |
void |
notifyPriorityEvent(int priorityBufferNumber)
Called when the first priority event is added to the head of the buffer queue.
|
void |
notifyRequiredSegmentId(int subpartitionId,
int segmentId)
In tiered storage shuffle mode, only required segments will be sent to prevent the redundant
buffer usage.
|
void |
notifyViewCreated(int subpartitionId,
ResultSubpartitionView view) |
int |
peekNextBufferSubpartitionId()
Returns the index of the subpartition where the next buffer locates, or -1 if there is no
buffer available and the subpartition to be consumed is not determined.
|
void |
releaseAllResources() |
void |
resumeConsumption() |
int |
unsynchronizedGetNumberOfQueuedBuffers() |
public UnionResultSubpartitionView(BufferAvailabilityListener availabilityListener, int numTotalViews)
public void notifyViewCreated(int subpartitionId,
ResultSubpartitionView view)
public int peekNextBufferSubpartitionId()
throws IOException
ResultSubpartitionViewpeekNextBufferSubpartitionId 在接口中 ResultSubpartitionViewIOException@Nullable public ResultSubpartition.BufferAndBacklog getNextBuffer() throws IOException
ResultSubpartitionViewBuffer instance of this queue iterator.
If there is currently no instance available, it will return null. This might
happen for example when a pipelined queue producer is slower than the consumer or a spilled
queue needs to read in more data.
Important: The consumer has to make sure that each buffer instance will
eventually be recycled with Buffer.recycleBuffer() after it has been consumed.
getNextBuffer 在接口中 ResultSubpartitionViewIOExceptionpublic void notifyDataAvailable()
notifyDataAvailable 在接口中 ResultSubpartitionViewpublic void notifyDataAvailable(ResultSubpartitionView view)
BufferAvailabilityListenernotifyDataAvailable 在接口中 BufferAvailabilityListenerview - the ResultSubpartitionView containing available data.public void notifyPriorityEvent(int priorityBufferNumber)
BufferAvailabilityListenernotifyPriorityEvent 在接口中 BufferAvailabilityListenernotifyPriorityEvent 在接口中 ResultSubpartitionViewpriorityBufferNumber - the sequence number that identifies the priority buffer.public void releaseAllResources()
throws IOException
releaseAllResources 在接口中 ResultSubpartitionViewIOExceptionpublic boolean isReleased()
isReleased 在接口中 ResultSubpartitionViewpublic void resumeConsumption()
resumeConsumption 在接口中 ResultSubpartitionViewpublic void acknowledgeAllDataProcessed()
public Throwable getFailureCause()
ResultSubpartitionViewResultSubpartitionView can decide whether the failure cause should be reported to
consumer as failure (primary failure) or ProducerFailedException (secondary failure).
Secondary failure can be reported only if producer (upstream task) is guaranteed to failover.
BEWARE: Incorrectly reporting failure cause as primary failure, can hide the root cause of the failure from the user.
getFailureCause 在接口中 ResultSubpartitionViewpublic ResultSubpartitionView.AvailabilityWithBacklog getAvailabilityAndBacklog(boolean isCreditAvailable)
ResultSubpartitionViewgetAvailabilityAndBacklog 在接口中 ResultSubpartitionViewisCreditAvailable - the availability of credits for this ResultSubpartitionView.public void notifyRequiredSegmentId(int subpartitionId,
int segmentId)
ResultSubpartitionViewnotifyRequiredSegmentId 在接口中 ResultSubpartitionViewsubpartitionId - The id of the corresponding subpartition.segmentId - The id of required segment.public int unsynchronizedGetNumberOfQueuedBuffers()
public int getNumberOfQueuedBuffers()
public void notifyNewBufferSize(int newBufferSize)
notifyNewBufferSize 在接口中 ResultSubpartitionViewCopyright © 2014–2024 The Apache Software Foundation. All rights reserved.