| Modifier and Type | Method and Description |
|---|---|
Collection<ResultPartitionID> |
NettyShuffleEnvironment.getPartitionsOccupyingLocalResources()
Report unreleased partitions.
|
| Modifier and Type | Method and Description |
|---|---|
boolean |
TaskEventPublisher.publish(ResultPartitionID partitionId,
TaskEvent event)
Publishes the event to the registered
EventListener instances. |
boolean |
TaskEventDispatcher.publish(ResultPartitionID partitionId,
TaskEvent event)
Publishes the event to the registered
EventListener instances. |
void |
TaskEventDispatcher.registerPartition(ResultPartitionID partitionId)
Registers the given partition for incoming task events allowing calls to
TaskEventDispatcher.subscribeToEvent(ResultPartitionID, EventListener, Class). |
void |
PartitionRequestClient.requestSubpartition(ResultPartitionID partitionId,
ResultSubpartitionIndexSet subpartitionIndexSet,
RemoteInputChannel inputChannel,
int delayMs)
Requests a range of remote sub partitions.
|
void |
NetworkSequenceViewReader.requestSubpartitionViewOrRegisterListener(ResultPartitionProvider partitionProvider,
ResultPartitionID resultPartitionId,
ResultSubpartitionIndexSet subpartitionIndexSet)
When the netty server receives the downstream task's partition request and the upstream task
has registered its partition, it will process the partition request immediately, otherwise it
will create a
PartitionRequestListener for given ResultPartitionID in ResultPartitionManager and notify the listener when the upstream task registers its
partition. |
void |
PartitionRequestClient.sendTaskEvent(ResultPartitionID partitionId,
TaskEvent event,
RemoteInputChannel inputChannel)
Sends a task event backwards to an intermediate result partition.
|
void |
TaskEventDispatcher.subscribeToEvent(ResultPartitionID partitionId,
EventListener<TaskEvent> eventListener,
Class<? extends TaskEvent> eventType)
Subscribes a listener to this dispatcher for events on a partition.
|
void |
TaskEventDispatcher.unregisterPartition(ResultPartitionID partitionId)
Removes the given partition from listening to incoming task events, thus forbidding calls to
TaskEventDispatcher.subscribeToEvent(ResultPartitionID, EventListener, Class). |
| Modifier and Type | Method and Description |
|---|---|
void |
NettyShuffleEnvironment.releasePartitionsLocally(Collection<ResultPartitionID> partitionIds) |
| Modifier and Type | Method and Description |
|---|---|
ResultPartitionID |
ResultPartitionWriter.getPartitionId() |
| Modifier and Type | Method and Description |
|---|---|
ResultPartitionID |
NettyPartitionRequestListener.getResultPartitionId() |
| Modifier and Type | Method and Description |
|---|---|
void |
NettyPartitionRequestClient.requestSubpartition(ResultPartitionID partitionId,
ResultSubpartitionIndexSet subpartitionIndexSet,
RemoteInputChannel inputChannel,
int delayMs)
Requests a remote intermediate result partition queue.
|
void |
NettyPartitionRequestClient.sendTaskEvent(ResultPartitionID partitionId,
TaskEvent event,
RemoteInputChannel inputChannel)
Sends a task event backwards to an intermediate result partition producer.
|
| Constructor and Description |
|---|
NettyPartitionRequestListener(ResultPartitionProvider resultPartitionProvider,
NetworkSequenceViewReader reader,
ResultSubpartitionIndexSet subpartitionIndexSet,
ResultPartitionID resultPartitionId) |
NettyPartitionRequestListener(ResultPartitionProvider resultPartitionProvider,
NetworkSequenceViewReader reader,
ResultSubpartitionIndexSet subpartitionIndexSet,
ResultPartitionID resultPartitionId,
long createTimestamp) |
| Modifier and Type | Field and Description |
|---|---|
protected ResultPartitionID |
ResultPartition.partitionId |
| Modifier and Type | Field and Description |
|---|---|
protected Map<ResultPartitionID,org.apache.flink.runtime.io.network.partition.AbstractPartitionTracker.PartitionInfo<K,M>> |
AbstractPartitionTracker.partitionInfos |
| Modifier and Type | Method and Description |
|---|---|
ResultPartitionID |
PartitionException.getPartitionId() |
ResultPartitionID |
ResultPartition.getPartitionId() |
ResultPartitionID |
PartitionRequestListener.getResultPartitionId()
Get the result partition id of the notifier.
|
ResultPartitionID |
TaskExecutorPartitionInfo.getResultPartitionId() |
ResultPartitionID |
PartitionTrackerEntry.getResultPartitionId() |
| Modifier and Type | Method and Description |
|---|---|
Map<ResultPartitionID,PartitionRequestListenerManager> |
ResultPartitionManager.getListenerManagers() |
Map<ResultPartitionID,ShuffleDescriptor> |
DataSetMetaInfo.getShuffleDescriptors() |
Collection<ResultPartitionID> |
ResultPartitionManager.getUnreleasedPartitions() |
| Modifier and Type | Method and Description |
|---|---|
ResultPartition |
ResultPartitionFactory.create(String taskNameWithSubtaskAndId,
int partitionIndex,
ResultPartitionID id,
ResultPartitionType type,
int numberOfSubpartitions,
int maxParallelism,
boolean isBroadcast,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory,
boolean isNumberOfPartitionConsumerUndefined) |
ResultSubpartitionView |
ResultPartitionManager.createSubpartitionView(ResultPartitionID partitionId,
ResultSubpartitionIndexSet subpartitionIndexSet,
BufferAvailabilityListener availabilityListener) |
ResultSubpartitionView |
ResultPartitionProvider.createSubpartitionView(ResultPartitionID partitionId,
ResultSubpartitionIndexSet indexSet,
BufferAvailabilityListener availabilityListener)
Returns the requested intermediate result partition input view.
|
Optional<ResultSubpartitionView> |
ResultPartitionManager.createSubpartitionViewOrRegisterListener(ResultPartitionID partitionId,
ResultSubpartitionIndexSet subpartitionIndexSet,
BufferAvailabilityListener availabilityListener,
PartitionRequestListener partitionRequestListener) |
Optional<ResultSubpartitionView> |
ResultPartitionProvider.createSubpartitionViewOrRegisterListener(ResultPartitionID partitionId,
ResultSubpartitionIndexSet indexSet,
BufferAvailabilityListener availabilityListener,
PartitionRequestListener partitionRequestListener)
If the upstream task's partition has been registered, returns the result subpartition input
view immediately, otherwise register the listener and return empty.
|
boolean |
PartitionTracker.isPartitionTracked(ResultPartitionID resultPartitionID)
Returns whether the given partition is being tracked.
|
boolean |
AbstractPartitionTracker.isPartitionTracked(ResultPartitionID resultPartitionID) |
void |
ResultPartitionManager.releasePartition(ResultPartitionID partitionId,
Throwable cause) |
void |
PartitionProducerStateProvider.requestPartitionProducerState(IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId,
java.util.function.Consumer<? super PartitionProducerStateProvider.ResponseHandle> responseConsumer)
Trigger the producer execution state request.
|
| Modifier and Type | Method and Description |
|---|---|
DataSetMetaInfo |
DataSetMetaInfo.addShuffleDescriptors(Map<ResultPartitionID,ShuffleDescriptor> shuffleDescriptors) |
void |
TaskExecutorPartitionTracker.promoteJobPartitions(Collection<ResultPartitionID> partitionsToPromote)
Promotes the given partitions.
|
void |
TaskExecutorPartitionTrackerImpl.promoteJobPartitions(Collection<ResultPartitionID> partitionsToPromote) |
CompletableFuture<Void> |
JobMasterPartitionTrackerImpl.stopTrackingAndPromotePartitions(Collection<ResultPartitionID> resultPartitionIds) |
CompletableFuture<Void> |
JobMasterPartitionTracker.stopTrackingAndPromotePartitions(Collection<ResultPartitionID> resultPartitionIds)
Promotes the given partitions, and stops the tracking of partitions that were promoted.
|
void |
TaskExecutorPartitionTracker.stopTrackingAndReleaseJobPartitions(Collection<ResultPartitionID> resultPartitionIds)
Releases the given partitions and stop the tracking of partitions that were released.
|
void |
TaskExecutorPartitionTrackerImpl.stopTrackingAndReleaseJobPartitions(Collection<ResultPartitionID> partitionsToRelease) |
default void |
JobMasterPartitionTracker.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> resultPartitionIds)
Releases the given partitions and stop the tracking of partitions that were released.
|
void |
JobMasterPartitionTrackerImpl.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> resultPartitionIds,
boolean releaseOnShuffleMaster) |
void |
JobMasterPartitionTracker.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> resultPartitionIds,
boolean releaseOnShuffleMaster)
Releases the given partitions and stop the tracking of partitions that were released.
|
Collection<PartitionTrackerEntry<K,M>> |
PartitionTracker.stopTrackingPartitions(Collection<ResultPartitionID> resultPartitionIds)
Stops the tracking of the given partitions.
|
Collection<PartitionTrackerEntry<K,M>> |
AbstractPartitionTracker.stopTrackingPartitions(Collection<ResultPartitionID> resultPartitionIds) |
| Constructor and Description |
|---|
BoundedBlockingResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
ResultSubpartition[] subpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
BufferWritingResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
ResultSubpartition[] subpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
PartitionException(String message,
ResultPartitionID partitionId) |
PartitionException(String message,
ResultPartitionID partitionId,
Throwable throwable) |
PartitionNotFoundException(ResultPartitionID partitionId) |
PipelinedResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
ResultSubpartition[] subpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
ResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numSubpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
SortMergeResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numSubpartitions,
int numTargetKeyGroups,
BatchShuffleReadBufferPool readBufferPool,
Executor readIOExecutor,
ResultPartitionManager partitionManager,
String resultFileBasePath,
BufferCompressor bufferCompressor,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
| Modifier and Type | Field and Description |
|---|---|
protected ResultPartitionID |
InputChannel.partitionId
The parent partition of the subpartitions consumed by this channel.
|
| Modifier and Type | Method and Description |
|---|---|
ResultPartitionID |
InputChannel.getPartitionId() |
| Constructor and Description |
|---|
InputChannel(SingleInputGate inputGate,
int channelIndex,
ResultPartitionID partitionId,
ResultSubpartitionIndexSet consumedSubpartitionIndexSet,
int initialBackoff,
int maxBackoff,
org.apache.flink.metrics.Counter numBytesIn,
org.apache.flink.metrics.Counter numBuffersIn) |
LocalInputChannel(SingleInputGate inputGate,
int channelIndex,
ResultPartitionID partitionId,
ResultSubpartitionIndexSet consumedSubpartitionIndexSet,
ResultPartitionManager partitionManager,
TaskEventPublisher taskEventPublisher,
int initialBackoff,
int maxBackoff,
org.apache.flink.metrics.Counter numBytesIn,
org.apache.flink.metrics.Counter numBuffersIn,
ChannelStateWriter stateWriter) |
PartitionConnectionException(ResultPartitionID partitionId,
Throwable throwable) |
RemoteChannelStateChecker(ResultPartitionID resultPartitionId,
String taskNameWithSubtask) |
RemoteInputChannel(SingleInputGate inputGate,
int channelIndex,
ResultPartitionID partitionId,
ResultSubpartitionIndexSet consumedSubpartitionIndexSet,
ConnectionID connectionId,
ConnectionManager connectionManager,
int initialBackOff,
int maxBackoff,
int partitionRequestListenerTimeout,
int networkBuffersPerChannel,
org.apache.flink.metrics.Counter numBytesIn,
org.apache.flink.metrics.Counter numBuffersIn,
ChannelStateWriter stateWriter) |
| Constructor and Description |
|---|
HsResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numSubpartitions,
int numTargetKeyGroups,
BatchShuffleReadBufferPool readBufferPool,
ScheduledExecutorService readIOExecutor,
ResultPartitionManager partitionManager,
String dataFileBashPath,
int networkBufferSize,
HybridShuffleConfiguration hybridShuffleConfiguration,
BufferCompressor bufferCompressor,
boolean isBroadcastOnly,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
| Modifier and Type | Method and Description |
|---|---|
static ResultPartitionID |
TieredStorageIdMappingUtils.convertId(TieredStoragePartitionId partitionId) |
ResultPartitionID |
TieredStoragePartitionId.getPartitionID() |
| Modifier and Type | Method and Description |
|---|---|
static TieredStoragePartitionId |
TieredStorageIdMappingUtils.convertId(ResultPartitionID resultPartitionId) |
| Constructor and Description |
|---|
TieredStoragePartitionId(ResultPartitionID partitionID) |
| Modifier and Type | Method and Description |
|---|---|
void |
TieredInternalShuffleMaster.addPartition(ResultPartitionID resultPartitionID) |
TieredResultPartition |
TieredResultPartitionFactory.createTieredResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numSubpartitions,
int maxParallelism,
Boolean isBroadCastOnly,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory,
FileChannelManager fileChannelManager,
BatchShuffleReadBufferPool batchShuffleReadBufferPool,
ScheduledExecutorService batchShuffleReadIOExecutor,
boolean isNumberOfPartitionConsumerUndefined) |
void |
TieredInternalShuffleMaster.releasePartition(ResultPartitionID resultPartitionID) |
| Constructor and Description |
|---|
TieredResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
int numSubpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory,
TieredStorageProducerClient tieredStorageProducerClient,
TieredStorageResourceRegistry tieredStorageResourceRegistry,
TieredStorageNettyServiceImpl nettyService,
List<TieredStorageMemorySpec> tieredStorageMemorySpecs,
TieredStorageMemoryManager storageMemoryManager) |
| Constructor and Description |
|---|
PartitionProducerDisposedException(ResultPartitionID resultPartitionID) |
| Modifier and Type | Method and Description |
|---|---|
void |
TaskManagerGateway.releasePartitions(org.apache.flink.api.common.JobID jobId,
Set<ResultPartitionID> partitionIds)
Batch release intermediate result partitions.
|
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<ExecutionState> |
JobMasterGateway.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID partitionId)
Requests the current state of the partition.
|
CompletableFuture<ExecutionState> |
JobMaster.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
| Modifier and Type | Method and Description |
|---|---|
void |
RpcTaskManagerGateway.releasePartitions(org.apache.flink.api.common.JobID jobId,
Set<ResultPartitionID> partitionIds) |
CompletableFuture<?> |
JobMasterGateway.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> partitionIds)
Notifies the
JobMasterPartitionTracker
to stop tracking the target result partitions and release the locally occupied resources on
TaskExecutors if any. |
CompletableFuture<?> |
JobMaster.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> partitionIds) |
| Modifier and Type | Method and Description |
|---|---|
ExecutionState |
SchedulerNG.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
ExecutionState |
ExecutionGraphHandler.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
ExecutionState |
SchedulerBase.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
| Modifier and Type | Method and Description |
|---|---|
ExecutionState |
AdaptiveScheduler.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
| Modifier and Type | Method and Description |
|---|---|
ResultPartitionID |
UnknownShuffleDescriptor.getResultPartitionID() |
ResultPartitionID |
NettyShuffleDescriptor.getResultPartitionID() |
ResultPartitionID |
ShuffleDescriptor.getResultPartitionID() |
| Modifier and Type | Method and Description |
|---|---|
Collection<ResultPartitionID> |
ShuffleEnvironment.getPartitionsOccupyingLocalResources()
Report partitions which still occupy some resources locally.
|
| Modifier and Type | Method and Description |
|---|---|
void |
ShuffleEnvironment.releasePartitionsLocally(Collection<ResultPartitionID> partitionIds)
Release local resources occupied by the given partitions.
|
CompletableFuture<?> |
JobShuffleContext.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> partitionIds)
Notifies the job to stop tracking and release the target result partitions, which means these
partitions will be removed and will be reproduced if used afterwards.
|
CompletableFuture<?> |
JobShuffleContextImpl.stopTrackingAndReleasePartitions(Collection<ResultPartitionID> partitionIds) |
| Constructor and Description |
|---|
NettyShuffleDescriptor(ResourceID producerLocation,
NettyShuffleDescriptor.PartitionConnectionInfo partitionConnectionInfo,
ResultPartitionID resultPartitionID) |
UnknownShuffleDescriptor(ResultPartitionID resultPartitionID) |
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<ExecutionState> |
PartitionProducerStateChecker.requestPartitionProducerState(org.apache.flink.api.common.JobID jobId,
IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId)
Requests the execution state of the execution producing a result partition.
|
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<Acknowledge> |
TaskExecutor.promotePartitions(org.apache.flink.api.common.JobID jobId,
Set<ResultPartitionID> partitionIds) |
CompletableFuture<Acknowledge> |
TaskExecutorGateway.promotePartitions(org.apache.flink.api.common.JobID jobId,
Set<ResultPartitionID> partitionIds)
Batch promote intermediate result partitions.
|
CompletableFuture<Acknowledge> |
TaskExecutorGatewayDecoratorBase.promotePartitions(org.apache.flink.api.common.JobID jobId,
Set<ResultPartitionID> partitionIds) |
void |
TaskExecutor.releasePartitions(org.apache.flink.api.common.JobID jobId,
Set<ResultPartitionID> partitionIds) |
void |
TaskExecutorGateway.releasePartitions(org.apache.flink.api.common.JobID jobId,
Set<ResultPartitionID> partitionIds)
Batch release intermediate result partitions.
|
void |
TaskExecutorGatewayDecoratorBase.releasePartitions(org.apache.flink.api.common.JobID jobId,
Set<ResultPartitionID> partitionIds) |
| Modifier and Type | Method and Description |
|---|---|
Set<ResultPartitionID> |
ClusterPartitionReport.ClusterPartitionReportEntry.getHostedPartitions() |
Map<ResultPartitionID,ShuffleDescriptor> |
ClusterPartitionReport.ClusterPartitionReportEntry.getShuffleDescriptors() |
Collection<ResultPartitionID> |
PartitionTable.stopTrackingPartitions(K key)
Stops the tracking of all partition for the given key.
|
| Modifier and Type | Method and Description |
|---|---|
void |
PartitionTable.startTrackingPartitions(K key,
Collection<ResultPartitionID> newPartitionIds)
Starts the tracking of the given partition for the given key.
|
void |
PartitionTable.stopTrackingPartitions(K key,
Collection<ResultPartitionID> partitionIds)
Stops the tracking of the given set of partitions for the given key.
|
| Constructor and Description |
|---|
ClusterPartitionReportEntry(IntermediateDataSetID dataSetId,
int numTotalPartitions,
Map<ResultPartitionID,ShuffleDescriptor> shuffleDescriptors) |
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<ExecutionState> |
RpcPartitionStateChecker.requestPartitionProducerState(org.apache.flink.api.common.JobID jobId,
IntermediateDataSetID resultId,
ResultPartitionID partitionId) |
| Modifier and Type | Method and Description |
|---|---|
void |
Task.requestPartitionProducerState(IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId,
java.util.function.Consumer<? super PartitionProducerStateProvider.ResponseHandle> responseConsumer) |
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.