| Modifier and Type | Method and Description |
|---|---|
IntermediateDataSetID |
InputGateDeploymentDescriptor.getConsumedResultId() |
IntermediateDataSetID |
ResultPartitionDeploymentDescriptor.getResultId() |
| Constructor and Description |
|---|
InputGateDeploymentDescriptor(IntermediateDataSetID consumedResultId,
ResultPartitionType consumedPartitionType,
IndexRange consumedSubpartitionIndexRange,
int numberOfInputChannels,
List<TaskDeploymentDescriptor.MaybeOffloaded<TaskDeploymentDescriptorFactory.ShuffleDescriptorGroup>> serializedInputChannels) |
InputGateDeploymentDescriptor(IntermediateDataSetID consumedResultId,
ResultPartitionType consumedPartitionType,
int consumedSubpartitionIndex,
TaskDeploymentDescriptorFactory.ShuffleDescriptorAndIndex[] inputChannels) |
| Modifier and Type | Method and Description |
|---|---|
IntermediateDataSetID |
IntermediateResult.getId() |
IntermediateDataSetID |
PartitionInfo.getIntermediateDataSetID() |
IntermediateDataSetID |
IntermediateResultInfo.getResultId()
Get the intermediate result id.
|
| Modifier and Type | Method and Description |
|---|---|
static Map<IntermediateDataSetID,JobVertexInputInfo> |
VertexInputInfoComputationUtils.computeVertexInputInfos(ExecutionJobVertex ejv,
Function<IntermediateDataSetID,IntermediateResult> intermediateResultRetriever) |
static Map<IntermediateDataSetID,JobVertexInputInfo> |
VertexInputInfoComputationUtils.computeVertexInputInfos(int parallelism,
List<? extends IntermediateResultInfo> inputs,
boolean isDynamicGraph) |
Map<IntermediateDataSetID,IntermediateResult> |
DefaultExecutionGraph.getAllIntermediateResults() |
Map<IntermediateDataSetID,IntermediateResult> |
ExecutionGraph.getAllIntermediateResults() |
Map<IntermediateDataSetID,JobVertexInputInfo> |
ParallelismAndInputInfos.getJobVertexInputInfos() |
| Modifier and Type | Method and Description |
|---|---|
JobVertexInputInfo |
VertexInputInfoStore.get(JobVertexID jobVertexId,
IntermediateDataSetID resultId)
Get a
JobVertexInputInfo. |
List<ShuffleDescriptor> |
DefaultExecutionGraph.getClusterPartitionShuffleDescriptors(IntermediateDataSetID intermediateDataSetID) |
List<ShuffleDescriptor> |
InternalExecutionGraphAccessor.getClusterPartitionShuffleDescriptors(IntermediateDataSetID intermediateResultPartition)
Get the shuffle descriptors of the cluster partitions ordered by partition number.
|
ExecutionVertexInputInfo |
ExecutionVertex.getExecutionVertexInputInfo(IntermediateDataSetID resultId) |
JobVertexInputInfo |
DefaultExecutionGraph.getJobVertexInputInfo(JobVertexID jobVertexId,
IntermediateDataSetID resultId) |
JobVertexInputInfo |
InternalExecutionGraphAccessor.getJobVertexInputInfo(JobVertexID jobVertexId,
IntermediateDataSetID resultId)
Get the input info of a certain input of a certain job vertex.
|
void |
VertexInputInfoStore.put(JobVertexID jobVertexId,
IntermediateDataSetID resultId,
JobVertexInputInfo info)
Put a
JobVertexInputInfo. |
| Modifier and Type | Method and Description |
|---|---|
static Map<IntermediateDataSetID,JobVertexInputInfo> |
VertexInputInfoComputationUtils.computeVertexInputInfos(ExecutionJobVertex ejv,
Function<IntermediateDataSetID,IntermediateResult> intermediateResultRetriever) |
void |
ExecutionJobVertex.connectToPredecessors(Map<IntermediateDataSetID,IntermediateResult> intermediateDataSets) |
void |
DefaultExecutionGraph.initializeJobVertex(ExecutionJobVertex ejv,
long createTimestamp,
Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos) |
void |
ExecutionGraph.initializeJobVertex(ExecutionJobVertex ejv,
long createTimestamp,
Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos)
Initialize the given execution job vertex, mainly includes creating execution vertices
according to the parallelism, and connecting to the predecessors.
|
| Constructor and Description |
|---|
PartitionInfo(IntermediateDataSetID intermediateResultPartitionID,
ShuffleDescriptor shuffleDescriptor) |
| Constructor and Description |
|---|
ParallelismAndInputInfos(int parallelism,
Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos) |
| Modifier and Type | Method and Description |
|---|---|
IntermediateDataSetID |
TaskExecutorPartitionInfo.getIntermediateDataSetId() |
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<Map<IntermediateDataSetID,DataSetMetaInfo>> |
ClusterPartitionManager.listDataSets()
Returns all datasets for which partitions are being tracked.
|
Map<IntermediateDataSetID,DataSetMetaInfo> |
ResourceManagerPartitionTracker.listDataSets()
Returns all data sets for which partitions are being tracked.
|
Map<IntermediateDataSetID,DataSetMetaInfo> |
ResourceManagerPartitionTrackerImpl.listDataSets() |
| Modifier and Type | Method and Description |
|---|---|
List<ShuffleDescriptor> |
ResourceManagerPartitionTracker.getClusterPartitionShuffleDescriptors(IntermediateDataSetID dataSetID)
Returns all the shuffle descriptors of cluster partitions for the intermediate dataset.
|
List<ShuffleDescriptor> |
ResourceManagerPartitionTrackerImpl.getClusterPartitionShuffleDescriptors(IntermediateDataSetID dataSetID) |
List<ShuffleDescriptor> |
JobMasterPartitionTracker.getClusterPartitionShuffleDescriptors(IntermediateDataSetID intermediateDataSetID)
Get the shuffle descriptors of the cluster partitions ordered by partition number.
|
List<ShuffleDescriptor> |
JobMasterPartitionTrackerImpl.getClusterPartitionShuffleDescriptors(IntermediateDataSetID intermediateDataSetID) |
CompletableFuture<List<ShuffleDescriptor>> |
ClusterPartitionManager.getClusterPartitionsShuffleDescriptors(IntermediateDataSetID intermediateDataSetID)
Get the shuffle descriptors of the cluster partitions ordered by partition number.
|
CompletableFuture<Void> |
ClusterPartitionManager.releaseClusterPartitions(IntermediateDataSetID dataSetToRelease)
Releases all partitions associated with the given dataset.
|
CompletableFuture<Void> |
ResourceManagerPartitionTracker.releaseClusterPartitions(IntermediateDataSetID dataSetId)
Issues a release calls to all task executors that are hosting partitions of the given data
set.
|
CompletableFuture<Void> |
ResourceManagerPartitionTrackerImpl.releaseClusterPartitions(IntermediateDataSetID dataSetId) |
void |
PartitionProducerStateProvider.requestPartitionProducerState(IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId,
Consumer<? super PartitionProducerStateProvider.ResponseHandle> responseConsumer)
Trigger the producer execution state request.
|
| Modifier and Type | Method and Description |
|---|---|
void |
TaskExecutorClusterPartitionReleaser.releaseClusterPartitions(ResourceID taskExecutorId,
Set<IntermediateDataSetID> dataSetsToRelease) |
void |
TaskExecutorPartitionTrackerImpl.stopTrackingAndReleaseClusterPartitions(Collection<IntermediateDataSetID> dataSetsToRelease) |
void |
TaskExecutorPartitionTracker.stopTrackingAndReleaseClusterPartitions(Collection<IntermediateDataSetID> dataSetsToRelease)
Releases partitions associated with the given datasets and stops tracking of partitions that
were released.
|
| Constructor and Description |
|---|
TaskExecutorPartitionInfo(ShuffleDescriptor shuffleDescriptor,
IntermediateDataSetID intermediateDataSetId,
int numberOfPartitions) |
| Modifier and Type | Method and Description |
|---|---|
IntermediateDataSetID |
InputGateID.getConsumedResultID() |
| Constructor and Description |
|---|
InputGateID(IntermediateDataSetID consumedResultID,
ExecutionAttemptID consumerID) |
SingleInputGate(String owningTaskName,
int gateIndex,
IntermediateDataSetID consumedResultId,
ResultPartitionType consumedPartitionType,
int numberOfInputChannels,
PartitionProducerStateProvider partitionProducerStateProvider,
org.apache.flink.util.function.SupplierWithException<BufferPool,IOException> bufferPoolFactory,
BufferDecompressor bufferDecompressor,
org.apache.flink.core.memory.MemorySegmentProvider memorySegmentProvider,
int segmentSize,
ThroughputCalculator throughputCalculator,
BufferDebloater bufferDebloater) |
| Modifier and Type | Method and Description |
|---|---|
static IntermediateDataSetID |
TieredStorageIdMappingUtils.convertId(TieredStorageTopicId topicId) |
| Modifier and Type | Method and Description |
|---|---|
static TieredStorageTopicId |
TieredStorageIdMappingUtils.convertId(IntermediateDataSetID intermediateDataSetID) |
| Modifier and Type | Method and Description |
|---|---|
static IntermediateDataSetID |
IntermediateDataSetID.fromByteBuf(org.apache.flink.shaded.netty4.io.netty.buffer.ByteBuf buf) |
IntermediateDataSetID |
IntermediateDataSet.getId() |
IntermediateDataSetID |
IntermediateResultPartitionID.getIntermediateDataSetID() |
IntermediateDataSetID |
JobEdge.getSourceId()
Gets the ID of the consumed data set.
|
| Modifier and Type | Method and Description |
|---|---|
List<IntermediateDataSetID> |
JobVertex.getIntermediateDataSetIdsToConsume() |
| Modifier and Type | Method and Description |
|---|---|
void |
JobVertex.addIntermediateDataSetIdToConsume(IntermediateDataSetID intermediateDataSetId) |
JobEdge |
JobVertex.connectNewDataSetAsInput(JobVertex input,
DistributionPattern distPattern,
ResultPartitionType partitionType,
IntermediateDataSetID intermediateDataSetId,
boolean isBroadcast) |
IntermediateDataSet |
JobVertex.getOrCreateResultDataSet(IntermediateDataSetID id,
ResultPartitionType partitionType) |
| Constructor and Description |
|---|
IntermediateDataSet(IntermediateDataSetID id,
ResultPartitionType resultType,
JobVertex producer) |
IntermediateResultPartitionID(IntermediateDataSetID intermediateDataSetID,
int partitionNum)
Creates an new intermediate result partition ID with
IntermediateDataSetID and the
partitionNum. |
| Modifier and Type | Method and Description |
|---|---|
IntermediateDataSetID |
DefaultLogicalResult.getId() |
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<ExecutionState> |
JobMasterGateway.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID partitionId)
Requests the current state of the partition.
|
CompletableFuture<ExecutionState> |
JobMaster.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
| Modifier and Type | Method and Description |
|---|---|
Map<IntermediateDataSetID,JobVertexInputInfo> |
ExecutionJobVertexInitializedEvent.getJobVertexInputInfos() |
| Constructor and Description |
|---|
ExecutionJobVertexInitializedEvent(JobVertexID jobVertexId,
int parallelism,
Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos) |
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<Map<IntermediateDataSetID,DataSetMetaInfo>> |
ResourceManager.listDataSets() |
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<List<ShuffleDescriptor>> |
ResourceManager.getClusterPartitionsShuffleDescriptors(IntermediateDataSetID intermediateDataSetID) |
CompletableFuture<Void> |
ResourceManager.releaseClusterPartitions(IntermediateDataSetID dataSetId) |
| Modifier and Type | Method and Description |
|---|---|
protected IntermediateDataSetID |
ClusterDataSetIdPathParameter.convertFromString(String value) |
| Modifier and Type | Method and Description |
|---|---|
protected String |
ClusterDataSetIdPathParameter.convertToString(IntermediateDataSetID value) |
| Modifier and Type | Method and Description |
|---|---|
static ClusterDataSetListResponseBody |
ClusterDataSetListResponseBody.from(Map<IntermediateDataSetID,DataSetMetaInfo> dataSets) |
| Modifier and Type | Method and Description |
|---|---|
List<IntermediateDataSetID> |
ClusterDatasetCorruptedException.getCorruptedClusterDatasetIds() |
| Modifier and Type | Method and Description |
|---|---|
ExecutionState |
SchedulerBase.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
ExecutionState |
ExecutionGraphHandler.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
ExecutionState |
SchedulerNG.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
| Constructor and Description |
|---|
ClusterDatasetCorruptedException(Throwable cause,
List<IntermediateDataSetID> corruptedClusterDatasetIds) |
| Modifier and Type | Method and Description |
|---|---|
ExecutionState |
AdaptiveScheduler.requestPartitionState(IntermediateDataSetID intermediateResultId,
ResultPartitionID resultPartitionId) |
| Modifier and Type | Method and Description |
|---|---|
void |
BatchJobRecoveryContext.initializeJobVertex(ExecutionJobVertex jobVertex,
int parallelism,
Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos,
long createTimestamp)
Initializes a given job vertex with the specified parallelism and input information.
|
void |
BatchJobRecoveryHandler.onExecutionJobVertexInitialization(JobVertexID jobVertexId,
int parallelism,
Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos)
Records the job vertex initialization event for use during future batch job recovery.
|
void |
DummyBatchJobRecoveryHandler.onExecutionJobVertexInitialization(JobVertexID jobVertexId,
int parallelism,
Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos) |
void |
DefaultBatchJobRecoveryHandler.onExecutionJobVertexInitialization(JobVertexID jobVertexId,
int parallelism,
Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos) |
| Modifier and Type | Method and Description |
|---|---|
IntermediateDataSetID |
ConsumedPartitionGroup.getIntermediateDataSetID()
Get the ID of IntermediateDataSet this ConsumedPartitionGroup belongs to.
|
IntermediateDataSetID |
SchedulingResultPartition.getResultId()
Gets id of the intermediate result.
|
| Modifier and Type | Method and Description |
|---|---|
IntermediateDataSetID |
PartitionDescriptor.getResultId() |
| Modifier and Type | Method and Description |
|---|---|
Map<IntermediateDataSetID,Integer> |
TaskInputsOutputsDescriptor.getInputChannelNums() |
Map<IntermediateDataSetID,ResultPartitionType> |
TaskInputsOutputsDescriptor.getInputPartitionTypes() |
Map<IntermediateDataSetID,Integer> |
TaskInputsOutputsDescriptor.getPartitionReuseCount() |
Map<IntermediateDataSetID,ResultPartitionType> |
TaskInputsOutputsDescriptor.getPartitionTypes() |
Map<IntermediateDataSetID,Integer> |
TaskInputsOutputsDescriptor.getSubpartitionNums() |
| Modifier and Type | Method and Description |
|---|---|
static int |
NettyShuffleUtils.computeNetworkBuffersForAnnouncing(int numBuffersPerChannel,
int numFloatingBuffersPerGate,
Optional<Integer> maxRequiredBuffersPerGate,
int sortShuffleMinParallelism,
int sortShuffleMinBuffers,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
static int |
NettyShuffleUtils.computeNetworkBuffersForAnnouncing(int numBuffersPerChannel,
int numFloatingBuffersPerGate,
Optional<Integer> maxRequiredBuffersPerGate,
int sortShuffleMinParallelism,
int sortShuffleMinBuffers,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
static int |
NettyShuffleUtils.computeNetworkBuffersForAnnouncing(int numBuffersPerChannel,
int numFloatingBuffersPerGate,
Optional<Integer> maxRequiredBuffersPerGate,
int sortShuffleMinParallelism,
int sortShuffleMinBuffers,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
static int |
NettyShuffleUtils.computeNetworkBuffersForAnnouncing(int numBuffersPerChannel,
int numFloatingBuffersPerGate,
Optional<Integer> maxRequiredBuffersPerGate,
int sortShuffleMinParallelism,
int sortShuffleMinBuffers,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
static int |
NettyShuffleUtils.computeNetworkBuffersForAnnouncing(int numBuffersPerChannel,
int numFloatingBuffersPerGate,
Optional<Integer> maxRequiredBuffersPerGate,
int sortShuffleMinParallelism,
int sortShuffleMinBuffers,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
static TaskInputsOutputsDescriptor |
TaskInputsOutputsDescriptor.from(int inputGateNums,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
static TaskInputsOutputsDescriptor |
TaskInputsOutputsDescriptor.from(int inputGateNums,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
static TaskInputsOutputsDescriptor |
TaskInputsOutputsDescriptor.from(int inputGateNums,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
static TaskInputsOutputsDescriptor |
TaskInputsOutputsDescriptor.from(int inputGateNums,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
static TaskInputsOutputsDescriptor |
TaskInputsOutputsDescriptor.from(int inputGateNums,
Map<IntermediateDataSetID,Integer> inputChannelNums,
Map<IntermediateDataSetID,Integer> partitionReuseCount,
Map<IntermediateDataSetID,Integer> subpartitionNums,
Map<IntermediateDataSetID,ResultPartitionType> inputPartitionTypes,
Map<IntermediateDataSetID,ResultPartitionType> partitionTypes) |
| Constructor and Description |
|---|
PartitionDescriptor(IntermediateDataSetID resultId,
int totalNumberOfPartitions,
IntermediateResultPartitionID partitionId,
ResultPartitionType partitionType,
int numberOfSubpartitions,
int connectionIndex,
boolean isBroadcast,
boolean isAllToAllDistribution,
boolean isNumberOfPartitionConsumerUndefined) |
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<ExecutionState> |
PartitionProducerStateChecker.requestPartitionProducerState(org.apache.flink.api.common.JobID jobId,
IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId)
Requests the execution state of the execution producing a result partition.
|
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<Acknowledge> |
TaskExecutorGatewayDecoratorBase.releaseClusterPartitions(Collection<IntermediateDataSetID> dataSetsToRelease,
org.apache.flink.api.common.time.Time timeout) |
CompletableFuture<Acknowledge> |
TaskExecutor.releaseClusterPartitions(Collection<IntermediateDataSetID> dataSetsToRelease,
org.apache.flink.api.common.time.Time timeout) |
CompletableFuture<Acknowledge> |
TaskExecutorGateway.releaseClusterPartitions(Collection<IntermediateDataSetID> dataSetsToRelease,
org.apache.flink.api.common.time.Time timeout)
Releases all cluster partitions belong to any of the given data sets.
|
| Modifier and Type | Method and Description |
|---|---|
IntermediateDataSetID |
ClusterPartitionReport.ClusterPartitionReportEntry.getDataSetId() |
| Constructor and Description |
|---|
ClusterPartitionReportEntry(IntermediateDataSetID dataSetId,
int numTotalPartitions,
Map<ResultPartitionID,ShuffleDescriptor> shuffleDescriptors) |
| Modifier and Type | Method and Description |
|---|---|
CompletableFuture<ExecutionState> |
RpcPartitionStateChecker.requestPartitionProducerState(org.apache.flink.api.common.JobID jobId,
IntermediateDataSetID resultId,
ResultPartitionID partitionId) |
| Modifier and Type | Method and Description |
|---|---|
void |
Task.requestPartitionProducerState(IntermediateDataSetID intermediateDataSetId,
ResultPartitionID resultPartitionId,
Consumer<? super PartitionProducerStateProvider.ResponseHandle> responseConsumer) |
Copyright © 2014–2025 The Apache Software Foundation. All rights reserved.