Package org.apache.flink.runtime.io.network.partition.hybrid.tiered.shuffle

Class TieredInternalShuffleMaster

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.tiered.shuffle.TieredInternalShuffleMaster

public class TieredInternalShuffleMaster
extends Object

A wrapper internal shuffle master class for tiered storage. All the tiered storage operations with the shuffle master should be wrapped in this class.

Constructor Summary

Constructors

Constructor	Description
TieredInternalShuffleMaster(ShuffleMasterContext shuffleMasterContext, ShuffleDescriptorRetriever shuffleDescriptorRetriever)

Method Summary

Modifier and Type	Method	Description
List<TierShuffleDescriptor>	addPartitionAndGetShuffleDescriptor(org.apache.flink.api.common.JobID jobID, int numSubpartitions, ResultPartitionID resultPartitionID)
void	close()
CompletableFuture<Collection<PartitionWithMetrics>>	getPartitionWithMetrics(JobShuffleContext jobShuffleContext, Duration timeout, Set<ResultPartitionID> expectedPartitions)
void	registerJob(JobShuffleContext context)	Registers the target job together with the corresponding JobShuffleContext to this shuffle master.
void	releasePartition(ShuffleDescriptor shuffleDescriptor)
void	restoreState(List<TieredInternalShuffleMasterSnapshot> snapshots, org.apache.flink.api.common.JobID jobId)
void	restoreState(TieredInternalShuffleMasterSnapshot clusterSnapshot)
void	snapshotState(CompletableFuture<AllTieredShuffleMasterSnapshots> snapshotFuture)
void	snapshotState(CompletableFuture<AllTieredShuffleMasterSnapshots> snapshotFuture, ShuffleMasterSnapshotContext context, org.apache.flink.api.common.JobID jobId)
boolean	supportsBatchSnapshot()
void	unregisterJob(org.apache.flink.api.common.JobID jobID)	Unregisters the target job from this shuffle master, which means the corresponding job has reached a global termination state and all the allocated resources except for the cluster partitions can be cleared.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TieredInternalShuffleMaster

public TieredInternalShuffleMaster(ShuffleMasterContext shuffleMasterContext,
                                   ShuffleDescriptorRetriever shuffleDescriptorRetriever)

Method Detail

supportsBatchSnapshot

public boolean supportsBatchSnapshot()

snapshotState

public void snapshotState(CompletableFuture<AllTieredShuffleMasterSnapshots> snapshotFuture,
                          ShuffleMasterSnapshotContext context,
                          org.apache.flink.api.common.JobID jobId)

snapshotState

public void snapshotState(CompletableFuture<AllTieredShuffleMasterSnapshots> snapshotFuture)

restoreState

public void restoreState(List<TieredInternalShuffleMasterSnapshot> snapshots,
                         org.apache.flink.api.common.JobID jobId)

restoreState

public void restoreState(TieredInternalShuffleMasterSnapshot clusterSnapshot)

getPartitionWithMetrics

public CompletableFuture<Collection<PartitionWithMetrics>> getPartitionWithMetrics(JobShuffleContext jobShuffleContext,
                                                                                   Duration timeout,
                                                                                   Set<ResultPartitionID> expectedPartitions)

registerJob

public void registerJob(JobShuffleContext context)

Registers the target job together with the corresponding JobShuffleContext to this shuffle master.

unregisterJob

public void unregisterJob(org.apache.flink.api.common.JobID jobID)

Unregisters the target job from this shuffle master, which means the corresponding job has reached a global termination state and all the allocated resources except for the cluster partitions can be cleared.

Parameters:

jobID - ID of the target job to be unregistered.

addPartitionAndGetShuffleDescriptor

public List<TierShuffleDescriptor> addPartitionAndGetShuffleDescriptor(org.apache.flink.api.common.JobID jobID,
                                                                       int numSubpartitions,
                                                                       ResultPartitionID resultPartitionID)

releasePartition

public void releasePartition(ShuffleDescriptor shuffleDescriptor)

close

public void close()