Package org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier

Interface TierMasterAgent

All Known Implementing Classes:

NoOpMasterAgent, RemoteTierMasterAgent

public interface TierMasterAgent

The master-side agent of a Tier.

Method Summary

Modifier and Type	Method	Description
TierShuffleDescriptor	addPartitionAndGetShuffleDescriptor(org.apache.flink.api.common.JobID jobID, int numSubpartitions, ResultPartitionID resultPartitionID)	Add a new tiered storage partition and get the TierShuffleDescriptor.
void	close()	Close this tier master agent.
default CompletableFuture<Map<ResultPartitionID,ShuffleMetrics>>	getPartitionWithMetrics(org.apache.flink.api.common.JobID jobId, Duration timeout, Set<ResultPartitionID> expectedPartitions)	Retrieves specified partitions and their metrics (identified by expectedPartitions), the metrics include sizes of sub-partitions in a result partition.
default boolean	partitionInRemote()	Is this tier manage the partition in remote cluster instead of flink taskmanager.
void	registerJob(org.apache.flink.api.common.JobID jobID, TierShuffleHandler tierShuffleHandler)	Register a job id with a TierShuffleHandler.
void	releasePartition(TierShuffleDescriptor shuffleDescriptor)	Release a tiered storage partition.
default void	restoreState(TieredShuffleMasterSnapshot snapshot)	Restores the state of the tier master agent from the provided snapshots for the specified job.
default void	restoreState(TieredShuffleMasterSnapshot snapshot, org.apache.flink.api.common.JobID jobId)	Restores the state of the tier master agent from the provided snapshots.
default void	snapshotState(CompletableFuture<TieredShuffleMasterSnapshot> snapshotFuture)	Triggers a snapshot of the tier master agent's state.
default void	snapshotState(CompletableFuture<TieredShuffleMasterSnapshot> snapshotFuture, ShuffleMasterSnapshotContext context, org.apache.flink.api.common.JobID jobId)	Triggers a snapshot of the tier master agent's state which related the specified job.
void	unregisterJob(org.apache.flink.api.common.JobID jobID)	Unregister a job id.

Method Detail

registerJob

void registerJob(org.apache.flink.api.common.JobID jobID,
                 TierShuffleHandler tierShuffleHandler)

Register a job id with a TierShuffleHandler.

unregisterJob

void unregisterJob(org.apache.flink.api.common.JobID jobID)

Unregister a job id.

addPartitionAndGetShuffleDescriptor

TierShuffleDescriptor addPartitionAndGetShuffleDescriptor(org.apache.flink.api.common.JobID jobID,
                                                          int numSubpartitions,
                                                          ResultPartitionID resultPartitionID)

Add a new tiered storage partition and get the TierShuffleDescriptor.

snapshotState

default void snapshotState(CompletableFuture<TieredShuffleMasterSnapshot> snapshotFuture,
                           ShuffleMasterSnapshotContext context,
                           org.apache.flink.api.common.JobID jobId)

Triggers a snapshot of the tier master agent's state which related the specified job.

snapshotState

default void snapshotState(CompletableFuture<TieredShuffleMasterSnapshot> snapshotFuture)

Triggers a snapshot of the tier master agent's state.

restoreState

default void restoreState(TieredShuffleMasterSnapshot snapshot,
                          org.apache.flink.api.common.JobID jobId)

Restores the state of the tier master agent from the provided snapshots.

restoreState

default void restoreState(TieredShuffleMasterSnapshot snapshot)

Restores the state of the tier master agent from the provided snapshots for the specified job.

getPartitionWithMetrics

default CompletableFuture<Map<ResultPartitionID,ShuffleMetrics>> getPartitionWithMetrics(org.apache.flink.api.common.JobID jobId,
                                                                                               Duration timeout,
                                                                                               Set<ResultPartitionID> expectedPartitions)

Retrieves specified partitions and their metrics (identified by expectedPartitions), the metrics include sizes of sub-partitions in a result partition.

Parameters:

jobId - ID of the target job

timeout - The timeout used for retrieve the specified partitions.

expectedPartitions - The set of identifiers for the result partitions whose metrics are to be fetched.

Returns:

A future will contain a map of the partitions with their metrics that could be retrieved from the expected partitions within the specified timeout period.

releasePartition

void releasePartition(TierShuffleDescriptor shuffleDescriptor)

Release a tiered storage partition.

Parameters:

shuffleDescriptor - the partition shuffle descriptor to be released

close

void close()

Close this tier master agent.

partitionInRemote

default boolean partitionInRemote()

Is this tier manage the partition in remote cluster instead of flink taskmanager.