org.apache.flink.runtime.scheduler.adaptivebatch

Interface BatchJobRecoveryContext

public interface BatchJobRecoveryContext

Context for batch job recovery.

Method Summary

Modifier and Type	Method and Description
void	failJob(Throwable cause, long timestamp, CompletableFuture<Map<String,String>> failureLabels) Trigger job failure.
ExecutionGraph	getExecutionGraph() Provides the ExecutionGraph associated with the job.
org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor	getMainThreadExecutor() Provides the main thread executor.
ShuffleMaster<?>	getShuffleMaster() Provides the ShuffleMaster associated with the job.
Set<ExecutionVertexID>	getTasksNeedingRestart(ExecutionVertexID vertexId, boolean considerResultConsumable) Retrieves a set of vertices that need to be restarted.
void	initializeJobVertex(ExecutionJobVertex jobVertex, int parallelism, Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos, long createTimestamp) Initializes a given job vertex with the specified parallelism and input information.
void	onRecoveringFailed() Notifies the recovery failed.
void	onRecoveringFinished(Set<JobVertexID> jobVerticesWithUnRecoveredCoordinators) Notifies the recovery finished.
void	resetVerticesInRecovering(Set<ExecutionVertexID> verticesToReset) Resets vertices specified by their IDs during recovery process.
void	updateResultPartitionBytesMetrics(Map<IntermediateResultPartitionID,ResultPartitionBytes> resultPartitionBytes) Updates the metrics related to the result partition sizes.
void	updateTopology(List<ExecutionJobVertex> newlyInitializedJobVertices) Updates the job topology with new job vertices that were initialized.

Method Detail

getExecutionGraph

ExecutionGraph getExecutionGraph()

Provides the ExecutionGraph associated with the job.

Returns:

The execution graph.

getShuffleMaster

ShuffleMaster<?> getShuffleMaster()

Provides the ShuffleMaster associated with the job.

Returns:

The shuffle master.

getMainThreadExecutor

org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor getMainThreadExecutor()

Provides the main thread executor.

Returns:

The main thread executor.

getTasksNeedingRestart

Set<ExecutionVertexID> getTasksNeedingRestart(ExecutionVertexID vertexId,
                                              boolean considerResultConsumable)

Retrieves a set of vertices that need to be restarted. If result consumption is considered (`basedOnResultConsumable` is true), the set will include all downstream vertices that have finished and upstream vertices that have missed partitions. Otherwise, only include downstream finished vertices.

Parameters:

vertexId - The ID of the vertex from which to compute the restart set.

considerResultConsumable - Indicates whether to consider result partition consumption while computing the vertices needing restart.

Returns:

A set of vertex IDs that need to be restarted.

resetVerticesInRecovering

void resetVerticesInRecovering(Set<ExecutionVertexID> verticesToReset)
                        throws Exception

Resets vertices specified by their IDs during recovery process.

Parameters:

verticesToReset - The set of vertices that require resetting.

Throws:

Exception

updateResultPartitionBytesMetrics

void updateResultPartitionBytesMetrics(Map<IntermediateResultPartitionID,ResultPartitionBytes> resultPartitionBytes)

Updates the metrics related to the result partition sizes.

Parameters:

resultPartitionBytes - Mapping of partition IDs to their respective result partition bytes.

initializeJobVertex

void initializeJobVertex(ExecutionJobVertex jobVertex,
                         int parallelism,
                         Map<IntermediateDataSetID,JobVertexInputInfo> jobVertexInputInfos,
                         long createTimestamp)
                  throws JobException

Initializes a given job vertex with the specified parallelism and input information.

Parameters:

jobVertex - The job vertex to initialize.

parallelism - The parallelism to set for the job vertex.

jobVertexInputInfos - The input information for the job vertex.

createTimestamp - The timestamp marking the creation of the job vertex.

Throws:

JobException

updateTopology

void updateTopology(List<ExecutionJobVertex> newlyInitializedJobVertices)

Updates the job topology with new job vertices that were initialized.

Parameters:

newlyInitializedJobVertices - List of job vertices that have been initialized.

onRecoveringFinished

void onRecoveringFinished(Set<JobVertexID> jobVerticesWithUnRecoveredCoordinators)

Notifies the recovery finished.

Parameters:

jobVerticesWithUnRecoveredCoordinators - A set of job vertex Ids is associated with job vertices whose operatorCoordinators did not successfully recover their state. If any execution within these job vertices needs to be restarted in the future, all other executions within the same job vertex must also be restarted to ensure the consistency and correctness of the state.

onRecoveringFailed

void onRecoveringFailed()

Notifies the recovery failed.

failJob

void failJob(Throwable cause,
             long timestamp,
             CompletableFuture<Map<String,String>> failureLabels)

Trigger job failure.