org.apache.flink.runtime.scheduler

Class DefaultScheduler

java.lang.Object

org.apache.flink.runtime.scheduler.SchedulerBase

org.apache.flink.runtime.scheduler.DefaultScheduler

All Implemented Interfaces:

AutoCloseable, CheckpointScheduling, GlobalFailureHandler, SchedulerNG, SchedulerOperations, org.apache.flink.util.AutoCloseableAsync

Direct Known Subclasses:

AdaptiveBatchScheduler

public class DefaultScheduler
extends SchedulerBase
implements SchedulerOperations

The future default scheduler.

Field Summary

Fields

Modifier and Type	Field and Description
protected ExecutionDeployer	executionDeployer
protected ExecutionSlotAllocator	executionSlotAllocator
protected org.slf4j.Logger	log

Fields inherited from class org.apache.flink.runtime.scheduler.SchedulerBase

executionVertexVersioner, inputsLocationsRetriever, jobInfo, jobManagerJobMetricGroup, operatorCoordinatorHandler, stateLocationRetriever

Constructor Summary

Constructors

Modifier

Constructor and Description

protected

DefaultScheduler(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, java.util.function.Consumer<org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor> startUpAction, org.apache.flink.util.concurrent.ScheduledExecutor delayExecutor, ClassLoader userCodeLoader, CheckpointsCleaner checkpointsCleaner, CheckpointRecoveryFactory checkpointRecoveryFactory, JobManagerJobMetricGroup jobManagerJobMetricGroup, SchedulingStrategyFactory schedulingStrategyFactory, FailoverStrategy.Factory failoverStrategyFactory, RestartBackoffTimeStrategy restartBackoffTimeStrategy, ExecutionOperations executionOperations, ExecutionVertexVersioner executionVertexVersioner, ExecutionSlotAllocatorFactory executionSlotAllocatorFactory, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, JobStatusListener jobStatusListener, Collection<org.apache.flink.core.failure.FailureEnricher> failureEnrichers, ExecutionGraphFactory executionGraphFactory, ShuffleMaster<?> shuffleMaster, org.apache.flink.api.common.time.Time rpcTimeout, VertexParallelismStore vertexParallelismStore, ExecutionDeployer.Factory executionDeployerFactory)

Method Summary

Modifier and Type	Method and Description
void	allocateSlotsAndDeploy(List<ExecutionVertexID> verticesToDeploy) Allocate slots and deploy the vertex when slots are returned.
protected void	cancelAllPendingSlotRequestsForVertex(ExecutionVertexID executionVertexId)
protected void	cancelAllPendingSlotRequestsInternal()
protected CompletableFuture<?>	cancelExecution(Execution execution)
protected FailureHandlingResultSnapshot	createFailureHandlingResultSnapshot(FailureHandlingResult failureHandlingResult)
protected long	getNumberOfRestarts()
void	handleGlobalFailure(Throwable error) Handles a global failure.
protected void	handleTaskFailure(Execution failedExecution, Throwable error)
protected void	onTaskFailed(Execution execution)
protected void	onTaskFinished(Execution execution, IOMetrics ioMetrics)
protected FailureHandlingResult	recordTaskFailure(Execution failedExecution, Throwable error)
protected void	startSchedulingInternal()

Methods inherited from class org.apache.flink.runtime.scheduler.SchedulerBase

acknowledgeCheckpoint, archiveFromFailureHandlingResult, archiveGlobalFailure, cancel, closeAsync, computeVertexParallelismStore, computeVertexParallelismStore, computeVertexParallelismStore, computeVertexParallelismStore, declineCheckpoint, deliverCoordinationRequestToCoordinator, deliverOperatorEventToCoordinator, failJob, getDefaultMaxParallelism, getExceptionHistory, getExecutionGraph, getExecutionJobVertex, getExecutionVertex, getJobGraph, getJobTerminationFuture, getMainThreadExecutor, getMarkPartitionFinishedStrategy, getResultPartitionAvailabilityChecker, getSchedulingTopology, notifyEndOfData, notifyKvStateRegistered, notifyKvStateUnregistered, registerJobMetrics, reportCheckpointMetrics, reportInitializationMetrics, requestCheckpointStats, requestJob, requestJobStatus, requestKvStateLocation, requestNextInputSplit, requestPartitionState, resetForNewExecution, resetForNewExecutions, restoreState, setGlobalFailureCause, startCheckpointScheduler, startScheduling, stopCheckpointScheduler, stopWithSavepoint, transitionExecutionGraphState, transitionToRunning, transitionToScheduled, triggerCheckpoint, triggerSavepoint, updateAccumulators, updateTaskExecutionState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.scheduler.SchedulerNG

requestJobResourceRequirements, updateJobResourceRequirements, updateTaskExecutionState

Methods inherited from interface org.apache.flink.util.AutoCloseableAsync

close

Field Detail

log

protected final org.slf4j.Logger log

executionSlotAllocator

protected final ExecutionSlotAllocator executionSlotAllocator

executionDeployer

protected final ExecutionDeployer executionDeployer

Constructor Detail

DefaultScheduler

protected DefaultScheduler(org.slf4j.Logger log,
                           JobGraph jobGraph,
                           Executor ioExecutor,
                           org.apache.flink.configuration.Configuration jobMasterConfiguration,
                           java.util.function.Consumer<org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor> startUpAction,
                           org.apache.flink.util.concurrent.ScheduledExecutor delayExecutor,
                           ClassLoader userCodeLoader,
                           CheckpointsCleaner checkpointsCleaner,
                           CheckpointRecoveryFactory checkpointRecoveryFactory,
                           JobManagerJobMetricGroup jobManagerJobMetricGroup,
                           SchedulingStrategyFactory schedulingStrategyFactory,
                           FailoverStrategy.Factory failoverStrategyFactory,
                           RestartBackoffTimeStrategy restartBackoffTimeStrategy,
                           ExecutionOperations executionOperations,
                           ExecutionVertexVersioner executionVertexVersioner,
                           ExecutionSlotAllocatorFactory executionSlotAllocatorFactory,
                           long initializationTimestamp,
                           org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor,
                           JobStatusListener jobStatusListener,
                           Collection<org.apache.flink.core.failure.FailureEnricher> failureEnrichers,
                           ExecutionGraphFactory executionGraphFactory,
                           ShuffleMaster<?> shuffleMaster,
                           org.apache.flink.api.common.time.Time rpcTimeout,
                           VertexParallelismStore vertexParallelismStore,
                           ExecutionDeployer.Factory executionDeployerFactory)
                    throws Exception

Throws:

Exception

Method Detail

getNumberOfRestarts

protected long getNumberOfRestarts()

Specified by:

getNumberOfRestarts in class SchedulerBase

cancelAllPendingSlotRequestsInternal

protected void cancelAllPendingSlotRequestsInternal()

Specified by:

cancelAllPendingSlotRequestsInternal in class SchedulerBase

startSchedulingInternal

protected void startSchedulingInternal()

Specified by:

startSchedulingInternal in class SchedulerBase

onTaskFinished

protected void onTaskFinished(Execution execution,
                              IOMetrics ioMetrics)

Specified by:

onTaskFinished in class SchedulerBase

onTaskFailed

protected void onTaskFailed(Execution execution)

Specified by:

onTaskFailed in class SchedulerBase

handleTaskFailure

protected void handleTaskFailure(Execution failedExecution,
                                 @Nullable
                                 Throwable error)

recordTaskFailure

protected FailureHandlingResult recordTaskFailure(Execution failedExecution,
                                                  @Nullable
                                                  Throwable error)

handleGlobalFailure

public void handleGlobalFailure(Throwable error)

Description copied from interface: GlobalFailureHandler

Handles a global failure.

Specified by:

handleGlobalFailure in interface GlobalFailureHandler

Parameters:

error - A cause that describes the global failure.

createFailureHandlingResultSnapshot

protected FailureHandlingResultSnapshot createFailureHandlingResultSnapshot(FailureHandlingResult failureHandlingResult)

cancelExecution

protected CompletableFuture<?> cancelExecution(Execution execution)

cancelAllPendingSlotRequestsForVertex

protected void cancelAllPendingSlotRequestsForVertex(ExecutionVertexID executionVertexId)

allocateSlotsAndDeploy

public void allocateSlotsAndDeploy(List<ExecutionVertexID> verticesToDeploy)

Description copied from interface: SchedulerOperations

Allocate slots and deploy the vertex when slots are returned. Vertices will be deployed only after all of them have been assigned slots. The given order will be respected, i.e. tasks with smaller indices will be deployed earlier. Only vertices in CREATED state will be accepted. Errors will happen if scheduling Non-CREATED vertices.

Specified by:

allocateSlotsAndDeploy in interface SchedulerOperations

Parameters:

verticesToDeploy - The execution vertices to deploy