org.apache.flink.runtime.resourcemanager.slotmanager

Class FineGrainedSlotManager

java.lang.Object

org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedSlotManager

All Implemented Interfaces:

AutoCloseable, SlotManager

public class FineGrainedSlotManager
extends Object
implements SlotManager

Implementation of SlotManager supporting fine-grained resource management.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager

SlotManager.RegistrationResult

Constructor Summary

Constructors

Constructor and Description

FineGrainedSlotManager(org.apache.flink.util.concurrent.ScheduledExecutor scheduledExecutor, SlotManagerConfiguration slotManagerConfiguration, SlotManagerMetricGroup slotManagerMetricGroup, ResourceTracker resourceTracker, org.apache.flink.runtime.resourcemanager.slotmanager.TaskManagerTracker taskManagerTracker, SlotStatusSyncer slotStatusSyncer, ResourceAllocationStrategy resourceAllocationStrategy)

Method Summary

Modifier and Type	Method and Description
void	clearResourceRequirements(org.apache.flink.api.common.JobID jobId) Notifies the slot manager that the resource requirements for the given job should be cleared.
void	close() Closes the slot manager.
void	freeSlot(SlotID slotId, AllocationID allocationId) Free the given slot from the given allocation.
Collection<SlotInfo>	getAllocatedSlotsOf(InstanceID instanceID)
ResourceProfile	getFreeResource()
ResourceProfile	getFreeResourceOf(InstanceID instanceID)
int	getNumberFreeSlots()
int	getNumberFreeSlotsOf(InstanceID instanceId)
int	getNumberRegisteredSlots()
int	getNumberRegisteredSlotsOf(InstanceID instanceId)
ResourceProfile	getRegisteredResource()
ResourceProfile	getRegisteredResourceOf(InstanceID instanceID)
long	getTaskManagerIdleSince(InstanceID instanceId)
void	processResourceRequirements(ResourceRequirements resourceRequirements) Notifies the slot manager about the resource requirements of a job.
SlotManager.RegistrationResult	registerTaskManager(TaskExecutorConnection taskExecutorConnection, SlotReport initialSlotReport, ResourceProfile totalResourceProfile, ResourceProfile defaultSlotResourceProfile) Registers a new task manager at the slot manager.
boolean	reportSlotStatus(InstanceID instanceId, SlotReport slotReport) Reports the current slot allocations for a task manager identified by the given instance id.
void	setFailUnfulfillableRequest(boolean failUnfulfillableRequest)
void	start(ResourceManagerId newResourceManagerId, Executor newMainThreadExecutor, ResourceAllocator newResourceAllocator, ResourceEventListener newResourceEventListener, BlockedTaskManagerChecker newBlockedTaskManagerChecker) Starts the slot manager with the given leader id and resource manager actions.
void	suspend() Suspends the component.
void	triggerResourceRequirementsCheck() Trigger the resource requirement check.
boolean	unregisterTaskManager(InstanceID instanceId, Exception cause) Unregisters the task manager identified by the given instance id and its associated slots from the slot manager.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FineGrainedSlotManager

public FineGrainedSlotManager(org.apache.flink.util.concurrent.ScheduledExecutor scheduledExecutor,
                              SlotManagerConfiguration slotManagerConfiguration,
                              SlotManagerMetricGroup slotManagerMetricGroup,
                              ResourceTracker resourceTracker,
                              org.apache.flink.runtime.resourcemanager.slotmanager.TaskManagerTracker taskManagerTracker,
                              SlotStatusSyncer slotStatusSyncer,
                              ResourceAllocationStrategy resourceAllocationStrategy)

Method Detail

setFailUnfulfillableRequest

public void setFailUnfulfillableRequest(boolean failUnfulfillableRequest)

Specified by:

setFailUnfulfillableRequest in interface SlotManager

triggerResourceRequirementsCheck

public void triggerResourceRequirementsCheck()

Description copied from interface: SlotManager

Trigger the resource requirement check. This method will be called when some slot statuses changed.

Specified by:

triggerResourceRequirementsCheck in interface SlotManager

start

public void start(ResourceManagerId newResourceManagerId,
                  Executor newMainThreadExecutor,
                  ResourceAllocator newResourceAllocator,
                  ResourceEventListener newResourceEventListener,
                  BlockedTaskManagerChecker newBlockedTaskManagerChecker)

Starts the slot manager with the given leader id and resource manager actions.

Specified by:

start in interface SlotManager

Parameters:

newResourceManagerId - to use for communication with the task managers

newMainThreadExecutor - to use to run code in the ResourceManager's main thread

newResourceAllocator - to use for resource (de-)allocations

newBlockedTaskManagerChecker - to query whether a task manager is blocked

newResourceEventListener - to use for notify resource not enough

suspend

public void suspend()

Suspends the component. This clears the internal state of the slot manager.

Specified by:

suspend in interface SlotManager

close

public void close()
           throws Exception

Closes the slot manager.

Specified by:

close in interface AutoCloseable

Throws:

Exception - if the close operation fails

clearResourceRequirements

public void clearResourceRequirements(org.apache.flink.api.common.JobID jobId)

Description copied from interface: SlotManager

Notifies the slot manager that the resource requirements for the given job should be cleared. The slot manager may assume that no further updates to the resource requirements will occur.

Specified by:

clearResourceRequirements in interface SlotManager

Parameters:

jobId - job for which to clear the requirements

processResourceRequirements

public void processResourceRequirements(ResourceRequirements resourceRequirements)

Description copied from interface: SlotManager

Notifies the slot manager about the resource requirements of a job.

Specified by:

processResourceRequirements in interface SlotManager

Parameters:

resourceRequirements - resource requirements of a job

registerTaskManager

public SlotManager.RegistrationResult registerTaskManager(TaskExecutorConnection taskExecutorConnection,
                                                          SlotReport initialSlotReport,
                                                          ResourceProfile totalResourceProfile,
                                                          ResourceProfile defaultSlotResourceProfile)

Description copied from interface: SlotManager

Registers a new task manager at the slot manager. This will make the task managers slots known and, thus, available for allocation.

Specified by:

registerTaskManager in interface SlotManager

Parameters:

taskExecutorConnection - for the new task manager

initialSlotReport - for the new task manager

totalResourceProfile - for the new task manager

defaultSlotResourceProfile - for the new task manager

Returns:

The result of task manager registration

unregisterTaskManager

public boolean unregisterTaskManager(InstanceID instanceId,
                                     Exception cause)

Description copied from interface: SlotManager

Unregisters the task manager identified by the given instance id and its associated slots from the slot manager.

Specified by:

unregisterTaskManager in interface SlotManager

Parameters:

instanceId - identifying the task manager to unregister

cause - for unregistering the TaskManager

Returns:

True if there existed a registered task manager with the given instance id

reportSlotStatus

public boolean reportSlotStatus(InstanceID instanceId,
                                SlotReport slotReport)

Reports the current slot allocations for a task manager identified by the given instance id.

Specified by:

reportSlotStatus in interface SlotManager

Parameters:

instanceId - identifying the task manager for which to report the slot status

slotReport - containing the status for all of its slots

Returns:

true if the slot status has been updated successfully, otherwise false

freeSlot

public void freeSlot(SlotID slotId,
                     AllocationID allocationId)

Free the given slot from the given allocation. If the slot is still allocated by the given allocation id, then the slot will be freed.

Specified by:

freeSlot in interface SlotManager

Parameters:

slotId - identifying the slot to free, will be ignored

allocationId - with which the slot is presumably allocated

getNumberRegisteredSlots

public int getNumberRegisteredSlots()

Specified by:

getNumberRegisteredSlots in interface SlotManager

getNumberRegisteredSlotsOf

public int getNumberRegisteredSlotsOf(InstanceID instanceId)

Specified by:

getNumberRegisteredSlotsOf in interface SlotManager

getNumberFreeSlots

public int getNumberFreeSlots()

Specified by:

getNumberFreeSlots in interface SlotManager

getNumberFreeSlotsOf

public int getNumberFreeSlotsOf(InstanceID instanceId)

Specified by:

getNumberFreeSlotsOf in interface SlotManager

getRegisteredResource

public ResourceProfile getRegisteredResource()

Specified by:

getRegisteredResource in interface SlotManager

getRegisteredResourceOf

public ResourceProfile getRegisteredResourceOf(InstanceID instanceID)

Specified by:

getRegisteredResourceOf in interface SlotManager

getFreeResource

public ResourceProfile getFreeResource()

Specified by:

getFreeResource in interface SlotManager

getFreeResourceOf

public ResourceProfile getFreeResourceOf(InstanceID instanceID)

Specified by:

getFreeResourceOf in interface SlotManager

getAllocatedSlotsOf

public Collection<SlotInfo> getAllocatedSlotsOf(InstanceID instanceID)

Specified by:

getAllocatedSlotsOf in interface SlotManager

getTaskManagerIdleSince

@VisibleForTesting
public long getTaskManagerIdleSince(InstanceID instanceId)