Package org.apache.flink.runtime.jobmaster.slotpool

Class DeclarativeSlotPoolBridge

java.lang.Object

org.apache.flink.runtime.jobmaster.slotpool.DeclarativeSlotPoolService

org.apache.flink.runtime.jobmaster.slotpool.DeclarativeSlotPoolBridge

All Implemented Interfaces:

AutoCloseable, AllocatedSlotActions, SlotPool, SlotPoolService

public class DeclarativeSlotPoolBridge
extends DeclarativeSlotPoolService
implements SlotPool

SlotPool implementation which uses the DeclarativeSlotPool to allocate slots.

Field Summary

Fields inherited from class org.apache.flink.runtime.jobmaster.slotpool.DeclarativeSlotPoolService

componentMainThreadExecutor, log

Constructor Summary

Constructors

Constructor

Description

DeclarativeSlotPoolBridge(org.apache.flink.api.common.JobID jobId, DeclarativeSlotPoolFactory declarativeSlotPoolFactory, org.apache.flink.util.clock.Clock clock, Duration rpcTimeout, Duration idleSlotTimeout, Duration batchSlotTimeout, RequestSlotMatchingStrategy requestSlotMatchingStrategy, Duration slotRequestMaxInterval, boolean slotBatchAllocatable, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor componentMainThreadExecutor)

Method Summary

Modifier and Type	Method	Description
Optional<PhysicalSlot>	allocateAvailableSlot(SlotRequestId slotRequestId, AllocationID allocationID, ResourceProfile requirementProfile)	Allocates the available slot with the given allocation id under the given request id for the given requirement profile.
<T> Optional<T>	castInto(Class<T> clazz)	Tries to cast this slot pool service into the given clazz.
void	disableBatchSlotRequestTimeoutCheck()	Disables batch slot request timeout check.
Collection<SlotInfo>	getAllocatedSlotsInformation()	Returns a list of SlotInfo objects about all slots that are currently allocated in the slot pool.
FreeSlotTracker	getFreeSlotTracker()	Returns all free slot tracker.
int	getNumPendingRequests()
void	notifyNotEnoughResourcesAvailable(Collection<ResourceRequirement> acquiredResources)	Notifies that not enough resources are available to fulfill the resource requirements.
Collection<SlotOffer>	offerSlots(TaskManagerLocation taskManagerLocation, TaskManagerGateway taskManagerGateway, Collection<SlotOffer> offers)	Offers multiple slots to the SlotPoolService.
protected void	onClose()	This method is called when the slot pool service is closed.
protected void	onFailAllocation(ResourceCounter previouslyFulfilledRequirements)	This method is called when an allocation fails.
protected void	onReleaseTaskManager(ResourceCounter previouslyFulfilledRequirement)	This method is called when a TaskManager is released.
protected void	onStart()	This method is called when the slot pool service is started.
void	releaseSlot(SlotRequestId slotRequestId, Throwable cause)	Releases the slot with the given SlotRequestId.
CompletableFuture<PhysicalSlot>	requestNewAllocatedBatchSlot(SlotRequestId slotRequestId, ResourceProfile resourceProfile, Collection<AllocationID> preferredAllocations)
CompletableFuture<PhysicalSlot>	requestNewAllocatedSlot(SlotRequestId slotRequestId, ResourceProfile resourceProfile, Collection<AllocationID> preferredAllocations, Duration timeout)	Request the allocation of a new slot from the resource manager.
void	setIsJobRestarting(boolean isJobRestarting)	To set whether the underlying is currently restarting or not.

Methods inherited from class org.apache.flink.runtime.jobmaster.slotpool.DeclarativeSlotPoolService

assertHasBeenStarted, close, connectToResourceManager, createAllocatedSlotReport, disconnectResourceManager, failAllocation, getDeclarativeSlotPool, getRelativeTimeMillis, getSlotServiceStatus, registerTaskManager, releaseFreeSlotsOnTaskManager, releaseTaskManager, start

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.jobmaster.slotpool.SlotPool

close, connectToResourceManager, createAllocatedSlotReport, disconnectResourceManager, registerTaskManager, releaseTaskManager, requestNewAllocatedBatchSlot, requestNewAllocatedSlot, start

Constructor Detail

DeclarativeSlotPoolBridge

public DeclarativeSlotPoolBridge(org.apache.flink.api.common.JobID jobId,
                                 DeclarativeSlotPoolFactory declarativeSlotPoolFactory,
                                 org.apache.flink.util.clock.Clock clock,
                                 Duration rpcTimeout,
                                 Duration idleSlotTimeout,
                                 Duration batchSlotTimeout,
                                 RequestSlotMatchingStrategy requestSlotMatchingStrategy,
                                 Duration slotRequestMaxInterval,
                                 boolean slotBatchAllocatable,
                                 @Nonnull
                                 org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor componentMainThreadExecutor)

Method Detail

castInto

public <T> Optional<T> castInto(Class<T> clazz)

Description copied from interface: SlotPoolService

Tries to cast this slot pool service into the given clazz.

Specified by:

castInto in interface SlotPoolService

Overrides:

castInto in class DeclarativeSlotPoolService

Type Parameters:

T - type of clazz

Parameters:

clazz - to cast the slot pool service into

Returns:

Optional.of(T) the target type if it can be cast; otherwise Optional.empty()

onStart

protected void onStart()

Description copied from class: DeclarativeSlotPoolService

This method is called when the slot pool service is started. It can be overridden by subclasses.

Overrides:

onStart in class DeclarativeSlotPoolService

onClose

protected void onClose()

Description copied from class: DeclarativeSlotPoolService

This method is called when the slot pool service is closed. It can be overridden by subclasses.

Overrides:

onClose in class DeclarativeSlotPoolService

setIsJobRestarting

public void setIsJobRestarting(boolean isJobRestarting)

To set whether the underlying is currently restarting or not. In the former case the slot pool bridge will accept all incoming slot offers.

Specified by:

setIsJobRestarting in interface SlotPool

Parameters:

isJobRestarting - whether this is restarting or not

offerSlots

public Collection<SlotOffer> offerSlots(TaskManagerLocation taskManagerLocation,
                                        TaskManagerGateway taskManagerGateway,
                                        Collection<SlotOffer> offers)

Description copied from interface: SlotPoolService

Offers multiple slots to the SlotPoolService. The slot offerings can be individually accepted or rejected by returning the collection of accepted slot offers.

Specified by:

offerSlots in interface SlotPool

Specified by:

offerSlots in interface SlotPoolService

Overrides:

offerSlots in class DeclarativeSlotPoolService

Parameters:

taskManagerLocation - from which the slot offers originate

taskManagerGateway - to talk to the slot offerer

offers - slot offers which are offered to the SlotPoolService

Returns:

A collection of accepted slot offers. The remaining slot offers are implicitly rejected.

onReleaseTaskManager

protected void onReleaseTaskManager(ResourceCounter previouslyFulfilledRequirement)

Description copied from class: DeclarativeSlotPoolService

This method is called when a TaskManager is released. It can be overridden by subclasses.

Overrides:

onReleaseTaskManager in class DeclarativeSlotPoolService

Parameters:

previouslyFulfilledRequirement - previouslyFulfilledRequirement by the released TaskManager

allocateAvailableSlot

public Optional<PhysicalSlot> allocateAvailableSlot(@Nonnull
                                                    SlotRequestId slotRequestId,
                                                    @Nonnull
                                                    AllocationID allocationID,
                                                    @Nonnull
                                                    ResourceProfile requirementProfile)

Description copied from interface: SlotPool

Allocates the available slot with the given allocation id under the given request id for the given requirement profile. The slot must be able to fulfill the requirement profile, otherwise an IllegalStateException will be thrown.

Specified by:

allocateAvailableSlot in interface SlotPool

Parameters:

slotRequestId - identifying the requested slot

allocationID - the allocation id of the requested available slot

requirementProfile - resource profile of the requirement for which to allocate the slot

Returns:

the previously available slot with the given allocation id, if a slot with this allocation id exists

requestNewAllocatedSlot

@Nonnull
public CompletableFuture<PhysicalSlot> requestNewAllocatedSlot(@Nonnull
                                                               SlotRequestId slotRequestId,
                                                               @Nonnull
                                                               ResourceProfile resourceProfile,
                                                               @Nonnull
                                                               Collection<AllocationID> preferredAllocations,
                                                               @Nullable
                                                               Duration timeout)

Description copied from interface: SlotPool

Request the allocation of a new slot from the resource manager. This method will not return a slot from the already available slots from the pool, but instead will add a new slot to that pool that is immediately allocated and returned.

Specified by:

requestNewAllocatedSlot in interface SlotPool

Parameters:

slotRequestId - identifying the requested slot

resourceProfile - resource profile that specifies the resource requirements for the requested slot

preferredAllocations - preferred allocations for the new allocated slot

timeout - timeout for the allocation procedure

Returns:

a newly allocated slot that was previously not available.

requestNewAllocatedBatchSlot

@Nonnull
public CompletableFuture<PhysicalSlot> requestNewAllocatedBatchSlot(@Nonnull
                                                                    SlotRequestId slotRequestId,
                                                                    @Nonnull
                                                                    ResourceProfile resourceProfile,
                                                                    @Nonnull
                                                                    Collection<AllocationID> preferredAllocations)

Specified by:

requestNewAllocatedBatchSlot in interface SlotPool

onFailAllocation

protected void onFailAllocation(ResourceCounter previouslyFulfilledRequirements)

Description copied from class: DeclarativeSlotPoolService

This method is called when an allocation fails. It can be overridden by subclasses.

Overrides:

onFailAllocation in class DeclarativeSlotPoolService

Parameters:

previouslyFulfilledRequirements - previouslyFulfilledRequirements by the failed allocation

releaseSlot

public void releaseSlot(@Nonnull
                        SlotRequestId slotRequestId,
                        @Nullable
                        Throwable cause)

Description copied from interface: AllocatedSlotActions

Releases the slot with the given SlotRequestId. Additionally, one can provide a cause for the slot release.

Specified by:

releaseSlot in interface AllocatedSlotActions

Parameters:

slotRequestId - identifying the slot to release

cause - of the slot release, null if none

notifyNotEnoughResourcesAvailable

public void notifyNotEnoughResourcesAvailable(Collection<ResourceRequirement> acquiredResources)

Description copied from interface: SlotPoolService

Notifies that not enough resources are available to fulfill the resource requirements.

Specified by:

notifyNotEnoughResourcesAvailable in interface SlotPoolService

Parameters:

acquiredResources - the resources that have been acquired

getAllocatedSlotsInformation

public Collection<SlotInfo> getAllocatedSlotsInformation()

Description copied from interface: SlotPool

Returns a list of SlotInfo objects about all slots that are currently allocated in the slot pool.

Specified by:

getAllocatedSlotsInformation in interface SlotPool

Returns:

a list of SlotInfo objects about all slots that are currently allocated in the slot pool.

getFreeSlotTracker

public FreeSlotTracker getFreeSlotTracker()

Description copied from interface: SlotPool

Returns all free slot tracker.

Specified by:

getFreeSlotTracker in interface SlotPool

Returns:

all free slot tracker

disableBatchSlotRequestTimeoutCheck

public void disableBatchSlotRequestTimeoutCheck()

Description copied from interface: SlotPool

Disables batch slot request timeout check. Invoked when someone else wants to take over the timeout check responsibility.

Specified by:

disableBatchSlotRequestTimeoutCheck in interface SlotPool

getNumPendingRequests

@VisibleForTesting
public int getNumPendingRequests()