org.apache.flink.runtime.io.network.partition.hybrid

Class HsFullSpillingStrategy

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.HsFullSpillingStrategy

All Implemented Interfaces:

HsSpillingStrategy

public class HsFullSpillingStrategy
extends Object
implements HsSpillingStrategy

A special implementation of HsSpillingStrategy that spilled all buffers to disk.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.network.partition.hybrid.HsSpillingStrategy

HsSpillingStrategy.Decision

Constructor Summary

Constructors

Constructor and Description
HsFullSpillingStrategy(HybridShuffleConfiguration hybridShuffleConfiguration)

Method Summary

Modifier and Type	Method and Description
HsSpillingStrategy.Decision	decideActionWithGlobalInfo(HsSpillingInfoProvider spillingInfoProvider) Make a decision based on global information.
Optional<HsSpillingStrategy.Decision>	onBufferConsumed(BufferIndexAndChannel consumedBuffer) Make a decision when a buffer is consumed.
Optional<HsSpillingStrategy.Decision>	onBufferFinished(int numTotalUnSpillBuffers, int currentPoolSize) Make a decision when a buffer becomes finished.
Optional<HsSpillingStrategy.Decision>	onMemoryUsageChanged(int numTotalRequestedBuffers, int currentPoolSize) Make a decision when memory usage is changed.
HsSpillingStrategy.Decision	onResultPartitionClosed(HsSpillingInfoProvider spillingInfoProvider) Make a decision when result partition is closed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HsFullSpillingStrategy

public HsFullSpillingStrategy(HybridShuffleConfiguration hybridShuffleConfiguration)

Method Detail

onBufferFinished

public Optional<HsSpillingStrategy.Decision> onBufferFinished(int numTotalUnSpillBuffers,
                                                              int currentPoolSize)

Description copied from interface: HsSpillingStrategy

Make a decision when a buffer becomes finished.

Specified by:

onBufferFinished in interface HsSpillingStrategy

Parameters:

numTotalUnSpillBuffers - total number of buffers not spill.

Returns:

A HsSpillingStrategy.Decision based on the provided information, or Optional.empty() if the decision cannot be made, which indicates global information is needed.

onBufferConsumed

public Optional<HsSpillingStrategy.Decision> onBufferConsumed(BufferIndexAndChannel consumedBuffer)

Description copied from interface: HsSpillingStrategy

Make a decision when a buffer is consumed.

Specified by:

onBufferConsumed in interface HsSpillingStrategy

Parameters:

consumedBuffer - the buffer that is consumed.

Returns:

A HsSpillingStrategy.Decision based on the provided information, or Optional.empty() if the decision cannot be made, which indicates global information is needed.

onMemoryUsageChanged

public Optional<HsSpillingStrategy.Decision> onMemoryUsageChanged(int numTotalRequestedBuffers,
                                                                  int currentPoolSize)

Description copied from interface: HsSpillingStrategy

Make a decision when memory usage is changed.

Specified by:

onMemoryUsageChanged in interface HsSpillingStrategy

Parameters:

numTotalRequestedBuffers - total number of buffers requested.

currentPoolSize - current value of buffer pool size.

Returns:

A HsSpillingStrategy.Decision based on the provided information, or Optional.empty() if the decision cannot be made, which indicates global information is needed.

decideActionWithGlobalInfo

public HsSpillingStrategy.Decision decideActionWithGlobalInfo(HsSpillingInfoProvider spillingInfoProvider)

Description copied from interface: HsSpillingStrategy

Make a decision based on global information. Because this method will directly touch the HsSpillingInfoProvider, the caller should take care of the thread safety.

Specified by:

decideActionWithGlobalInfo in interface HsSpillingStrategy

Parameters:

spillingInfoProvider - that provides information about the current status.

Returns:

A HsSpillingStrategy.Decision based on the global information.

onResultPartitionClosed

public HsSpillingStrategy.Decision onResultPartitionClosed(HsSpillingInfoProvider spillingInfoProvider)

Description copied from interface: HsSpillingStrategy

Make a decision when result partition is closed. Because this method will directly touch the HsSpillingInfoProvider, the caller should take care of the thread safety.

Specified by:

onResultPartitionClosed in interface HsSpillingStrategy

Parameters:

spillingInfoProvider - that provides information about the current status.

Returns:

A HsSpillingStrategy.Decision based on the global information.