org.apache.hadoop.mapred.gridmix.emulators.resourceusage

Class TotalHeapUsageEmulatorPlugin

java.lang.Object

org.apache.hadoop.mapred.gridmix.emulators.resourceusage.TotalHeapUsageEmulatorPlugin

All Implemented Interfaces:

ResourceUsageEmulatorPlugin, Progressive

public class TotalHeapUsageEmulatorPlugin
extends Object
implements ResourceUsageEmulatorPlugin

A ResourceUsageEmulatorPlugin that emulates the total heap usage by loading the JVM heap memory. Adding smaller chunks of data to the heap will essentially use up some heap space thus forcing the JVM to expand its heap and thus resulting into increase in the heap usage.

TotalHeapUsageEmulatorPlugin emulates the heap usage in steps. The frequency of emulation can be configured via HEAP_EMULATION_PROGRESS_INTERVAL. Heap usage values are matched via emulation only at specific interval boundaries.

TotalHeapUsageEmulatorPlugin is a wrapper program for managing the heap usage emulation feature. It internally uses an emulation algorithm (called as core and described using TotalHeapUsageEmulatorPlugin.HeapUsageEmulatorCore) for performing the actual emulation. Multiple calls to this core engine should use up some amount of heap.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TotalHeapUsageEmulatorPlugin.DefaultHeapUsageEmulator This is the core engine to emulate the heap usage.
static interface	TotalHeapUsageEmulatorPlugin.HeapUsageEmulatorCore Defines the core heap usage emulation algorithm.

Field Summary

Fields

Modifier and Type	Field and Description
protected TotalHeapUsageEmulatorPlugin.HeapUsageEmulatorCore	emulatorCore
static String	HEAP_EMULATION_PROGRESS_INTERVAL The frequency (based on task progress) with which memory-emulation code is run.
static String	HEAP_LOAD_RATIO Determines the unit increase per call to the core engine's load API.
static String	MIN_HEAP_FREE_RATIO The minimum buffer reserved for other non-emulation activities.
static int	ONE_MB

Constructor Summary

Constructors

Constructor and Description
TotalHeapUsageEmulatorPlugin()
TotalHeapUsageEmulatorPlugin(TotalHeapUsageEmulatorPlugin.HeapUsageEmulatorCore core) For testing.

Method Summary

Modifier and Type	Method and Description
void	emulate() Emulate the resource usage to match the usage target.
protected long	getMaxHeapUsageInMB()
float	getProgress()
protected long	getTotalHeapUsageInMB()
void	initialize(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.tools.rumen.ResourceUsageMetrics metrics, org.apache.hadoop.yarn.util.ResourceCalculatorPlugin monitor, Progressive progress) Initialize the plugin.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

emulatorCore

protected TotalHeapUsageEmulatorPlugin.HeapUsageEmulatorCore emulatorCore

HEAP_EMULATION_PROGRESS_INTERVAL

public static final String HEAP_EMULATION_PROGRESS_INTERVAL

The frequency (based on task progress) with which memory-emulation code is run. If the value is set to 0.1 then the emulation will happen at 10% of the task's progress. The default value of this parameter is DEFAULT_EMULATION_PROGRESS_INTERVAL.

See Also:

Constant Field Values

MIN_HEAP_FREE_RATIO

public static final String MIN_HEAP_FREE_RATIO

The minimum buffer reserved for other non-emulation activities.

See Also:

Constant Field Values

HEAP_LOAD_RATIO

public static final String HEAP_LOAD_RATIO

Determines the unit increase per call to the core engine's load API. This is expressed as a percentage of the difference between the expected total heap usage and the current usage.

See Also:

Constant Field Values

ONE_MB

public static final int ONE_MB

See Also:

Constant Field Values

Constructor Detail

TotalHeapUsageEmulatorPlugin

public TotalHeapUsageEmulatorPlugin()

TotalHeapUsageEmulatorPlugin

public TotalHeapUsageEmulatorPlugin(TotalHeapUsageEmulatorPlugin.HeapUsageEmulatorCore core)

For testing.

Method Detail

getTotalHeapUsageInMB

protected long getTotalHeapUsageInMB()

getMaxHeapUsageInMB

protected long getMaxHeapUsageInMB()

getProgress

public float getProgress()

Specified by:

getProgress in interface Progressive

emulate

public void emulate()
             throws IOException,
                    InterruptedException

Description copied from interface: ResourceUsageEmulatorPlugin

Emulate the resource usage to match the usage target. The plugin can use the given ResourceCalculatorPlugin to query for the current resource usage.

Specified by:

emulate in interface ResourceUsageEmulatorPlugin

Throws:

IOException

InterruptedException

initialize

public void initialize(org.apache.hadoop.conf.Configuration conf,
                       org.apache.hadoop.tools.rumen.ResourceUsageMetrics metrics,
                       org.apache.hadoop.yarn.util.ResourceCalculatorPlugin monitor,
                       Progressive progress)

Description copied from interface: ResourceUsageEmulatorPlugin

Initialize the plugin. This might involve - initializing the variables - calibrating the plugin

Specified by:

initialize in interface ResourceUsageEmulatorPlugin