software.amazon.kinesis.worker.metricstats

Class WorkerMetricStats

java.lang.Object

software.amazon.kinesis.worker.metricstats.WorkerMetricStats

All Implemented Interfaces:

EntityDAO.Entity

Direct Known Subclasses:

WorkerMetricStats.LeaseTableWorkerMetricStats, WorkerMetricStats.LegacyWorkerMetricStats

public class WorkerMetricStats
extends Object
implements EntityDAO.Entity

DataModel for a WorkerMetric, this data model is used to store the current state of a Worker in terms of relevant WorkerMetric(CPU, Memory, Network). workerId : unique worker identifier, this is equivalent to the owner attribute from the lease table. entityType : discriminator written to DDB so shared-table scans can filter by entity type. lastUpdateTime : wall epoch in seconds when the entry was last updated metricStats : Map of WorkerMetric to last N values for it. e.g. entry "CPU" : [10,20,12,10] etc operatingRange : Map of WorkerMetric to its operating range. First item in the list of values defines the max limit. properties : Arbitrary key-value properties for extensibility. supportCode : Tracks the highest "breaking" feature ordinal supported by this worker. Used by the leader to determine fleet-wide feature support for backward-compatibility and rollback safety. supportCodeUpdateEpochSeconds : wall epoch in seconds when the support code was last heartbeated. Since older versions performing partial DDB updates would leave stale supportCode values after rollback, this timestamp lets the leader distinguish current from stale. metricStatsMap : runtime computed WorkerMetric name to its average value map. This field is not stored in ddb and is used during Lease assignment only Inner subclasses WorkerMetricStats.LegacyWorkerMetricStats (PK: wid) and WorkerMetricStats.LeaseTableWorkerMetricStats (PK: leaseKey) override the partition key annotation on getWorkerId() for the legacy worker-metrics table vs. the unified lease table respectively. The DAO delegates select the appropriate subclass for the DynamoDB Enhanced Client.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	WorkerMetricStats.Features Add any new "breaking" feature to the END of this list.
static class	WorkerMetricStats.LeaseTableWorkerMetricStats Schema for lease table uses "leaseKey" as the partition key.
static class	WorkerMetricStats.LegacyWorkerMetricStats Schema for worker stats table uses "wid" as the partition key.
static class	WorkerMetricStats.WorkerSupportInfo Immutable snapshot of a worker's support code information.

Field Summary

Fields

Modifier and Type	Field and Description
static String	ENTITY_TYPE_ATTRIBUTE_NAME
static String	KEY_LAST_UPDATE_TIME
static String	KEY_WORKER_ID

Constructor Summary

Constructors

Constructor and Description
WorkerMetricStats()

Method Summary

Modifier and Type	Method and Description
double	computePercentageToReachAverage(Map<String,Double> workerMetricToFleetLevelAverage) Determines percentage of load to reach the mean for the worker.
boolean	containsMetricStat(String workerMetricName) Returns true if given is available for the current worker else false
void	extrapolateMetricStatValuesForAddedLease(Map<String,Double> workerMetricToFleetLevelAverage, int averageLeaseCount) Increase the metric stat value corresponding to the added single lease.
void	extrapolateMetricStatValuesForAddedThroughput(Map<String,Double> workerMetricsToFleetLevelAverageMap, double averageThroughput, double increaseThroughput, double averageLeaseCount) Increase the WorkerMetricStats value by given increaseLoadPercentage.
String	getEntityTypeDdbValue() DDB Enhanced Client accessor: serializes entityType using EntityType.getDdbValue().
double	getMetricStat(String workerMetricName) Returns the value for given WorkerMetricStats name.
boolean	isAnyWorkerMetricAboveAverageUtilizationOrOperatingRange(Map<String,Double> workerMetricToFleetLevelAverage)
boolean	isAnyWorkerMetricFailing() Returns true if any of the metric stat values has -1 in last index which represents that the metric stat value was not successfully fetched in last attempt by worker.
boolean	isExpired(Duration maxAge) Determines whether this worker metrics entry is expired (i.e., the worker is no longer active).
boolean	isStale(Duration staleCleanupThreshold) Determines if this worker metric entry is stale and eligible for cleanup/deletion.
boolean	isUsingDefaultWorkerMetric() If a worker is not using an explicit WorkerMetric such as CPU, Memory, or Network, then it is said to be using the default WorkerMetric.
boolean	isValidWorkerMetric() WorkerMetricStats entry is invalid if any of the field from lastUpdateTime, operatingRange, resourcesStats are not present or if resourcesStats is empty or if any of the WorkerMetrics having resourceStats does not have operatingRange or if operating range values are not present or if maxUtilization is 0 for any WorkerMetric
boolean	isWorkerMetricAboveOperatingRange(String workerMetricName) Evaluates if the given metric stat is above operatingRange for the given WorkerMetric name.
void	setEntityTypeDdbValue(String ddbValue) DDB Enhanced Client accessor: deserializes entityType from the DDB value string.
boolean	willAnyMetricStatsGoAboveAverageUtilizationOrOperatingRange(Map<String,Double> workerMetricsToFleetLevelAverageMap, double averageThroughput, double increaseThroughput, double averageLeaseCount)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface software.amazon.kinesis.leases.EntityDAO.Entity

getEntityType

Field Detail

KEY_LAST_UPDATE_TIME

public static final String KEY_LAST_UPDATE_TIME

See Also:

Constant Field Values

KEY_WORKER_ID

public static final String KEY_WORKER_ID

See Also:

Constant Field Values

ENTITY_TYPE_ATTRIBUTE_NAME

public static final String ENTITY_TYPE_ATTRIBUTE_NAME

See Also:

Constant Field Values

Constructor Detail

WorkerMetricStats

public WorkerMetricStats()

Method Detail

getEntityTypeDdbValue

public String getEntityTypeDdbValue()

DDB Enhanced Client accessor: serializes entityType using EntityType.getDdbValue().

setEntityTypeDdbValue

public void setEntityTypeDdbValue(String ddbValue)

DDB Enhanced Client accessor: deserializes entityType from the DDB value string.

containsMetricStat

public boolean containsMetricStat(String workerMetricName)

Returns true if given is available for the current worker else false

getMetricStat

public double getMetricStat(String workerMetricName)

Returns the value for given WorkerMetricStats name.

extrapolateMetricStatValuesForAddedThroughput

public void extrapolateMetricStatValuesForAddedThroughput(Map<String,Double> workerMetricsToFleetLevelAverageMap,
                                                          double averageThroughput,
                                                          double increaseThroughput,
                                                          double averageLeaseCount)

Increase the WorkerMetricStats value by given increaseLoadPercentage. This is done during execution of LAM and as assignments are happening the current metric stat value is increased based on increaseLoadPercentage.

willAnyMetricStatsGoAboveAverageUtilizationOrOperatingRange

public boolean willAnyMetricStatsGoAboveAverageUtilizationOrOperatingRange(Map<String,Double> workerMetricsToFleetLevelAverageMap,
                                                                           double averageThroughput,
                                                                           double increaseThroughput,
                                                                           double averageLeaseCount)

extrapolateMetricStatValuesForAddedLease

public void extrapolateMetricStatValuesForAddedLease(Map<String,Double> workerMetricToFleetLevelAverage,
                                                     int averageLeaseCount)

Increase the metric stat value corresponding to the added single lease. This is done during execution of LAM and as assignments are happening the load is increase for LAM to determine workers for assignment. The increase is done considering that for a WorkerMetric the fleet level average would be met when fleet level average leases are assigned to a worker and thus 1 lease addition increases the metric stat value by fleet level average of metric stat by averageLeaseCount

computePercentageToReachAverage

public double computePercentageToReachAverage(Map<String,Double> workerMetricToFleetLevelAverage)

Determines percentage of load to reach the mean for the worker. In case of multiple worker metrics the metric stat value closest to mean is used to determine the percentage value. This value is indication of how much load in percentage to current load the worker can take to reach mean value.

Parameters:

workerMetricToFleetLevelAverage - : WorkerMetric to fleet level mean value.

Returns:

percentage to reach mean based on the WorkerMetric closest to its corresponding average.

isAnyWorkerMetricFailing

public boolean isAnyWorkerMetricFailing()

Returns true if any of the metric stat values has -1 in last index which represents that the metric stat value was not successfully fetched in last attempt by worker.

Returns:

true if any metric stat value has -1 in last index, false otherwise.

isValidWorkerMetric

public boolean isValidWorkerMetric()

WorkerMetricStats entry is invalid if any of the field from lastUpdateTime, operatingRange, resourcesStats are not present or if resourcesStats is empty or if any of the WorkerMetrics having resourceStats does not have operatingRange or if operating range values are not present or if maxUtilization is 0 for any WorkerMetric

Returns:

true if the entry is valid false otherwise.

isAnyWorkerMetricAboveAverageUtilizationOrOperatingRange

public boolean isAnyWorkerMetricAboveAverageUtilizationOrOperatingRange(Map<String,Double> workerMetricToFleetLevelAverage)

isUsingDefaultWorkerMetric

public boolean isUsingDefaultWorkerMetric()

If a worker is not using an explicit WorkerMetric such as CPU, Memory, or Network, then it is said to be using the default WorkerMetric. Load management then falls back to throughput.

Returns:

true if the worker is not using an explicit WorkerMetric.

isWorkerMetricAboveOperatingRange

public boolean isWorkerMetricAboveOperatingRange(String workerMetricName)

Evaluates if the given metric stat is above operatingRange for the given WorkerMetric name. If the WorkerMetric does not exist returns false

Parameters:

workerMetricName - WorkerMetric name to evaluate

Returns:

true if metric stat exists and is above operatingRange for the WorkerMetric

isExpired

public boolean isExpired(Duration maxAge)

Determines whether this worker metrics entry is expired (i.e., the worker is no longer active). A worker is considered expired if the time elapsed since its lastUpdateTime exceeds the given maximum allowed age.

For example, if the max age is 2 * workerMetricsReporterFreqInMillis, then a worker that hasn't updated in more than that duration is considered expired/dead.

Parameters:

maxAge - the maximum allowed age since lastUpdateTime. If the elapsed time since lastUpdateTime exceeds this duration, the worker is considered expired.

Returns:

true if this worker's lastUpdateTime is older than maxAge from now, false otherwise.

isStale

public boolean isStale(Duration staleCleanupThreshold)

Determines if this worker metric entry is stale and eligible for cleanup/deletion.

A stale entry is one that hasn't been updated for longer than the staleCleanupThreshold. Stale entries are typically much older than expired entries — they represent workers that have been dead for a long time and whose entries should be garbage-collected from the table.

Parameters:

staleCleanupThreshold - the maximum allowed age. If the elapsed time since lastUpdateTime exceeds this duration, the entry is considered stale.

Returns:

true if this entry's lastUpdateTime is older than staleCleanupThreshold from now.