org.apache.hadoop.yarn.server.timelineservice.storage.common

Class HBaseTimelineStorageUtils

java.lang.Object

org.apache.hadoop.yarn.server.timelineservice.storage.common.HBaseTimelineStorageUtils

public final class HBaseTimelineStorageUtils
extends Object

A bunch of utility functions used in HBase TimelineService backend.

Field Summary

Fields

Modifier and Type	Field and Description
static long	MILLIS_ONE_DAY milliseconds in one day.

Method Summary

Methods

Modifier and Type	Method and Description
static byte[]	calculateTheClosestNextRowKeyForPrefix(byte[] rowKeyPrefix) Given a row key prefix stored in a byte array, return a byte array for its immediate next row key.
static Attribute[]	combineAttributes(Attribute[] attributes, AggregationOperation aggOp) Combines the input array of attributes and the input aggregation operation into a new array of attributes.
static String	convertApplicationIdToString(org.apache.hadoop.yarn.api.records.ApplicationId appId) A utility method that converts ApplicationId to string without using FastNumberFormat in order to avoid the incompatibility issue caused by mixing hadoop-common 2.5.1 and hadoop-yarn-api 3.0 in this module.
static org.apache.hadoop.hbase.Cell	createNewCell(byte[] row, byte[] family, byte[] qualifier, long ts, byte[] newValue, byte[] tags) creates a cell with the given inputs.
static org.apache.hadoop.hbase.Cell	createNewCell(org.apache.hadoop.hbase.Cell origCell, byte[] newValue) creates a new cell based on the input cell but with the new value.
static String	getAggregationCompactionDimension(List<org.apache.hadoop.hbase.Tag> tags) returns app id from the list of tags.
static AggregationOperation	getAggregationOperationFromTagsList(List<org.apache.hadoop.hbase.Tag> tags) Returns the first seen aggregation operation as seen in the list of input tags or null otherwise.
static org.apache.hadoop.hbase.Tag	getTagFromAttribute(Map.Entry<String,byte[]> attribute) Creates a Tag from the input attribute.
static org.apache.hadoop.conf.Configuration	getTimelineServiceHBaseConf(org.apache.hadoop.conf.Configuration conf)
static long	getTopOfTheDayTimestamp(long ts) returns the timestamp of that day's start (which is midnight 00:00:00 AM) for a given input timestamp.
static int	invertInt(int key) Converts an int into it's inverse int to be used in (row) keys where we want to have the largest int value in the top of the table (scans start at the largest int first).
static boolean	isIntegralValue(Object obj) Checks if passed object is of integral type(Short/Integer/Long).
static void	setMetricsTimeRange(org.apache.hadoop.hbase.client.Query query, byte[] metricsCf, long tsBegin, long tsEnd)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MILLIS_ONE_DAY

public static final long MILLIS_ONE_DAY

milliseconds in one day.

See Also:

Constant Field Values

Method Detail

combineAttributes

public static Attribute[] combineAttributes(Attribute[] attributes,
                            AggregationOperation aggOp)

Combines the input array of attributes and the input aggregation operation into a new array of attributes.

Parameters:

attributes - Attributes to be combined.

aggOp - Aggregation operation.

Returns:

array of combined attributes.

getAggregationOperationFromTagsList

public static AggregationOperation getAggregationOperationFromTagsList(List<org.apache.hadoop.hbase.Tag> tags)

Returns the first seen aggregation operation as seen in the list of input tags or null otherwise.

Parameters:

tags - list of HBase tags.

Returns:

AggregationOperation

getTagFromAttribute

public static org.apache.hadoop.hbase.Tag getTagFromAttribute(Map.Entry<String,byte[]> attribute)

Creates a Tag from the input attribute.

Parameters:

attribute - Attribute from which tag has to be fetched.

Returns:

a HBase Tag.

createNewCell

public static org.apache.hadoop.hbase.Cell createNewCell(org.apache.hadoop.hbase.Cell origCell,
                                         byte[] newValue)
                                                  throws IOException

creates a new cell based on the input cell but with the new value.

Parameters:

origCell - Original cell

newValue - new cell value

Returns:

cell

Throws:

IOException - while creating new cell.

createNewCell

public static org.apache.hadoop.hbase.Cell createNewCell(byte[] row,
                                         byte[] family,
                                         byte[] qualifier,
                                         long ts,
                                         byte[] newValue,
                                         byte[] tags)
                                                  throws IOException

creates a cell with the given inputs.

Parameters:

row - row of the cell to be created

family - column family name of the new cell

qualifier - qualifier for the new cell

ts - timestamp of the new cell

newValue - value of the new cell

tags - tags in the new cell

Returns:

cell

Throws:

IOException - while creating the cell.

getAggregationCompactionDimension

public static String getAggregationCompactionDimension(List<org.apache.hadoop.hbase.Tag> tags)

returns app id from the list of tags.

Parameters:

tags - cell tags to be looked into

Returns:

App Id as the AggregationCompactionDimension

invertInt

public static int invertInt(int key)

Converts an int into it's inverse int to be used in (row) keys where we want to have the largest int value in the top of the table (scans start at the largest int first).

Parameters:

key - value to be inverted so that the latest version will be first in a scan.

Returns:

inverted int

getTopOfTheDayTimestamp

public static long getTopOfTheDayTimestamp(long ts)

returns the timestamp of that day's start (which is midnight 00:00:00 AM) for a given input timestamp.

Parameters:

ts - Timestamp.

Returns:

timestamp of that day's beginning (midnight)

convertApplicationIdToString

public static String convertApplicationIdToString(org.apache.hadoop.yarn.api.records.ApplicationId appId)

A utility method that converts ApplicationId to string without using FastNumberFormat in order to avoid the incompatibility issue caused by mixing hadoop-common 2.5.1 and hadoop-yarn-api 3.0 in this module. This is a work-around implementation as discussed in YARN-6905.

Parameters:

appId - application id

Returns:

the string representation of the given application id

getTimelineServiceHBaseConf

public static org.apache.hadoop.conf.Configuration getTimelineServiceHBaseConf(org.apache.hadoop.conf.Configuration conf)
                                                                        throws MalformedURLException

Parameters:

conf - Yarn configuration. Used to see if there is an explicit config pointing to the HBase config file to read. It should not be null or a NullPointerException will be thrown.

Returns:

a configuration with the HBase configuration from the classpath, optionally overwritten by the timeline service configuration URL if specified.

Throws:

MalformedURLException - if a timeline service HBase configuration URL is specified but is a malformed URL.

calculateTheClosestNextRowKeyForPrefix

public static byte[] calculateTheClosestNextRowKeyForPrefix(byte[] rowKeyPrefix)

Given a row key prefix stored in a byte array, return a byte array for its immediate next row key.

Parameters:

rowKeyPrefix - The provided row key prefix, represented in an array.

Returns:

the closest next row key of the provided row key.

isIntegralValue

public static boolean isIntegralValue(Object obj)

Checks if passed object is of integral type(Short/Integer/Long).

Parameters:

obj - Object to be checked.

Returns:

true if object passed is of type Short or Integer or Long, false otherwise.

setMetricsTimeRange

public static void setMetricsTimeRange(org.apache.hadoop.hbase.client.Query query,
                       byte[] metricsCf,
                       long tsBegin,
                       long tsEnd)