c

com.linkedin.feathr.swj.aggregate

AggregationWithDeaggBase

abstract class AggregationWithDeaggBase extends AggregationSpec

Base class for aggregation functions that support the deagg operation (e.g. Sum/Avg). This class provides common functionality to keep track of a running count aggregation while performing the implemented aggregation type. Keeping track of the running count is important in order to identify when all records have been deagged out so the aggregate value can be properly reset. This is important to maintain current expectation where aggregate value is null when there are no records to aggregate together and to prevent floating point rounding errors causing the deagged aggregate value to be infinitesimally small instead of equal to zero. This class abstracts way the management of the count aggregation column so child classes only have to worry about implementing the aggregation function for the value column.

Linear Supertypes
AggregationSpec, Serializable, Serializable, AnyRef, Any
Known Subclasses
AvgPoolingAggregate, SumPoolingAggregate
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AggregationWithDeaggBase
  2. AggregationSpec
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new AggregationWithDeaggBase()

Abstract Value Members

  1. abstract def aggValue(aggregate: Any, valueRecord: Any, valueDataType: DataType): Any

    Analogous to AggregationSpec.agg(), we aggregate the parsed value record to the intermediate aggregate value

    Analogous to AggregationSpec.agg(), we aggregate the parsed value record to the intermediate aggregate value

    aggregate

    The intermediate aggregate value

    valueRecord

    Parsed value record to aggregate

    valueDataType

    The value record datatype

    returns

    The result of aggregating the value record to the intermediate aggregate value

  2. abstract def aggregation: AggregationType

    <invalid inheritdoc annotation>

    <invalid inheritdoc annotation>

    Definition Classes
    AggregationWithDeaggBase → AggregationSpec
  3. abstract def countCol: String

    Name of the column that keeps track of the count which is required for all aggregation functions that support deagg in order to identify when all records have been deagged out

  4. abstract def deaggValue(aggregate: Any, valueRecord: Any, valueDataType: DataType): Any

    Analogous to AggregationSpec.deagg(), we deagg the parsed value record from the intermediate aggregate value

    Analogous to AggregationSpec.deagg(), we deagg the parsed value record from the intermediate aggregate value

    aggregate

    The intermediate aggregate value

    valueRecord

    Parsed value record to deagg

    valueDataType

    The value record datatype

    returns

    The result of deagging the value record from the intermediate aggregate value

  5. abstract def metricName: String

    <invalid inheritdoc annotation>

    <invalid inheritdoc annotation>

    Definition Classes
    AggregationWithDeaggBase → AggregationSpec
  6. abstract def valueCol: String

    Name of the metric column or a Spark SQL column expression for derived metric that will be aggregated

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. val CountAggregate: CountAggregate
  5. def agg(aggregate: Any, record: Any, dataType: DataType): Any
    Definition Classes
    AggregationWithDeaggBase → AggregationSpec
  6. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  7. def calculateAggregate(aggregate: Any, dataType: DataType): Any
    Definition Classes
    AggregationWithDeaggBase → AggregationSpec
  8. def calculateAggregateWithCount(aggregate: Any, valueDataType: DataType, count: Any, countDataType: DataType): Any

    Analogous to AggregationSpec.calculateAggregate(), perform an additional step to calculate the final aggregate value.

    Analogous to AggregationSpec.calculateAggregate(), perform an additional step to calculate the final aggregate value. This method has access to the current running count if needed for the calculation (e.g. AVG). Default implementation is to return the aggregate value as is.

    aggregate

    The intermediate aggregate value

    valueDataType

    The value record datatype

    count

    The intermediate count value

    countDataType

    The count record datatype

    returns

    The calculated aggregate value

  9. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  10. def deagg(aggregate: Any, record: Any, dataType: DataType): Any
    Definition Classes
    AggregationWithDeaggBase → AggregationSpec
  11. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  13. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  16. def isCalculateAggregateNeeded: Boolean
    Definition Classes
    AggregationSpec
  17. def isIncrementalAgg: Boolean
    Definition Classes
    AggregationWithDeaggBase → AggregationSpec
  18. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  19. def metricCol: String
    Definition Classes
    AggregationWithDeaggBase → AggregationSpec
  20. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  21. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  23. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  24. def toString(): String
    Definition Classes
    AnyRef → Any
  25. def updateAggDataType(featureColumnMetaData: FeatureColumnMetaData): Unit
    Definition Classes
    AggregationWithDeaggBase → AggregationSpec
  26. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AggregationSpec

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped