Type Members

1.  case class FactData(dataSource: DataFrame, joinKey: Seq[String], timestampCol: String, aggFeatures: List[SlidingWindowFeature]) extends Product with Serializable

   Defines the fact dataset.

   Defines the fact dataset.

   dataSource

   fact dataset as a DataFrame

   joinKey

   column name for the join key. It can be something like 'x.id' representing nested column

   timestampCol

   Timestamp column name. The data is assumed to be epoc of type long

   aggFeatures

   A list of SlidingWindowFeature representing all sliding window aggregated features defined on one fact dataset.

2.  case class GroupBySpec(groupByCol: String, limit: Int) extends Serializable with Product

   Defines the group by specification for a sliding window aggregated feature

   Defines the group by specification for a sliding window aggregated feature

   groupByCol

   Name of the group by column. It can be something like 'x.id' representing nested column

   limit

   Applies a top-K limit on the group by results

3.  case class LabelData(dataSource: DataFrame, joinKey: Seq[String], timestampCol: String) extends Product with Serializable

   Defines the label dataset.

   Defines the label dataset.

   dataSource

   label dataset as a DataFrame

   joinKey

   column name for the join key. It can be something like 'x.id' representing nested column

   timestampCol

   Timestamp column name. The data is assumed to be epoc of type long

4.  case class LateralViewParams(lateralViewDef: String, lateralViewItemAlias: String, lateralViewFilter: Option[String]) extends Serializable with Product

   Defines lateral view specification for a sliding window aggregated feature

5.  case class SlidingWindowFeature(name: String, agg: AggregationSpec, windowSpec: WindowSpec, filterCondition: Option[String] = None, groupBy: Option[GroupBySpec] = None, lateralView: Option[LateralViewParams] = None) extends Serializable with Product

   Defines a sliding window aggregated feature

   Defines a sliding window aggregated feature

   name

   Name of the feature. Used to uniquely identify a column in the DataFrame

   agg

   An AggregationSpec for the sliding window aggregated feature

   windowSpec

   A WindowSpec for the sliding window aggregated feature

   filterCondition

   Filter condition to determine if a given record should be considered in the sliding window aggregation. The String value should represent a Spark SQL column expression

   groupBy

   An optional GroupBySpec for the sliding window aggregated feature

6.  case class WindowSpec(width: Duration, delay: Duration = Duration.ZERO) extends Serializable with Product

   Defines the window specification for a sliding window aggregated feature

   Defines the window specification for a sliding window aggregated feature

   width

   Window width

   delay

   Simulated delay. Default to zero.