Class

org.apache.spark.whylogs

DatasetProfileAggregator

Related Doc: package whylogs

Permalink

case class DatasetProfileAggregator(datasetName: String, sessionTimeInMillis: Long, timeColumn: String = null, groupByColumns: Seq[String] = Seq()) extends sql.expressions.Aggregator[Row, DatasetProfile, ScalaDatasetProfile] with Serializable with Product

A dataset aggregator. It aggregates Row into DatasetProfile objects underneath the hood.

datasetName

the name of the dataset

sessionTimeInMillis

the session time for the profile

Linear Supertypes
Product, Equals, sql.expressions.Aggregator[Row, DatasetProfile, ScalaDatasetProfile], Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DatasetProfileAggregator
  2. Product
  3. Equals
  4. Aggregator
  5. Serializable
  6. Serializable
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DatasetProfileAggregator(datasetName: String, sessionTimeInMillis: Long, timeColumn: String = null, groupByColumns: Seq[String] = Seq())

    Permalink

    datasetName

    the name of the dataset

    sessionTimeInMillis

    the session time for the profile

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def bufferEncoder: Encoder[DatasetProfile]

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator
  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  7. val datasetName: String

    Permalink

    the name of the dataset

  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. def finish(reduction: DatasetProfile): ScalaDatasetProfile

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator
  11. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  12. val groupByColumns: Seq[String]

    Permalink
  13. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  14. def merge(profile1: DatasetProfile, profile2: DatasetProfile): DatasetProfile

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator
  15. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  16. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  17. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  18. def outputEncoder: Encoder[ScalaDatasetProfile]

    Permalink

    To understand the detailed implementation of this class, see ExpressionEncoder.

    To understand the detailed implementation of this class, see ExpressionEncoder.

    We use some internal Spark API here.

    Definition Classes
    DatasetProfileAggregator → Aggregator
  19. def reduce(profile: DatasetProfile, row: Row): DatasetProfile

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator
  20. val sessionTimeInMillis: Long

    Permalink

    the session time for the profile

  21. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  22. val timeColumn: String

    Permalink
  23. def toColumn: TypedColumn[Row, ScalaDatasetProfile]

    Permalink
    Definition Classes
    Aggregator
  24. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. def zero: DatasetProfile

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator

Inherited from Product

Inherited from Equals

Inherited from sql.expressions.Aggregator[Row, DatasetProfile, ScalaDatasetProfile]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped