Type Members

1.  case class FlatMapGroupsWithStateExec(func: (Any, Iterator[Any], LogicalGroupState[Any]) => Iterator[Any], keyDeserializer: Expression, valueDeserializer: Expression, initialStateDeserializer: Expression, groupingAttributes: Seq[Attribute], initialStateGroupAttrs: Seq[Attribute], dataAttributes: Seq[Attribute], initialStateDataAttrs: Seq[Attribute], outputObjAttr: Attribute, stateInfo: Option[StatefulOperatorStateInfo], stateEncoder: ExpressionEncoder[Any], stateFormatVersion: Int, outputMode: OutputMode, timeoutConf: GroupStateTimeout, batchTimestampMs: Option[Long], eventTimeWatermarkForLateEvents: Option[Long], eventTimeWatermarkForEviction: Option[Long], initialState: SparkPlan, hasInitialState: Boolean, skipEmittingInitialStateKeys: Boolean, child: SparkPlan) extends SparkPlan with FlatMapGroupsWithStateExecBase with BinaryExecNode with ObjectProducerExec with Product with Serializable

   Physical operator for executing FlatMapGroupsWithState

   Physical operator for executing FlatMapGroupsWithState

   func

   function called on each group

   keyDeserializer

   used to extract the key object for each group.

   valueDeserializer

   used to extract the items in the iterator from an input row.

   initialStateDeserializer

   used to extract the state object from the initialState dataset

   groupingAttributes

   used to group the data

   dataAttributes

   used to read the data

   outputObjAttr

   Defines the output object

   stateEncoder

   used to serialize/deserialize state before calling func

   outputMode

   the output mode of func

   timeoutConf

   used to timeout groups that have not received data in a while

   batchTimestampMs

   processing timestamp of the current batch.

   eventTimeWatermarkForLateEvents

   event time watermark for filtering late events

   eventTimeWatermarkForEviction

   event time watermark for state eviction

   initialState

   the user specified initial state

   hasInitialState

   indicates whether the initial state is provided or not

   skipEmittingInitialStateKeys

   whether to skip emitting initial state df keys

   child

   the physical plan for the underlying data

2.  trait FlatMapGroupsWithStateExecBase extends SparkPlan with StateStoreWriter with WatermarkSupport

   Physical operator for executing FlatMapGroupsWithState