Type Members

1.  class CompositeKeyStateEncoder[K, V] extends StateTypesEncoder[V]
2.  class StateTypesEncoder[V] extends AnyRef

   Helper class providing APIs to encode the grouping key, and user provided values to Spark UnsafeRow.

   Helper class providing APIs to encode the grouping key, and user provided values to Spark UnsafeRow.

   CAUTION: StateTypesEncoder class instance is *not* thread-safe. This class reuses the keyProjection and valueProjection for encoding grouping key and state value respectively. As UnsafeProjection is not thread safe, this class is also not thread safe.

   V

   - value type

3.  class TTLEncoder extends AnyRef

   Class for TTL with single key serialization

4.  class TimerKeyEncoder extends AnyRef

   Class for timer state serialization

5.  case class TransformWithStateExec(keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], statefulProcessor: StatefulProcessor[Any, Any, Any], timeMode: TimeMode, outputMode: OutputMode, keyEncoder: ExpressionEncoder[Any], outputObjAttr: Attribute, stateInfo: Option[StatefulOperatorStateInfo], batchTimestampMs: Option[Long], eventTimeWatermarkForLateEvents: Option[Long], eventTimeWatermarkForEviction: Option[Long], child: SparkPlan, isStreaming: Boolean = true, hasInitialState: Boolean = false, initialStateGroupingAttrs: Seq[Attribute], initialStateDataAttrs: Seq[Attribute], initialStateDeserializer: Expression, initialState: SparkPlan) extends TransformWithStateExecBase with ObjectProducerExec with Product with Serializable

   Physical operator for executing TransformWithState

   Physical operator for executing TransformWithState

   keyDeserializer

   used to extract the key object for each group.

   valueDeserializer

   used to extract the items in the iterator from an input row.

   groupingAttributes

   used to group the data

   dataAttributes

   used to read the data

   statefulProcessor

   processor methods called on underlying data

   timeMode

   The time mode semantics of the stateful processor for timers and TTL.

   outputMode

   defines the output mode for the statefulProcessor

   keyEncoder

   expression encoder for the key type

   outputObjAttr

   Defines the output object

   batchTimestampMs

   processing timestamp of the current batch.

   eventTimeWatermarkForLateEvents

   event time watermark for filtering late events

   eventTimeWatermarkForEviction

   event time watermark for state eviction

   child

   the physical plan for the underlying data

   isStreaming

   defines whether the query is streaming or batch

6.  abstract class TransformWithStateExecBase extends SparkPlan with BinaryExecNode with StateStoreWriter with WatermarkSupport with TransformWithStateMetadataUtils

   This is the base class for physical node that execute TransformWithState.

   This is the base class for physical node that execute TransformWithState.

   It contains some common logics like state store metrics handling, co-locate initial state with the incoming data, and etc. Concrete physical node like TransformWithStateInPySparkExec and TransformWithStateExec should extend this class.

7.  trait TransformWithStateMetadataUtils extends SchemaValidationUtils with Logging

   This trait contains utils functions related to TransformWithState metadata.

   This trait contains utils functions related to TransformWithState metadata. This is used both in Scala and Python side of TransformWithState metadata support when calling init() with DriverStatefulProcessorHandleImpl, and get the state schema and state metadata on driver during physical planning phase.

8.  case class TransformWithStateOperatorProperties(timeMode: String, outputMode: String, stateVariables: List[TransformWithStateVariableInfo]) extends Product with Serializable
9.  case class TransformWithStateVariableInfo(stateName: String, stateVariableType: StateVariableType, ttlEnabled: Boolean) extends Product with Serializable