Packages

c

org.apache.spark.sql.execution.streaming

StatefulProcessorHandleImpl

class StatefulProcessorHandleImpl extends StatefulProcessorHandleImplBase with Logging

Class that provides a concrete implementation of a StatefulProcessorHandle. Note that we keep track of valid transitions as various functions are invoked to track object lifecycle.

Linear Supertypes
Logging, StatefulProcessorHandleImplBase, StatefulProcessorHandle, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. StatefulProcessorHandleImpl
  2. Logging
  3. StatefulProcessorHandleImplBase
  4. StatefulProcessorHandle
  5. Serializable
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new StatefulProcessorHandleImpl(store: StateStore, runId: UUID, keyEncoder: ExpressionEncoder[Any], timeMode: TimeMode, isStreaming: Boolean = true, batchTimestampMs: Option[Long] = None, metrics: Map[String, SQLMetric] = Map.empty)

    store

    - instance of state store

    runId

    - unique id for the current run

    keyEncoder

    - encoder for the key

    isStreaming

    - defines whether the query is streaming or batch

    batchTimestampMs

    - timestamp for the current batch if available

    metrics

    - metrics to be updated as part of stateful processing

Type Members

  1. implicit class LogStringContext extends AnyRef
    Definition Classes
    Logging

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  6. var currState: StatefulProcessorHandleState
    Attributes
    protected
    Definition Classes
    StatefulProcessorHandleImplBase
  7. def deleteIfExists(stateName: String): Unit

    Function to delete and purge state variable if defined previously

    Function to delete and purge state variable if defined previously

    stateName

    - name of the state variable

    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  8. def deleteTimer(expiryTimestampMs: Long): Unit

    Function to delete a timer for the given expiryTimestampMs

    Function to delete a timer for the given expiryTimestampMs

    expiryTimestampMs

    - timestamp in milliseconds for the timer to delete

    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  9. def doTtlCleanup(): Unit

    Performs the user state cleanup based on assigned TTl values.

    Performs the user state cleanup based on assigned TTl values. Any state which is expired will be cleaned up from StateStore.

  10. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  11. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  12. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  13. def getExpiredTimers(expiryTimestampMs: Long): Iterator[(Any, Long)]

    Function to retrieve all expired registered timers for all grouping keys

    Function to retrieve all expired registered timers for all grouping keys

    expiryTimestampMs

    Threshold for expired timestamp in milliseconds, this function will return all timers that have timestamp less than passed threshold

    returns

    - iterator of registered timers for all grouping keys

  14. def getHandleState: StatefulProcessorHandleState
    Definition Classes
    StatefulProcessorHandleImplBase
  15. def getListState[T](stateName: String, ttlConfig: TTLConfig)(implicit arg0: Encoder[T]): ListState[T]
    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  16. def getListState[T](stateName: String, valEncoder: Encoder[T], ttlConfig: TTLConfig): ListState[T]

    Function to create new or return existing list state variable of given type with ttl.

    Function to create new or return existing list state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any values in listState which have expired after ttlDuration will not returned on get() and will be eventually removed from the state.

    The user must ensure to call this function only within the init() method of the StatefulProcessor.

    T

    - type of state variable

    stateName

    - name of the state variable

    valEncoder

    - SQL encoder for state variable

    ttlConfig

    - the ttl configuration (time to live duration etc.)

    returns

    - instance of ListState of type T that can be used to store state persistently

    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  17. def getMapState[K, V](stateName: String, ttlConfig: TTLConfig)(implicit arg0: Encoder[K], arg1: Encoder[V]): MapState[K, V]
    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  18. def getMapState[K, V](stateName: String, userKeyEnc: Encoder[K], valEncoder: Encoder[V], ttlConfig: TTLConfig): MapState[K, V]
    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  19. def getQueryInfo(): QueryInfo
    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  20. def getValueState[T](stateName: String, ttlConfig: TTLConfig)(implicit arg0: Encoder[T]): ValueState[T]
    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  21. def getValueState[T](stateName: String, valEncoder: Encoder[T], ttlConfig: TTLConfig): ValueState[T]
    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  22. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  23. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  24. def initializeLogIfNecessary(isInterpreter: Boolean): Unit
    Attributes
    protected
    Definition Classes
    Logging
  25. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  26. def isTraceEnabled(): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  27. def listTimers(): Iterator[Long]

    Function to list all the registered timers for given implicit key Note: calling listTimers() within the handleInputRows method of the StatefulProcessor will return all the unprocessed registered timers, including the one being fired within the invocation of handleInputRows.

    Function to list all the registered timers for given implicit key Note: calling listTimers() within the handleInputRows method of the StatefulProcessor will return all the unprocessed registered timers, including the one being fired within the invocation of handleInputRows.

    returns

    - iterator of all the registered timers for given implicit key

    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  28. def log: Logger
    Attributes
    protected
    Definition Classes
    Logging
  29. def logDebug(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  30. def logDebug(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  31. def logDebug(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  32. def logDebug(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  33. def logError(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  34. def logError(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  35. def logError(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  36. def logError(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  37. def logInfo(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  38. def logInfo(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  39. def logInfo(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  40. def logInfo(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  41. def logName: String
    Attributes
    protected
    Definition Classes
    Logging
  42. def logTrace(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  43. def logTrace(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  44. def logTrace(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  45. def logTrace(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  46. def logWarning(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  47. def logWarning(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  48. def logWarning(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  49. def logWarning(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  50. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  51. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  52. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  53. def registerTimer(expiryTimestampMs: Long): Unit

    Function to register a timer for the given expiryTimestampMs

    Function to register a timer for the given expiryTimestampMs

    expiryTimestampMs

    - timestamp in milliseconds for the timer to expire

    Definition Classes
    StatefulProcessorHandleImpl → StatefulProcessorHandle
  54. def setHandleState(newState: StatefulProcessorHandleState): Unit
    Definition Classes
    StatefulProcessorHandleImplBase
  55. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  56. def toString(): String
    Definition Classes
    AnyRef → Any
  57. def verifyStateVarOperations(operationType: String, requiredState: StatefulProcessorHandleState): Unit
    Definition Classes
    StatefulProcessorHandleImplBase
  58. def verifyTimerOperations(operationType: String): Unit
    Definition Classes
    StatefulProcessorHandleImplBase
  59. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  60. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  61. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  62. def withLogContext(context: Map[String, String])(body: => Unit): Unit
    Attributes
    protected
    Definition Classes
    Logging

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from Logging

Inherited from StatefulProcessorHandleImplBase

Inherited from StatefulProcessorHandle

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped