class ListStateImpl[S] extends ListStateMetricsImpl with ListState[S] with Logging
Provides concrete implementation for list of values associated with a state variable used in the streaming transformWithState operator.
- S
- data type of object that will be stored in the list
- Alphabetic
- By Inheritance
- ListStateImpl
- Logging
- ListState
- Serializable
- ListStateMetricsImpl
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new ListStateImpl(store: StateStore, stateName: String, keyExprEnc: ExpressionEncoder[Any], valEncoder: ExpressionEncoder[Any], metrics: Map[String, SQLMetric] = Map.empty)
- store
- reference to the StateStore instance to be used for storing state
- stateName
- name of logical state partition
- keyExprEnc
- Spark SQL encoder for key
- valEncoder
- Spark SQL encoder for value
- metrics
- metrics to be updated as part of stateful processing
Type Members
- implicit class LogStringContext extends AnyRef
- Definition Classes
- Logging
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def appendList(newState: Array[S]): Unit
Append an entire list to the existing value.
Append an entire list to the existing value.
- Definition Classes
- ListStateImpl → ListState
- def appendValue(newState: S): Unit
Append an entry to the list.
Append an entry to the list.
- Definition Classes
- ListStateImpl → ListState
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def baseStateName: String
- Definition Classes
- ListStateImpl → ListStateMetricsImpl
- def clear(): Unit
Remove this state.
Remove this state.
- Definition Classes
- ListStateImpl → ListState
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def exists(): Boolean
Whether state exists or not.
Whether state exists or not.
- Definition Classes
- ListStateImpl → ListState
- def exprEncSchema: StructType
- Definition Classes
- ListStateImpl → ListStateMetricsImpl
- def get(): Iterator[S]
Get the state value if it exists.
Get the state value if it exists. If the state does not exist in state store, an empty iterator is returned.
- Definition Classes
- ListStateImpl → ListState
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def getEntryCount(encodedKey: UnsafeRow): Long
Function to get the number of entries in the list state for a given grouping key
Function to get the number of entries in the list state for a given grouping key
- encodedKey
- encoded grouping key
- returns
- number of entries in the list state
- Definition Classes
- ListStateMetricsImpl
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def initializeLogIfNecessary(isInterpreter: Boolean): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isTraceEnabled(): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def log: Logger
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logName: String
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def put(newState: Array[S]): Unit
Update the value of the list.
Update the value of the list.
- Definition Classes
- ListStateImpl → ListState
- def removeEntryCount(encodedKey: UnsafeRow): Unit
Function to remove the number of entries in the list state for a given grouping key
Function to remove the number of entries in the list state for a given grouping key
- encodedKey
- encoded grouping key
- Definition Classes
- ListStateMetricsImpl
- def stateStore: StateStore
- Definition Classes
- ListStateImpl → ListStateMetricsImpl
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def updateEntryCount(encodedKey: UnsafeRow, updatedCount: Long): Unit
Function to update the number of entries in the list state for a given grouping key
Function to update the number of entries in the list state for a given grouping key
- encodedKey
- encoded grouping key
- updatedCount
- updated count of entries in the list state
- Definition Classes
- ListStateMetricsImpl
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- def withLogContext(context: Map[String, String])(body: => Unit): Unit
- Attributes
- protected
- Definition Classes
- Logging
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)