class StateStoreConf extends Serializable
A class that contains configuration parameters for StateStores.
- Alphabetic
- By Inheritance
- StateStoreConf
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- val compressionCodec: String
The compression codec used to compress delta and snapshot files.
- val enableStateStoreCheckpointIds: Boolean
When creating new state store checkpoint, which format version to use.
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- val extraOptions: Map[String, String]
- val formatValidationCheckValue: Boolean
Whether to validate the value side.
Whether to validate the value side. This config is applied to both validators as below:
- whether to validate the value format when the format validation is enabled. - whether to validate the value schema when the state schema check is enabled.
- val formatValidationEnabled: Boolean
Whether validate the underlying format or not.
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- val maintenanceInterval: Long
The interval of maintenance tasks.
- val maxVersionsToRetainInMemory: Int
Maximum count of versions a State Store implementation should retain in memory
- val minDeltasForSnapshot: Int
Minimum number of delta files in a chain after which HDFSBackedStateStore will consider generating a snapshot.
- val minVersionsToDelete: Long
Minimum number of stale checkpoint versions that need to be present in the DFS checkpoint directory for old state checkpoint version deletion to be invoked.
Minimum number of stale checkpoint versions that need to be present in the DFS checkpoint directory for old state checkpoint version deletion to be invoked. This is to amortize the cost of discovering and deleting old checkpoint versions.
- val minVersionsToRetain: Int
Minimum versions a State Store implementation should retain to allow rollbacks
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- val numStateStoreMaintenanceThreads: Int
Size of MaintenanceThreadPool to perform maintenance tasks for StateStore
- val providerClass: String
Optional fully qualified name of the subclass of StateStoreProvider managing state data.
Optional fully qualified name of the subclass of StateStoreProvider managing state data. That is, the implementation of the State Store to use.
- val skipNullsForStreamStreamJoins: Boolean
Whether to skip null values for hash based stream-stream joins.
- val sqlConfs: Map[String, String]
Additional configurations related to state store.
Additional configurations related to state store. This will capture all configs in SQLConf that start with
spark.sql.streaming.stateStore. - val stateSchemaCheckEnabled: Boolean
whether to validate state schema during query run.
- val stateStoreEncodingFormat: String
The interval of maintenance tasks.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)