abstract class RocksDBDataEncoder extends DataEncoder
- Alphabetic
- By Inheritance
- RocksDBDataEncoder
- DataEncoder
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new RocksDBDataEncoder(keyStateEncoderSpec: KeyStateEncoderSpec, valueSchema: StructType)
Abstract Value Members
- abstract def decodeKey(bytes: Array[Byte]): UnsafeRow
Decodes a complete key from its serialized byte form.
Decodes a complete key from its serialized byte form.
For NoPrefixKeyStateEncoder: Decodes the entire key For PrefixKeyScanStateEncoder: Decodes only the prefix portion
- bytes
Serialized byte array containing the encoded key
- returns
UnsafeRow containing the decoded key columns
- Definition Classes
- DataEncoder
- Exceptions thrown
UnsupportedOperationExceptionfor unsupported encoder types
- abstract def decodePrefixKeyForRangeScan(bytes: Array[Byte]): UnsafeRow
Decodes range scan key bytes back into an UnsafeRow, preserving proper ordering.
Decodes range scan key bytes back into an UnsafeRow, preserving proper ordering.
This method reverses the special encoding done by encodePrefixKeyForRangeScan: - Interprets sign byte markers - Reverses bit flipping for negative floating point values - Handles null values
- bytes
Serialized byte array containing the encoded range scan key
- returns
UnsafeRow containing the decoded range scan columns
- Definition Classes
- DataEncoder
- Exceptions thrown
UnsupportedOperationExceptionif called on an encoder that doesn't support range scans
- abstract def decodeRemainingKey(bytes: Array[Byte]): UnsafeRow
Decodes the remaining portion of a split key from its serialized form.
Decodes the remaining portion of a split key from its serialized form.
For PrefixKeyScanStateEncoder: Decodes columns after the prefix For RangeKeyScanStateEncoder: Decodes non-ordering columns
- bytes
Serialized byte array containing the encoded remaining key portion
- returns
UnsafeRow containing the decoded remaining key columns
- Definition Classes
- DataEncoder
- Exceptions thrown
UnsupportedOperationExceptionif called on an encoder that doesn't support split keys
- abstract def decodeValue(bytes: Array[Byte]): UnsafeRow
Decodes a value from its serialized byte form.
Decodes a value from its serialized byte form.
- bytes
Serialized byte array containing the encoded value
- returns
UnsafeRow containing the decoded value columns
- Definition Classes
- DataEncoder
- abstract def encodeKey(row: UnsafeRow): Array[Byte]
Encodes a complete key row into bytes.
Encodes a complete key row into bytes. Used as the primary key for state lookups.
- row
An UnsafeRow containing all key columns as defined in the keySchema
- returns
Serialized byte array representation of the key
- Definition Classes
- DataEncoder
- abstract def encodePrefixKeyForRangeScan(row: UnsafeRow): Array[Byte]
Encodes key columns used for range scanning, ensuring proper sort order in RocksDB.
Encodes key columns used for range scanning, ensuring proper sort order in RocksDB.
This method handles special encoding for numeric types to maintain correct sort order: - Adds sign byte markers for numeric types - Flips bits for negative floating point values - Preserves null ordering
- row
An UnsafeRow containing the columns needed for range scan (specified by orderingOrdinals)
- returns
Serialized bytes that will maintain correct sort order in RocksDB
- Definition Classes
- DataEncoder
- Exceptions thrown
UnsupportedOperationExceptionif called on an encoder that doesn't support range scans
- abstract def encodeRemainingKey(row: UnsafeRow): Array[Byte]
Encodes the non-prefix portion of a key row.
Encodes the non-prefix portion of a key row. Used with prefix scan and range scan state lookups where the key is split into prefix and remaining portions.
For prefix scans: Encodes columns after the prefix columns For range scans: Encodes columns not included in the ordering columns
- row
An UnsafeRow containing only the remaining key columns
- returns
Serialized byte array of the remaining key portion
- Definition Classes
- DataEncoder
- Exceptions thrown
UnsupportedOperationExceptionif called on an encoder that doesn't support split keys
- abstract def encodeValue(row: UnsafeRow): Array[Byte]
Encodes a value row into bytes.
Encodes a value row into bytes.
- row
An UnsafeRow containing the value columns as defined in the valueSchema
- returns
Serialized byte array representation of the value
- Definition Classes
- DataEncoder
- abstract def supportsSchemaEvolution: Boolean
- Definition Classes
- DataEncoder
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def decodeStateSchemaIdRow(bytes: Array[Byte]): StateSchemaIdRow
- def decodeToUnsafeRow(bytes: Array[Byte], reusedRow: UnsafeRow): UnsafeRow
- def decodeToUnsafeRow(bytes: Array[Byte], numFields: Int): UnsafeRow
- val doubleFlipBitMask: Long
- val doubleSignBitMask: Long
- def encodeUnsafeRow(row: UnsafeRow): Array[Byte]
Encode the UnsafeRow of N bytes as a N+1 byte array.
Encode the UnsafeRow of N bytes as a N+1 byte array.
- Note
This creates a new byte array and memcopies the UnsafeRow to the new array.
- def encodeWithStateSchemaId(schemaIdRow: StateSchemaIdRow): Array[Byte]
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- val floatFlipBitMask: Int
- val floatSignBitMask: Int
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- val keySchema: StructType
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- val negativeValMarker: Byte
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- val nullValMarker: Byte
- val positiveValMarker: Byte
- val reusedKeyRow: UnsafeRow
- val reusedValueRow: UnsafeRow
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def unsupportedOperationForKeyStateEncoder(operation: String): UnsupportedOperationException
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)