trait DataEncoder extends AnyRef
The DataEncoder can encode UnsafeRows into raw bytes in two ways:
- Using the direct byte layout of the UnsafeRow
- Converting the UnsafeRow into an Avro row, and encoding that In both of these cases, the raw bytes that are written into RockDB have headers, footers and other metadata, but they also have data that is provided by the callers. The metadata in each row does not need to be written as Avro or UnsafeRow, but the actual data provided by the caller does. The classes that use this trait require specialized partial encoding which makes them much easier to cache and use, which is why each DataEncoder deals with multiple schemas.
- Alphabetic
- By Inheritance
- DataEncoder
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def decodeKey(bytes: Array[Byte]): UnsafeRow
Decodes a complete key from its serialized byte form.
Decodes a complete key from its serialized byte form.
For NoPrefixKeyStateEncoder: Decodes the entire key For PrefixKeyScanStateEncoder: Decodes only the prefix portion
- bytes
Serialized byte array containing the encoded key
- returns
UnsafeRow containing the decoded key columns
- Exceptions thrown
UnsupportedOperationExceptionfor unsupported encoder types
- abstract def decodePrefixKeyForRangeScan(bytes: Array[Byte]): UnsafeRow
Decodes range scan key bytes back into an UnsafeRow, preserving proper ordering.
Decodes range scan key bytes back into an UnsafeRow, preserving proper ordering.
This method reverses the special encoding done by encodePrefixKeyForRangeScan: - Interprets sign byte markers - Reverses bit flipping for negative floating point values - Handles null values
- bytes
Serialized byte array containing the encoded range scan key
- returns
UnsafeRow containing the decoded range scan columns
- Exceptions thrown
UnsupportedOperationExceptionif called on an encoder that doesn't support range scans
- abstract def decodeRemainingKey(bytes: Array[Byte]): UnsafeRow
Decodes the remaining portion of a split key from its serialized form.
Decodes the remaining portion of a split key from its serialized form.
For PrefixKeyScanStateEncoder: Decodes columns after the prefix For RangeKeyScanStateEncoder: Decodes non-ordering columns
- bytes
Serialized byte array containing the encoded remaining key portion
- returns
UnsafeRow containing the decoded remaining key columns
- Exceptions thrown
UnsupportedOperationExceptionif called on an encoder that doesn't support split keys
- abstract def decodeValue(bytes: Array[Byte]): UnsafeRow
Decodes a value from its serialized byte form.
Decodes a value from its serialized byte form.
- bytes
Serialized byte array containing the encoded value
- returns
UnsafeRow containing the decoded value columns
- abstract def encodeKey(row: UnsafeRow): Array[Byte]
Encodes a complete key row into bytes.
Encodes a complete key row into bytes. Used as the primary key for state lookups.
- row
An UnsafeRow containing all key columns as defined in the keySchema
- returns
Serialized byte array representation of the key
- abstract def encodePrefixKeyForRangeScan(row: UnsafeRow): Array[Byte]
Encodes key columns used for range scanning, ensuring proper sort order in RocksDB.
Encodes key columns used for range scanning, ensuring proper sort order in RocksDB.
This method handles special encoding for numeric types to maintain correct sort order: - Adds sign byte markers for numeric types - Flips bits for negative floating point values - Preserves null ordering
- row
An UnsafeRow containing the columns needed for range scan (specified by orderingOrdinals)
- returns
Serialized bytes that will maintain correct sort order in RocksDB
- Exceptions thrown
UnsupportedOperationExceptionif called on an encoder that doesn't support range scans
- abstract def encodeRemainingKey(row: UnsafeRow): Array[Byte]
Encodes the non-prefix portion of a key row.
Encodes the non-prefix portion of a key row. Used with prefix scan and range scan state lookups where the key is split into prefix and remaining portions.
For prefix scans: Encodes columns after the prefix columns For range scans: Encodes columns not included in the ordering columns
- row
An UnsafeRow containing only the remaining key columns
- returns
Serialized byte array of the remaining key portion
- Exceptions thrown
UnsupportedOperationExceptionif called on an encoder that doesn't support split keys
- abstract def encodeValue(row: UnsafeRow): Array[Byte]
Encodes a value row into bytes.
Encodes a value row into bytes.
- row
An UnsafeRow containing the value columns as defined in the valueSchema
- returns
Serialized byte array representation of the value
- abstract def supportsSchemaEvolution: Boolean
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)