object HiveHashFunction extends InterpretedHashFunction
- Alphabetic
- By Inheritance
- HiveHashFunction
- InterpretedHashFunction
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hash(value: Any, dataType: DataType, seed: Long, isCollationAware: Boolean, legacyCollationAwareHashing: Boolean): Long
Computes hash of a given
valueof typedataType.Computes hash of a given
valueof typedataType. The caller needs to check the validity of inputvalue. TheisCollationAwareboolean flag indicates whether hashing should take a string's collation into account. If not, the bytes of the string are hashed, otherwise the collation key of the string is hashed.- Definition Classes
- HiveHashFunction → InterpretedHashFunction
- def hash(value: Any, dataType: DataType, seed: Long): Long
This method is intended for callers using the old hash API and preserves compatibility for supported data types.
This method is intended for callers using the old hash API and preserves compatibility for supported data types. It must only be used for data types that do not include collated strings or complex types (e.g., structs, arrays, maps) that may contain collated strings.
The caller is responsible for ensuring that
dataTypedoes not involve collation-aware fields. This is validated via an internal assertion.- Definition Classes
- InterpretedHashFunction
- Exceptions thrown
IllegalArgumentExceptionifdataTypecontains non-UTF8 binary collation.
- def hashCalendarInterval(calendarInterval: CalendarInterval): Long
Hive allows input intervals to be defined using units below but the intervals have to be from the same category: - year, month (stored as HiveIntervalYearMonth) - day, hour, minute, second, nanosecond (stored as HiveIntervalDayTime)
Hive allows input intervals to be defined using units below but the intervals have to be from the same category: - year, month (stored as HiveIntervalYearMonth) - day, hour, minute, second, nanosecond (stored as HiveIntervalDayTime)
e.g. (INTERVAL '30' YEAR + INTERVAL '-23' DAY) fails in Hive
This method mimics HiveIntervalDayTime.hashCode() in Hive.
Two differences wrt Hive due to how intervals are stored in Spark vs Hive:
- If the
INTERVALis backed as HiveIntervalYearMonth in Hive, then this method will not produce Hive compatible result. The reason being Spark's representation of calendar does not have such categories based on the interval and is unified.- Spark's CalendarInterval has precision upto microseconds but Hive's HiveIntervalDayTime can store data with precision upto nanoseconds. So, any input intervals with nanosecond values will lead to wrong output hashes (i.e. non adherent with Hive output)
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashInt(i: Int, seed: Long): Long
- Attributes
- protected
- Definition Classes
- HiveHashFunction → InterpretedHashFunction
- def hashLong(l: Long, seed: Long): Long
- Attributes
- protected
- Definition Classes
- HiveHashFunction → InterpretedHashFunction
- def hashTimestamp(timestamp: Long): Long
Mimics TimestampWritable.hashCode() in Hive
- def hashUnsafeBytes(base: AnyRef, offset: Long, len: Int, seed: Long): Long
- Attributes
- protected
- Definition Classes
- HiveHashFunction → InterpretedHashFunction
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def normalizeDecimal(input: BigDecimal): BigDecimal
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)