Packages

package avro

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. avro
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. case class AvroDataToCatalyst(child: Expression, jsonFormatSchema: String, options: Map[String, String]) extends UnaryExpression with ExpectsInputTypes with Product with Serializable
  2. class AvroDeserializer extends AnyRef

    A deserializer to deserialize data in avro format to data in catalyst format.

  3. class AvroOptions extends Logging with Serializable

    Options for Avro Reader and Writer stored in case insensitive manner.

  4. class AvroSerializer extends Logging

    A serializer to serialize data in catalyst format to data in avro format.

  5. case class CatalystDataToAvro(child: Expression, jsonFormatSchema: Option[String]) extends UnaryExpression with Product with Serializable
  6. class IncompatibleSchemaException extends Exception
  7. class SparkAvroKeyOutputFormat extends AvroKeyOutputFormat[GenericRecord]

Value Members

  1. object AvroOptions extends Serializable
  2. object AvroUtils extends Logging
  3. object SchemaConverters

    This object contains method that are used to convert sparkSQL schemas to avro schemas and vice versa.

  4. object functions

Deprecated Value Members

  1. def from_avro(data: Column, jsonFormatSchema: String): Column

    Converts a binary column of avro format into its corresponding catalyst value.

    Converts a binary column of avro format into its corresponding catalyst value. The specified schema must match the read data, otherwise the behavior is undefined: it may fail or return arbitrary result.

    data

    the binary column.

    jsonFormatSchema

    the avro schema in JSON string format.

    Annotations
    @deprecated
    Deprecated

    (Since version 3.0.0) Please use 'org.apache.spark.sql.avro.functions.from_avro' instead.

    Since

    2.4.0

  2. def to_avro(data: Column): Column

    Converts a column into binary of avro format.

    Converts a column into binary of avro format.

    data

    the data column.

    Annotations
    @deprecated
    Deprecated

    (Since version 3.0.0) Please use 'org.apache.spark.sql.avro.functions.to_avro' instead.

    Since

    2.4.0

Inherited from AnyRef

Inherited from Any

Ungrouped