Packages

case class OrcTable(name: String, sparkSession: SparkSession, options: CaseInsensitiveStringMap, paths: Seq[String], userSpecifiedSchema: Option[StructType], fallbackFileFormat: Class[_ <: FileFormat]) extends FileTable with Product with Serializable

Linear Supertypes
Serializable, Product, Equals, FileTable, SupportsWrite, SupportsRead, Table, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. OrcTable
  2. Serializable
  3. Product
  4. Equals
  5. FileTable
  6. SupportsWrite
  7. SupportsRead
  8. Table
  9. AnyRef
  10. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new OrcTable(name: String, sparkSession: SparkSession, options: CaseInsensitiveStringMap, paths: Seq[String], userSpecifiedSchema: Option[StructType], fallbackFileFormat: Class[_ <: FileFormat])

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def capabilities(): Set[TableCapability]
    Definition Classes
    FileTable → Table
  6. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  7. def columns(): Array[connector.catalog.Column]
    Definition Classes
    Table
  8. def constraints(): Array[Constraint]
    Definition Classes
    Table
  9. lazy val dataSchema: StructType
    Definition Classes
    FileTable
  10. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  11. val fallbackFileFormat: Class[_ <: FileFormat]

    Returns a V1 FileFormat class of the same file data source.

    Returns a V1 FileFormat class of the same file data source. This is a solution for the following cases: 1. File datasource V2 implementations cause regression. Users can disable the problematic data source via SQL configuration and fall back to FileFormat. 2. Catalog support is required, which is still under development for data source V2.

    Definition Classes
    OrcTable → FileTable
  12. lazy val fileIndex: PartitioningAwareFileIndex
    Definition Classes
    FileTable
  13. def formatName: String

    The string that represents the format that this data source provider uses.

    The string that represents the format that this data source provider uses. This is overridden by children to provide a nice alias for the data source. For example:

    override def formatName(): String = "ORC"
    Definition Classes
    OrcTable → FileTable
  14. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  15. def id(): String
    Definition Classes
    Table
  16. def inferSchema(files: Seq[FileStatus]): Option[StructType]

    When possible, this method should return the schema of the given files.

    When possible, this method should return the schema of the given files. When the format does not support inference, or no valid files are given should return None. In these cases Spark will require that user specify the schema manually.

    Definition Classes
    OrcTable → FileTable
  17. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  18. def mergedOptions(options: CaseInsensitiveStringMap): CaseInsensitiveStringMap

    Merge the options of FileTable and the table operation while respecting the keys of the table operation.

    Merge the options of FileTable and the table operation while respecting the keys of the table operation.

    options

    The options of the table operation.

    Attributes
    protected
    Definition Classes
    FileTable
  19. def mergedWriteInfo(writeInfo: LogicalWriteInfo): LogicalWriteInfo

    Merge the options of FileTable and the LogicalWriteInfo while respecting the keys of the options carried by LogicalWriteInfo.

    Merge the options of FileTable and the LogicalWriteInfo while respecting the keys of the options carried by LogicalWriteInfo.

    Attributes
    protected
    Definition Classes
    FileTable
  20. val name: String
    Definition Classes
    OrcTable → Table
  21. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  22. def newScanBuilder(options: CaseInsensitiveStringMap): OrcScanBuilder
    Definition Classes
    OrcTable → SupportsRead
  23. def newWriteBuilder(info: LogicalWriteInfo): WriteBuilder
    Definition Classes
    OrcTable → SupportsWrite
  24. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  25. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  26. val options: CaseInsensitiveStringMap
  27. def partitioning(): Array[Transform]
    Definition Classes
    FileTable → Table
  28. val paths: Seq[String]
  29. def productElementNames: Iterator[String]
    Definition Classes
    Product
  30. def properties(): Map[String, String]
    Definition Classes
    FileTable → Table
  31. lazy val schema: StructType
    Definition Classes
    FileTable → Table
  32. val sparkSession: SparkSession
  33. def supportsDataType(dataType: DataType): Boolean

    Returns whether this format supports the given DataType in read/write path.

    Returns whether this format supports the given DataType in read/write path. By default all data types are supported.

    Definition Classes
    OrcTable → FileTable
  34. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  35. val userSpecifiedSchema: Option[StructType]
  36. def version(): String
    Definition Classes
    Table
  37. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  38. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  39. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from FileTable

Inherited from SupportsWrite

Inherited from SupportsRead

Inherited from Table

Inherited from AnyRef

Inherited from Any

Ungrouped