Package com.lancedb.lance.spark

Class LanceDataset

java.lang.Object
com.lancedb.lance.spark.LanceDataset
All Implemented Interfaces:
org.apache.spark.sql.connector.catalog.SupportsMetadataColumns, org.apache.spark.sql.connector.catalog.SupportsRead, org.apache.spark.sql.connector.catalog.SupportsWrite, org.apache.spark.sql.connector.catalog.Table
public class LanceDataset extends Object implements org.apache.spark.sql.connector.catalog.SupportsRead, org.apache.spark.sql.connector.catalog.SupportsWrite, org.apache.spark.sql.connector.catalog.SupportsMetadataColumns
Lance Spark Dataset.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final org.apache.spark.sql.connector.catalog.MetadataColumn[]
    METADATA_COLUMNS
     
    protected final org.apache.spark.sql.types.StructType
    sparkSchema
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    LanceDataset(LanceConfig config, org.apache.spark.sql.types.StructType sparkSchema)
    Creates a Lance dataset.

  • Method Summary

    Modifier and Type
    Method
    Description
    Set<org.apache.spark.sql.connector.catalog.TableCapability>
    capabilities()
     
    LanceConfig
    config()
     
    org.apache.spark.sql.connector.catalog.MetadataColumn[]
    metadataColumns()
     
    String
    name()
     
    org.apache.spark.sql.connector.read.ScanBuilder
    newScanBuilder(org.apache.spark.sql.util.CaseInsensitiveStringMap caseInsensitiveStringMap)
     
    org.apache.spark.sql.connector.write.WriteBuilder
    newWriteBuilder(org.apache.spark.sql.connector.write.LogicalWriteInfo logicalWriteInfo)
     
    org.apache.spark.sql.types.StructType
    schema()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.sql.connector.catalog.SupportsMetadataColumns

    canRenameConflictingMetadataColumns

    Methods inherited from interface org.apache.spark.sql.connector.catalog.Table

    columns, partitioning, properties

  • Field Details

    • METADATA_COLUMNS

      public static final org.apache.spark.sql.connector.catalog.MetadataColumn[] METADATA_COLUMNS

    • sparkSchema

      protected final org.apache.spark.sql.types.StructType sparkSchema

  • Constructor Details

    • LanceDataset

      public LanceDataset(LanceConfig config, org.apache.spark.sql.types.StructType sparkSchema)
      Creates a Lance dataset.
      Parameters:
      config - read config
      sparkSchema - spark struct type

  • Method Details

    • config

      public LanceConfig config()

    • newScanBuilder

      public org.apache.spark.sql.connector.read.ScanBuilder newScanBuilder(org.apache.spark.sql.util.CaseInsensitiveStringMap caseInsensitiveStringMap)
      Specified by:
      newScanBuilder in interface org.apache.spark.sql.connector.catalog.SupportsRead

    • name

      public String name()
      Specified by:
      name in interface org.apache.spark.sql.connector.catalog.Table

    • schema

      public org.apache.spark.sql.types.StructType schema()
      Specified by:
      schema in interface org.apache.spark.sql.connector.catalog.Table

    • capabilities

      public Set<org.apache.spark.sql.connector.catalog.TableCapability> capabilities()
      Specified by:
      capabilities in interface org.apache.spark.sql.connector.catalog.Table

    • newWriteBuilder

      public org.apache.spark.sql.connector.write.WriteBuilder newWriteBuilder(org.apache.spark.sql.connector.write.LogicalWriteInfo logicalWriteInfo)
      Specified by:
      newWriteBuilder in interface org.apache.spark.sql.connector.catalog.SupportsWrite

    • metadataColumns

      public org.apache.spark.sql.connector.catalog.MetadataColumn[] metadataColumns()
      Specified by:
      metadataColumns in interface org.apache.spark.sql.connector.catalog.SupportsMetadataColumns