Package com.lancedb.lance.spark

Class LanceDataSource

java.lang.Object
com.lancedb.lance.spark.LanceDataSource
All Implemented Interfaces:
org.apache.spark.sql.connector.catalog.SupportsCatalogOptions, org.apache.spark.sql.connector.catalog.TableProvider, org.apache.spark.sql.sources.DataSourceRegister
public abstract class LanceDataSource extends Object implements org.apache.spark.sql.connector.catalog.SupportsCatalogOptions, org.apache.spark.sql.sources.DataSourceRegister

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    name
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    LanceDataSource()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    abstract LanceDataset
    createDataset(LanceConfig config, org.apache.spark.sql.types.StructType sparkSchema)
     
    String
    extractCatalog(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
     
    org.apache.spark.sql.connector.catalog.Identifier
    extractIdentifier(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
     
    org.apache.spark.sql.connector.catalog.Table
    getTable(org.apache.spark.sql.types.StructType schema, org.apache.spark.sql.connector.expressions.Transform[] partitioning, Map<String,String> properties)
     
    org.apache.spark.sql.types.StructType
    inferSchema(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
     
    String
    shortName()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.sql.connector.catalog.SupportsCatalogOptions

    extractTimeTravelTimestamp, extractTimeTravelVersion

    Methods inherited from interface org.apache.spark.sql.connector.catalog.TableProvider

    inferPartitioning, supportsExternalMetadata

  • Field Details

  • Constructor Details

    • LanceDataSource

      public LanceDataSource()

  • Method Details

    • inferSchema

      public org.apache.spark.sql.types.StructType inferSchema(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
      Specified by:
      inferSchema in interface org.apache.spark.sql.connector.catalog.TableProvider

    • getTable

      public org.apache.spark.sql.connector.catalog.Table getTable(org.apache.spark.sql.types.StructType schema, org.apache.spark.sql.connector.expressions.Transform[] partitioning, Map<String,String> properties)
      Specified by:
      getTable in interface org.apache.spark.sql.connector.catalog.TableProvider

    • shortName

      public String shortName()
      Specified by:
      shortName in interface org.apache.spark.sql.sources.DataSourceRegister

    • extractIdentifier

      public org.apache.spark.sql.connector.catalog.Identifier extractIdentifier(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
      Specified by:
      extractIdentifier in interface org.apache.spark.sql.connector.catalog.SupportsCatalogOptions

    • extractCatalog

      public String extractCatalog(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
      Specified by:
      extractCatalog in interface org.apache.spark.sql.connector.catalog.SupportsCatalogOptions

    • createDataset

      public abstract LanceDataset createDataset(LanceConfig config, org.apache.spark.sql.types.StructType sparkSchema)