com.lancedb.lance.spark

Class LanceDataset

java.lang.Object

com.lancedb.lance.spark.LanceDataset

All Implemented Interfaces:

org.apache.spark.sql.connector.catalog.SupportsMetadataColumns, org.apache.spark.sql.connector.catalog.SupportsRead, org.apache.spark.sql.connector.catalog.SupportsWrite, org.apache.spark.sql.connector.catalog.Table

public class LanceDataset
extends Object
implements org.apache.spark.sql.connector.catalog.SupportsRead, org.apache.spark.sql.connector.catalog.SupportsWrite, org.apache.spark.sql.connector.catalog.SupportsMetadataColumns

Lance Spark Dataset.

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.spark.sql.connector.catalog.MetadataColumn[]	METADATA_COLUMNS

Constructor Summary

Constructors

Constructor and Description
LanceDataset(LanceConfig config, org.apache.spark.sql.types.StructType sparkSchema) Creates a Lance dataset.

Method Summary

Modifier and Type	Method and Description
Set<org.apache.spark.sql.connector.catalog.TableCapability>	capabilities()
org.apache.spark.sql.connector.catalog.MetadataColumn[]	metadataColumns()
String	name()
org.apache.spark.sql.connector.read.ScanBuilder	newScanBuilder(org.apache.spark.sql.util.CaseInsensitiveStringMap caseInsensitiveStringMap)
org.apache.spark.sql.connector.write.WriteBuilder	newWriteBuilder(org.apache.spark.sql.connector.write.LogicalWriteInfo logicalWriteInfo)
org.apache.spark.sql.types.StructType	schema()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.connector.catalog.SupportsMetadataColumns

canRenameConflictingMetadataColumns

Methods inherited from interface org.apache.spark.sql.connector.catalog.Table

columns, partitioning, properties

Field Detail

METADATA_COLUMNS

public static final org.apache.spark.sql.connector.catalog.MetadataColumn[] METADATA_COLUMNS

Constructor Detail

LanceDataset

public LanceDataset(LanceConfig config,
                    org.apache.spark.sql.types.StructType sparkSchema)

Creates a Lance dataset.

Parameters:

config - read config

sparkSchema - spark struct type

Method Detail

newScanBuilder

public org.apache.spark.sql.connector.read.ScanBuilder newScanBuilder(org.apache.spark.sql.util.CaseInsensitiveStringMap caseInsensitiveStringMap)

Specified by:

newScanBuilder in interface org.apache.spark.sql.connector.catalog.SupportsRead

name

public String name()

Specified by:

name in interface org.apache.spark.sql.connector.catalog.Table

schema

public org.apache.spark.sql.types.StructType schema()

Specified by:

schema in interface org.apache.spark.sql.connector.catalog.Table

capabilities

public Set<org.apache.spark.sql.connector.catalog.TableCapability> capabilities()

Specified by:

capabilities in interface org.apache.spark.sql.connector.catalog.Table

newWriteBuilder

public org.apache.spark.sql.connector.write.WriteBuilder newWriteBuilder(org.apache.spark.sql.connector.write.LogicalWriteInfo logicalWriteInfo)

Specified by:

newWriteBuilder in interface org.apache.spark.sql.connector.catalog.SupportsWrite

metadataColumns

public org.apache.spark.sql.connector.catalog.MetadataColumn[] metadataColumns()

Specified by:

metadataColumns in interface org.apache.spark.sql.connector.catalog.SupportsMetadataColumns