Package com.lancedb.lance.spark.read

Class LanceScan

java.lang.Object

com.lancedb.lance.spark.read.LanceScan

All Implemented Interfaces:

Serializable, org.apache.spark.sql.connector.read.Batch, org.apache.spark.sql.connector.read.Scan, org.apache.spark.sql.connector.read.SupportsReportStatistics, org.apache.spark.sql.internal.connector.SupportsMetadata

public class LanceScan
extends Object
implements org.apache.spark.sql.connector.read.Batch, org.apache.spark.sql.connector.read.Scan, org.apache.spark.sql.internal.connector.SupportsMetadata, org.apache.spark.sql.connector.read.SupportsReportStatistics, Serializable

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
LanceScan(org.apache.spark.sql.types.StructType schema, LanceConfig config, Optional<String> whereConditions, Optional<Integer> limit, Optional<Integer> offset, Optional<List<com.lancedb.lance.ipc.ColumnOrdering>> topNSortOrders, Optional<org.apache.spark.sql.connector.expressions.aggregate.Aggregation> pushedAggregation)

Method Summary

Modifier and Type	Method	Description
org.apache.spark.sql.connector.read.PartitionReaderFactory	createReaderFactory()
org.apache.spark.sql.connector.read.Statistics	estimateStatistics()
scala.collection.immutable.Map<String,String>	getMetaData()
org.apache.spark.sql.connector.read.InputPartition[]	planInputPartitions()
org.apache.spark.sql.types.StructType	readSchema()
org.apache.spark.sql.connector.read.Batch	toBatch()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.connector.read.Scan

description, reportDriverMetrics, supportedCustomMetrics, toContinuousStream, toMicroBatchStream

Constructor Details

LanceScan

public LanceScan(org.apache.spark.sql.types.StructType schema,
 LanceConfig config,
 Optional<String> whereConditions,
 Optional<Integer> limit,
 Optional<Integer> offset,
 Optional<List<com.lancedb.lance.ipc.ColumnOrdering>> topNSortOrders,
 Optional<org.apache.spark.sql.connector.expressions.aggregate.Aggregation> pushedAggregation)

Method Details

toBatch

public org.apache.spark.sql.connector.read.Batch toBatch()

Specified by:

toBatch in interface org.apache.spark.sql.connector.read.Scan

planInputPartitions

public org.apache.spark.sql.connector.read.InputPartition[] planInputPartitions()

Specified by:

planInputPartitions in interface org.apache.spark.sql.connector.read.Batch

createReaderFactory

public org.apache.spark.sql.connector.read.PartitionReaderFactory createReaderFactory()

Specified by:

createReaderFactory in interface org.apache.spark.sql.connector.read.Batch

readSchema

public org.apache.spark.sql.types.StructType readSchema()

Specified by:

readSchema in interface org.apache.spark.sql.connector.read.Scan

getMetaData

public scala.collection.immutable.Map<String,String> getMetaData()

Specified by:

getMetaData in interface org.apache.spark.sql.internal.connector.SupportsMetadata

estimateStatistics

public org.apache.spark.sql.connector.read.Statistics estimateStatistics()

Specified by:

estimateStatistics in interface org.apache.spark.sql.connector.read.SupportsReportStatistics