Package com.lancedb.lance.spark.utils

Class SchemaConverter

java.lang.Object

com.lancedb.lance.spark.utils.SchemaConverter

public class SchemaConverter
extends Object

Utility class for converting Spark schema types to JsonArrow schema types used by the Lance Namespace API.

Method Summary

Modifier and Type	Method	Description
static org.apache.spark.sql.types.StructType	processSchemaWithProperties(org.apache.spark.sql.types.StructType sparkSchema, Map<String,String> properties)	Processes a Spark schema with table properties to add metadata for vector and blob columns.
static com.lancedb.lance.namespace.model.JsonArrowSchema	toJsonArrowSchema(org.apache.spark.sql.types.StructType sparkSchema)	Converts a Spark StructType to JsonArrowSchema.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

toJsonArrowSchema

public static com.lancedb.lance.namespace.model.JsonArrowSchema toJsonArrowSchema(org.apache.spark.sql.types.StructType sparkSchema)

Converts a Spark StructType to JsonArrowSchema.

Parameters:

sparkSchema - the Spark StructType to convert

Returns:

JsonArrowSchema representation

processSchemaWithProperties

public static org.apache.spark.sql.types.StructType processSchemaWithProperties(org.apache.spark.sql.types.StructType sparkSchema,
 Map<String,String> properties)

Processes a Spark schema with table properties to add metadata for vector and blob columns.

Parameters:

sparkSchema - the original Spark StructType

properties - table properties that may contain vector column metadata or blob encoding

Returns:

StructType with metadata added for vector and blob columns