Packages

package catalog

Type Members

  1. class DeltaCatalog extends AbstractDeltaCatalog

    A Spark catalog plugin for Delta Lake tables that implements the Spark DataSource V2 Catalog API.

    A Spark catalog plugin for Delta Lake tables that implements the Spark DataSource V2 Catalog API.

    To use this catalog, configure it in your Spark session:

    
    // Scala example
    val spark = SparkSession
      .builder()
      .appName("...")
      .config("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog")
      .config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
      .getOrCreate()
    
    
    // Python example
    spark = SparkSession
      .builder
      .appName("...")
      .config("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog")
      .config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
      .getOrCreate()
    
    

    Architecture and Delegation Logic

    This class sits in the delta-spark (unified) module and provides a single entry point for Delta Lake catalog operations.

    The unified module can access both implementations:

    • V1 connector: DeltaTableV2 - Legacy connector using DeltaLog, full read/write support
    • V2 connector: SparkTable - sparkV2 connector, read-only support

    See DeltaV2Mode for V1 vs V2 connector definitions and enable mode configuration.

Ungrouped