Type Members

1.  case class CompositeLimit(bytes: ReadMaxBytes, files: ReadMaxFiles) extends ReadLimit with Product with Serializable

   A read limit that admits the given soft-max of bytes or max files.

2.  abstract class DeltaSharingExceptionWithErrorCode extends IllegalStateException
3.  trait DeltaSharingOptionParser extends AnyRef
4.  class DeltaSharingOptions extends DeltaSharingReadOptions with Serializable

   Options for the Delta Sharing data source.

5.  trait DeltaSharingReadOptions extends DeltaSharingOptionParser
6.  class DeltaSharingServerException extends DeltaSharingExceptionWithErrorCode
7.  case class DeltaSharingSource(spark: SparkSession, deltaLog: RemoteDeltaLog, options: DeltaSharingOptions) extends Source with SupportsAdmissionControl with SupportsTriggerAvailableNow with Logging with Product with Serializable

   A streaming source for a Delta Sharing table.

   A streaming source for a Delta Sharing table.

   When a new stream is started, delta sharing starts by constructing a RemoteDeltaSnapshot at the current version of the table. This snapshot is broken up into batches until all existing data has been processed. Subsequent processing is done by tailing the change log looking for new data. This results in the streaming query returning the same answer as a batch query that had processed the entire dataset at any given point.

8.  case class DeltaSharingSourceOffset(sourceVersion: Long, tableId: String, tableVersion: Long, index: Long, isStartingVersion: Boolean) extends Offset with Product with Serializable

   Tracks how far we processed in when reading changes from the Sharing Server.

   Tracks how far we processed in when reading changes from the Sharing Server.

   sourceVersion

   The version of serialization that this offset is encoded with.

   tableId

   The id of the table we are reading from. Used to detect misconfiguration when restarting a query.

   tableVersion

   The version of the table that we are currently processing.

   index

   The index in the sequence of AddFiles in this version. Used to break large commits into multiple batches. This index is created by sorting on url.

   isStartingVersion

   Whether this offset denotes a query that is starting rather than processing changes. When starting a new query, we first process all data present in the table at the start and then move on to processing new data that has arrived.

9.  sealed trait DeltaStartingVersion extends AnyRef

   Definitions for the starting version of a Delta stream.

10.  class MissingEndStreamActionException extends IllegalStateException
11.  case class ReadMaxBytes(maxBytes: Long) extends ReadLimit with Product with Serializable

    A read limit that admits a soft-max of maxBytes per micro-batch.

12.  case class RemoteDeltaCDFRelation(spark: SparkSession, snapshotToUse: RemoteSnapshot, client: DeltaSharingClient, table: Table, cdfOptions: Map[String, String]) extends BaseRelation with PrunedFilteredScan with Product with Serializable
13.  case class RemoteDeltaSnapshotFileIndex(params: RemoteDeltaFileIndexParams, limitHint: Option[Long]) extends RemoteDeltaFileIndexBase with Product with Serializable
14.  class RemoteSnapshot extends Logging

    An immutable snapshot of a Delta table at some delta version

15.  case class StartingVersion(version: Long) extends DeltaStartingVersion with Product with Serializable