Packages

package state

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. Protected

Package Members

  1. package metadata
  2. package utils

Type Members

  1. case class FromSnapshotOptions(snapshotStartBatchId: Long, snapshotPartitionId: Int) extends Product with Serializable
  2. case class ReadChangeFeedOptions(changeStartBatchId: Long, changeEndBatchId: Long) extends Product with Serializable
  3. class StataDataSourceCommittedBatchUnavailable extends StateDataSourceException
  4. class StateDataSource extends TableProvider with DataSourceRegister with Logging

    An implementation of TableProvider with DataSourceRegister for State Store data source.

  5. class StateDataSourceConflictOptions extends StateDataSourceException
  6. abstract class StateDataSourceException extends SparkRuntimeException
  7. class StateDataSourceInternalError extends StateDataSourceException
  8. class StateDataSourceInvalidOptionValue extends StateDataSourceException
  9. class StateDataSourceInvalidOptionValueIsEmpty extends StateDataSourceException
  10. class StateDataSourceInvalidOptionValueIsNegative extends StateDataSourceException
  11. class StateDataSourceNoPartitionDiscoveredInStateStore extends StateDataSourceException
  12. class StateDataSourceOffsetLogUnavailable extends StateDataSourceException
  13. class StateDataSourceOffsetMetadataLogUnavailable extends StateDataSourceException
  14. class StateDataSourceReadOperatorMetadataFailure extends StateDataSourceException
  15. class StateDataSourceReadStateSchemaFailure extends StateDataSourceException
  16. class StateDataSourceUnspecifiedRequiredOption extends StateDataSourceException
  17. class StatePartitionReader extends StatePartitionReaderBase

    An implementation of StatePartitionReaderBase for the normal mode of State Data Source.

    An implementation of StatePartitionReaderBase for the normal mode of State Data Source. It reads the state at a particular batchId.

  18. abstract class StatePartitionReaderBase extends PartitionReader[InternalRow] with Logging

    An implementation of PartitionReader for State data source.

    An implementation of PartitionReader for State data source. This is used to support general read from a state store instance, rather than specific to the operator.

  19. class StatePartitionReaderFactory extends PartitionReaderFactory

    An implementation of PartitionReaderFactory for State data source.

    An implementation of PartitionReaderFactory for State data source. This is used to support general read from a state store instance, rather than specific to the operator.

  20. class StateScan extends Scan with Batch

    An implementation of Scan with Batch for State Store data source.

  21. class StateScanBuilder extends ScanBuilder

    An implementation of ScanBuilder for State Store data source.

  22. case class StateSourceOptions(resolvedCpLocation: String, batchId: Long, operatorId: Int, storeName: String, joinSide: JoinSideValues, readChangeFeed: Boolean, fromSnapshotOptions: Option[FromSnapshotOptions], readChangeFeedOptions: Option[ReadChangeFeedOptions], stateVarName: Option[String], readRegisteredTimers: Boolean, flattenCollectionTypes: Boolean) extends Product with Serializable
  23. class StateStoreChangeDataPartitionReader extends StatePartitionReaderBase

    An implementation of StatePartitionReaderBase for the readChangeFeed mode of State Data Source.

    An implementation of StatePartitionReaderBase for the readChangeFeed mode of State Data Source. It reads the change of state over batches of a particular partition.

  24. class StateStoreInputPartition extends InputPartition

    An implementation of InputPartition for State Store data source.

  25. case class StateStoreReaderInfo(keyStateEncoderSpecOpt: Option[KeyStateEncoderSpec], stateStoreColFamilySchemaOpt: Option[StateStoreColFamilySchema], transformWithStateVariableInfoOpt: Option[TransformWithStateVariableInfo], stateSchemaProviderOpt: Option[StateSchemaProvider]) extends Product with Serializable
  26. class StateTable extends Table with SupportsRead with SupportsMetadataColumns

    An implementation of Table with SupportsRead for State Store data source.

  27. class StreamStreamJoinStatePartitionReader extends PartitionReader[InternalRow] with Logging

    An implementation of PartitionReader for State Store data source, specifically to read the partition for the state from stream-stream join.

  28. class StreamStreamJoinStatePartitionReaderFactory extends PartitionReaderFactory

    An implementation of PartitionReaderFactory for State Store data source, specifically to build a PartitionReader for reading the state from stream-stream join.

Value Members

  1. object StateDataSourceErrors

    Object for grouping error messages from (most) exceptions thrown from State Data Source.

    Object for grouping error messages from (most) exceptions thrown from State Data Source. State Metadata Data Source may (re/co)use this object.

    ERROR_CLASS has a prefix of "STDS_" representing STateDataSource.

  2. object StateSourceOptions extends DataSourceOptions with Serializable
  3. object StateTable

    Companion object for StateTable class to place constants and nested objects.

    Companion object for StateTable class to place constants and nested objects. Currently storing capability of the table and the definition of metadata column(s).

  4. object StreamStreamJoinStateHelper

    A helper object to read the state schema for stream-stream join.

    A helper object to read the state schema for stream-stream join.

    The parameter excludeAuxColumns in methods represents whether the result schema should include the columns the operator added in addition to the input schema.

Ungrouped