org.apache.spark.sql.cassandra

CassandraSQLContext

class CassandraSQLContext extends SQLContext

Allows to execute SQL queries against Cassandra and access results as org.apache.spark.sql.SchemaRDD collections. Predicate pushdown to Cassandra is supported.

Example:

import com.datastax.spark.connector._

val sparkMasterHost = "127.0.0.1"
val cassandraHost = "127.0.0.1"

// Tell Spark the address of one Cassandra node:
val conf = new SparkConf(true).set("spark.cassandra.connection.host", cassandraHost)

// Connect to the Spark cluster:
val sc = new SparkContext("spark://" + sparkMasterHost + ":7077", "example", conf)

// Create CassandraSQLContext:
val cc = new CassandraSQLContext(sc)

// Execute SQL query:
val rdd = cc.sql("SELECT * FROM keyspace.table ...")
Linear Supertypes
SQLContext, Serializable, Serializable, UDFRegistration, ExpressionConversions, SQLConf, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. CassandraSQLContext
  2. SQLContext
  3. Serializable
  4. Serializable
  5. UDFRegistration
  6. ExpressionConversions
  7. SQLConf
  8. Logging
  9. AnyRef
  10. Any
Implicitly
  1. by any2stringadd
  2. by any2stringfmt
  3. by any2ArrowAssoc
  4. by any2Ensuring
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CassandraSQLContext(sc: SparkContext)

Type Members

  1. implicit class DslAttribute extends AnyRef

    Definition Classes
    ExpressionConversions
  2. implicit class DslExpression extends ImplicitOperators

    Definition Classes
    ExpressionConversions
  3. implicit class DslString extends ImplicitOperators

    Definition Classes
    ExpressionConversions
  4. implicit class DslSymbol extends ImplicitAttribute

    Definition Classes
    ExpressionConversions
  5. abstract class ImplicitAttribute extends ImplicitOperators

    Definition Classes
    ExpressionConversions
  6. abstract class QueryExecution extends AnyRef

    Attributes
    protected
    Definition Classes
    SQLContext
    Annotations
    @DeveloperApi()
  7. class SparkPlanner extends SparkStrategies

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. def +(other: String): String

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to StringAdd performed by method any2stringadd in scala.Predef.
    Definition Classes
    StringAdd
  5. def ->[B](y: B): (CassandraSQLContext, B)

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to ArrowAssoc[CassandraSQLContext] performed by method any2ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()
  6. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  7. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  8. lazy val analyzer: Analyzer

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  9. def applySchema(rowRDD: RDD[Row], schema: StructType): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @DeveloperApi()
  10. def approxCountDistinct(e: Expression, rsd: Double): ApproxCountDistinct

    Definition Classes
    ExpressionConversions
  11. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  12. def avg(e: Expression): Average

    Definition Classes
    ExpressionConversions
  13. implicit def binaryToLiteral(a: Array[Byte]): Literal

    Definition Classes
    ExpressionConversions
  14. implicit def booleanToLiteral(b: Boolean): Literal

    Definition Classes
    ExpressionConversions
  15. implicit def byteToLiteral(b: Byte): Literal

    Definition Classes
    ExpressionConversions
  16. def cacheTable(tableName: String): Unit

    Definition Classes
    SQLContext
  17. def cassandraSql(cassandraQuery: String): SchemaRDD

    Executes SQL query against Cassandra and returns SchemaRDD representing the result.

  18. lazy val catalog: CassandraCatalog with OverrideCatalog

    A catalyst metadata catalog that points to Cassandra.

    A catalyst metadata catalog that points to Cassandra.

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    CassandraSQLContext → SQLContext
  19. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  20. val conf: SparkConf

  21. def count(e: Expression): Count

    Definition Classes
    ExpressionConversions
  22. def countDistinct(e: Expression*): CountDistinct

    Definition Classes
    ExpressionConversions
  23. def createParquetFile[A <: Product](path: String, allowExisting: Boolean, conf: Configuration)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  24. implicit def createSchemaRDD[A <: Product](rdd: RDD[A])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): SchemaRDD

    Definition Classes
    SQLContext
  25. implicit def decimalToLiteral(d: BigDecimal): Literal

    Definition Classes
    ExpressionConversions
  26. implicit def doubleToLiteral(d: Double): Literal

    Definition Classes
    ExpressionConversions
  27. lazy val emptyResult: RDD[Row]

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  28. def ensuring(cond: (CassandraSQLContext) ⇒ Boolean, msg: ⇒ Any): CassandraSQLContext

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to Ensuring[CassandraSQLContext] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  29. def ensuring(cond: (CassandraSQLContext) ⇒ Boolean): CassandraSQLContext

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to Ensuring[CassandraSQLContext] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  30. def ensuring(cond: Boolean, msg: ⇒ Any): CassandraSQLContext

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to Ensuring[CassandraSQLContext] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  31. def ensuring(cond: Boolean): CassandraSQLContext

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to Ensuring[CassandraSQLContext] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  32. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  33. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  34. def executePlan(plan: LogicalPlan): QueryExecution

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    CassandraSQLContext → SQLContext
  35. def executeSql(sql: String): QueryExecution

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  36. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  37. def first(e: Expression): First

    Definition Classes
    ExpressionConversions
  38. implicit def floatToLiteral(f: Float): Literal

    Definition Classes
    ExpressionConversions
  39. def formatted(fmtstr: String): String

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to StringFormat performed by method any2stringfmt in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @inline()
  40. lazy val functionRegistry: FunctionRegistry

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  41. def getAllConfs: Map[String, String]

    Definition Classes
    SQLConf
  42. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  43. def getConf(key: String, defaultValue: String): String

    Definition Classes
    SQLConf
  44. def getConf(key: String): String

    Definition Classes
    SQLConf
  45. def getKeyspace: String

    Returns keyspace set previously by setKeyspace or throws IllegalStateException if keyspace has not been set yet.

  46. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  47. implicit def intToLiteral(i: Int): Literal

    Definition Classes
    ExpressionConversions
  48. def isCached(tableName: String): Boolean

    Definition Classes
    SQLContext
  49. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  50. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging
  51. def jsonFile(path: String, samplingRatio: Double): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  52. def jsonFile(path: String, schema: StructType): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  53. def jsonFile(path: String): SchemaRDD

    Definition Classes
    SQLContext
  54. def jsonRDD(json: RDD[String], samplingRatio: Double): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  55. def jsonRDD(json: RDD[String], schema: StructType): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @Experimental()
  56. def jsonRDD(json: RDD[String]): SchemaRDD

    Definition Classes
    SQLContext
  57. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging
  58. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  59. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  60. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  61. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  62. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  63. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  64. def logName: String

    Attributes
    protected
    Definition Classes
    Logging
  65. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  66. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  67. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  68. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  69. implicit def logicalPlanToSparkQuery(plan: LogicalPlan): SchemaRDD

    Definition Classes
    SQLContext
    Annotations
    @DeveloperApi()
  70. implicit def longToLiteral(l: Long): Literal

    Definition Classes
    ExpressionConversions
  71. def lower(e: Expression): Lower

    Definition Classes
    ExpressionConversions
  72. def max(e: Expression): Max

    Definition Classes
    ExpressionConversions
  73. def min(e: Expression): Min

    Definition Classes
    ExpressionConversions
  74. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  75. final def notify(): Unit

    Definition Classes
    AnyRef
  76. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  77. val optimizer: Optimizer.type

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  78. def parquetFile(path: String): SchemaRDD

    Definition Classes
    SQLContext
  79. def parseSql(sql: String): LogicalPlan

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  80. val parser: SqlParser

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  81. val planner: SparkPlanner with CassandraStrategies

    Modified Catalyst planner that does Cassandra-specific predicate pushdown

    Modified Catalyst planner that does Cassandra-specific predicate pushdown

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    CassandraSQLContext → SQLContext
  82. val prepareForExecution: RuleExecutor[SparkPlan] { val batches: List[this.Batch] }

    Attributes
    protected[org.apache.spark.sql]
    Definition Classes
    SQLContext
  83. def registerFunction[T](name: String, func: Function22[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  84. def registerFunction[T](name: String, func: Function21[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  85. def registerFunction[T](name: String, func: Function20[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  86. def registerFunction[T](name: String, func: Function19[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  87. def registerFunction[T](name: String, func: Function18[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  88. def registerFunction[T](name: String, func: Function17[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  89. def registerFunction[T](name: String, func: Function16[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  90. def registerFunction[T](name: String, func: Function15[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  91. def registerFunction[T](name: String, func: Function14[_, _, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  92. def registerFunction[T](name: String, func: Function13[_, _, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  93. def registerFunction[T](name: String, func: Function12[_, _, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  94. def registerFunction[T](name: String, func: Function11[_, _, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  95. def registerFunction[T](name: String, func: Function10[_, _, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  96. def registerFunction[T](name: String, func: Function9[_, _, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  97. def registerFunction[T](name: String, func: Function8[_, _, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  98. def registerFunction[T](name: String, func: Function7[_, _, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  99. def registerFunction[T](name: String, func: Function6[_, _, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  100. def registerFunction[T](name: String, func: Function5[_, _, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  101. def registerFunction[T](name: String, func: Function4[_, _, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  102. def registerFunction[T](name: String, func: Function3[_, _, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  103. def registerFunction[T](name: String, func: Function2[_, _, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  104. def registerFunction[T](name: String, func: Function1[_, T])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): Unit

    Definition Classes
    UDFRegistration
  105. def registerRDDAsTable(rdd: SchemaRDD, tableName: String): Unit

    Definition Classes
    SQLContext
  106. def setConf(key: String, value: String): Unit

    Definition Classes
    SQLConf
  107. def setConf(props: Properties): Unit

    Definition Classes
    SQLConf
  108. def setKeyspace(ks: String): Unit

    Sets default Cassandra keyspace to be used when accessing tables with unqualified names.

  109. val settings: Map[String, String]

    Attributes
    protected[org.apache.spark]
    Definition Classes
    SQLConf
  110. implicit def shortToLiteral(s: Short): Literal

    Definition Classes
    ExpressionConversions
  111. val sparkContext: SparkContext

    Definition Classes
    SQLContext
  112. def sql(cassandraQuery: String): SchemaRDD

    Delegates to cassandraSql

    Delegates to cassandraSql

    Definition Classes
    CassandraSQLContext → SQLContext
  113. implicit def stringToLiteral(s: String): Literal

    Definition Classes
    ExpressionConversions
  114. def sum(e: Expression): Sum

    Definition Classes
    ExpressionConversions
  115. def sumDistinct(e: Expression): SumDistinct

    Definition Classes
    ExpressionConversions
  116. implicit def symbolToUnresolvedAttribute(s: Symbol): UnresolvedAttribute

    Definition Classes
    ExpressionConversions
  117. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  118. def table(tableName: String): SchemaRDD

    Definition Classes
    SQLContext
  119. implicit def timestampToLiteral(t: Timestamp): Literal

    Definition Classes
    ExpressionConversions
  120. def toString(): String

    Definition Classes
    AnyRef → Any
  121. def uncacheTable(tableName: String): Unit

    Definition Classes
    SQLContext
  122. def upper(e: Expression): Upper

    Definition Classes
    ExpressionConversions
  123. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  124. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  125. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  126. def →[B](y: B): (CassandraSQLContext, B)

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to ArrowAssoc[CassandraSQLContext] performed by method any2ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc

Shadowed Implicit Value Members

  1. val self: Any

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to StringAdd performed by method any2stringadd in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (cassandraSQLContext: StringAdd).self
    Definition Classes
    StringAdd
  2. val self: Any

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to StringFormat performed by method any2stringfmt in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (cassandraSQLContext: StringFormat).self
    Definition Classes
    StringFormat

Deprecated Value Members

  1. def x: CassandraSQLContext

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to ArrowAssoc[CassandraSQLContext] performed by method any2ArrowAssoc in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (cassandraSQLContext: ArrowAssoc[CassandraSQLContext]).x
    Definition Classes
    ArrowAssoc
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) Use leftOfArrow instead

  2. def x: CassandraSQLContext

    Implicit information
    This member is added by an implicit conversion from CassandraSQLContext to Ensuring[CassandraSQLContext] performed by method any2Ensuring in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (cassandraSQLContext: Ensuring[CassandraSQLContext]).x
    Definition Classes
    Ensuring
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) Use resultOfEnsuring instead

Inherited from SQLContext

Inherited from Serializable

Inherited from Serializable

Inherited from UDFRegistration

Inherited from ExpressionConversions

Inherited from SQLConf

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion any2stringadd from CassandraSQLContext to StringAdd

Inherited by implicit conversion any2stringfmt from CassandraSQLContext to StringFormat

Inherited by implicit conversion any2ArrowAssoc from CassandraSQLContext to ArrowAssoc[CassandraSQLContext]

Inherited by implicit conversion any2Ensuring from CassandraSQLContext to Ensuring[CassandraSQLContext]

Ungrouped