object RDDGenerator
- Annotations
- @Experimental()
- Alphabetic
- By Inheritance
- RDDGenerator
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def arbitraryRDD[T](sc: SparkContext, minPartitions: Int = 1)(generator: => Gen[T])(implicit arg0: ClassTag[T]): Arbitrary[RDD[T]]
Generate an RDD of the desired type.
Generate an RDD of the desired type. Attempt to try different number of partitions so as to catch problems with empty partitions, etc.
minPartitions defaults to 1, but when generating data too large for a single machine you must choose a larger value.
- T
The required type for the RDD
- sc
Spark Context
- minPartitions
defaults to 1
- generator
used to create the generator. This function will be used to create the generator as many times as required.
- def arbitrarySizedRDD[T](sc: SparkContext, minPartitions: Int = 1)(generator: (Int) => Gen[T])(implicit arg0: ClassTag[T]): Arbitrary[RDD[T]]
Generate an RDD of the desired type with its size accessible in the lambda.
Generate an RDD of the desired type with its size accessible in the lambda. Attempt to try different number of partitions so as to catch problems with empty partitions, etc.
minPartitions defaults to 1, but when generating data too large for a single machine you must choose a larger value.
- T
The required type for the RDD
- sc
Spark Context
- minPartitions
defaults to 1
- generator
used to create the generator. This function will be used to create the generator as many times as required.
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native() @HotSpotIntrinsicCandidate()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def genRDD[T](sc: SparkContext, minPartitions: Int = 1)(generator: => Gen[T])(implicit arg0: ClassTag[T]): Gen[RDD[T]]
Generate an RDD of the desired type.
Generate an RDD of the desired type. Attempt to try different number of partitions so as to catch problems with empty partitions, etc.
minPartitions defaults to 1, but when generating data too large for a single machine you must choose a larger value.
- T
The required type for the RDD
- sc
Spark Context
- minPartitions
defaults to 1
- generator
used to create the generator. This function will be used to create the generator as many times as required.
- def genSizedRDD[T](sc: SparkContext, minPartitions: Int = 1)(generator: (Int) => Gen[T])(implicit arg0: ClassTag[T]): Gen[RDD[T]]
Generate an RDD of the desired type with its size accessible in the lambda.
Generate an RDD of the desired type with its size accessible in the lambda. Attempt to try different number of partitions so as to catch problems with empty partitions, etc.
minPartitions defaults to 1, but when generating data too large for a single machine you must choose a larger value.
- T
The required type for the RDD
- sc
Spark Context
- minPartitions
defaults to 1
- generator
used to create the generator. This function will be used to create the generator as many times as required.
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated