Packages

class FeathrClient extends AnyRef

FeathrClient is the entry point into Feathr for joining observation data with features. To achieve this, instantiate this class using the FeathrClient#getFeathrClientWithConfStrings or FeathrClient#getFeathrClientWithConfFile, then use either joinFeatures or joinFeatureAsRdd method.

The FeathrClient takes in a FeatureGroups object, which can be created from the featureDefConf files using the FeatureGroupsGenerator class.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. FeathrClient
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. type KeyTagStringTuple = Seq[String]

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. def generateFeatures(featureGenSpec: FeatureGenSpec): Map[TaggedFeatureName, SparkFeaturizedDataset]

    Generates features by extracting feature data from its source.

    Generates features by extracting feature data from its source. It returns generated features, the DataFrame for feature data and the feature related metadata. The API takes in FeatureGenSpec as the input, which is expected to contain all the necessary information for generating features.

    The FeathrClient does the planning and hands over the execution to DataFrameFeatureGenerator.

    featureGenSpec

    feature generation specification

    returns

    generated feature data as SparkFeaturizedDataset, if having output processor, return the merge processed feature data of the all processors otherwise, return the unprocessed data

    Annotations
    @InternalApi()
  10. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. def joinFeatures(joinConfig: FeatureJoinConfig, obsData: SparkFeaturizedDataset, jobContext: JoinJobContext = JoinJobContext()): SparkFeaturizedDataset

    Joins observation data on the feature data.

    Joins observation data on the feature data. Observation data is loaded as SparkFeaturizedDataset, and the joined data is returned as a SparkFeaturizedDataset.

    This API takes feathr's internal feature-join.md instead of the PDL config, and it will be only used inside feathr, to support LocalJoinJob, which are used by other repo for testing. We should remove it once the users can migrate to use the PDL config.

    The feature values in the DataFrame are represented as tensors. Values are represented as struct of arrays which is a columnar representation where each dimension and tensor value is in its own column. The schema in the latter case is the one specified for tensors avro-schemas repo for details).

    joinConfig

    feathr offline's FeatureJoinConfig

    obsData

    Observation data taken in as a SparkFeaturizedDataset.

    jobContext

    JoinJobContext

    returns

    Joined observation and feature data as a SparkFeaturizedDataset.

    Annotations
    @InternalApi()
  14. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  18. def toString(): String
    Definition Classes
    AnyRef → Any
  19. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  20. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped