Type Members

1.  abstract class FeatureDerivationFunctionSpark extends FeatureDerivationFunctionBase

   Spark DataFrame based derived feature base class, Feathr user can extend this class to implement their customized derivation function to produce derived features

   Spark DataFrame based derived feature base class, Feathr user can extend this class to implement their customized derivation function to produce derived features

   see TestDataFrameDerivationFunctionExtractor in feathr repo for example

2.  abstract class OutputProcessor extends Serializable

   base class for a post processor in feature data generation User can apply customized operation on the feature data, e.g, write to disk/console see example in feathr repo, e.g, WriteToConsolePostProcessor, WriteToHDFSPostProcessor

3.  abstract class SeqJoinCustomAggregation extends Serializable
4.  abstract class SimpleAnchorExtractorSpark extends AnchorExtractorBase[Any]

   Spark DataFrame-based simple anchor extractor.

   Spark DataFrame-based simple anchor extractor.

   We strongly recommend developer extends this SimpleAnchorExtractorSpark trait (when SQL based syntax is not able to express the transformation logic) to implement customized transformation logic, instead of extends GenericAnchorExtractorSpark, As this trait is more efficient than GenericAnchorExtractorSpark in Feathr. (less expensive join between observation and transformed feature data), as SimpleAnchorExtractorSpark guarantees that all rows/columns in the input DataFrame will be "passed through" into the returned DataFrame which helps to avoid the number of join in Feathr, thanks to the fact that Feathr uses the output dataframe of SimpleAnchorExtractorSpark to apply other feature anchors (as long as the anchors share same source and keyExtractor) to improve performance significantly.

   see SQLConfigurableAnchorExtractor, TestxFeatureDataExtractorV2, SimpleConfigurableAnchorExtractor in feathr repo for example

5.  abstract class SourceKeyExtractor extends Serializable

   This is the base class for user customized key extractor for a source (feature dataset, rather than observation dataset) in an anchor, it is used in the 'keyExtractor' of an anchor, 'keyExtractor' is a replacement for 'key' section in the anchor.

   This is the base class for user customized key extractor for a source (feature dataset, rather than observation dataset) in an anchor, it is used in the 'keyExtractor' of an anchor, 'keyExtractor' is a replacement for 'key' section in the anchor. Note that, though this is called 'keyExtractor', it is actually a general purpose interface used to generate the fact table that will be passed to anchorExtractors later. Arbitrary transformations are allowed in appendKeyColumns(), only requirement is that it generates the key columns in the output dataframe

   For example, swaAnchorWithKeyExtractor: { source: "swaSource" keyExtractor: "com.linkedin.feathr.offline.SimpleSampleKeyExtractor" features: { f: { def: "aggregationWindow" aggregation: SUM window: 3d } } }

   We provide a few default source key extractors, such as MVELSourceKeyExtractor, SQLSourceKeyExtractor, SpecificRecordSourceKeyExtractor in repo 'feathr'. So internally, Feathr will create one of these source key extractors, if 'key' section is being used instead of 'keyExtractor' in the anchor.

   There're also some simple examples such as SimpleSampleKeyExtractor, SimpleSampleKeyExtractor2 in repo 'feathr'