Package org.elasticsearch.index.rankeval

Class PrecisionAtK

java.lang.Object
org.elasticsearch.index.rankeval.PrecisionAtK
All Implemented Interfaces:
org.elasticsearch.common.io.stream.NamedWriteable, org.elasticsearch.common.io.stream.Writeable, org.elasticsearch.common.xcontent.ToXContent, org.elasticsearch.common.xcontent.ToXContentObject, EvaluationMetric
public class PrecisionAtK
extends java.lang.Object
implements EvaluationMetric
Metric implementing Precision@K (https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Precision).
By default documents with a rating equal or bigger than 1 are considered to be "relevant" for this calculation. This value can be changed using the `relevant_rating_threshold` parameter.
The `ignore_unlabeled` parameter (default to false) controls if unrated documents should be ignored. The `k` parameter (defaults to 10) controls the search window size.

  • Nested Class Summary

    Nested Classes
    Modifier and Type Class Description
    static class  PrecisionAtK.Detail  

    Nested classes/interfaces inherited from interface org.elasticsearch.common.xcontent.ToXContent

    org.elasticsearch.common.xcontent.ToXContent.DelegatingMapParams, org.elasticsearch.common.xcontent.ToXContent.MapParams, org.elasticsearch.common.xcontent.ToXContent.Params

    Nested classes/interfaces inherited from interface org.elasticsearch.common.io.stream.Writeable

    org.elasticsearch.common.io.stream.Writeable.Reader<V extends java.lang.Object>, org.elasticsearch.common.io.stream.Writeable.Writer<V extends java.lang.Object>

  • Field Summary

    Fields
    Modifier and Type Field Description
    static java.lang.String NAME  

    Fields inherited from interface org.elasticsearch.common.xcontent.ToXContent

    EMPTY_PARAMS

  • Constructor Summary

    Constructors
    Constructor Description
    PrecisionAtK()  
    PrecisionAtK​(boolean ignoreUnlabeled)  
    PrecisionAtK​(int relevantRatingThreshold, boolean ignoreUnlabeled, int k)
    Metric implementing Precision@K.

  • Method Summary

    Modifier and Type Method Description
    boolean equals​(java.lang.Object obj)  
    EvalQueryQuality evaluate​(java.lang.String taskId, org.elasticsearch.search.SearchHit[] hits, java.util.List<RatedDocument> ratedDocs)
    Compute precision at k based on provided relevant document IDs.
    java.util.OptionalInt forcedSearchSize()
    Metrics can define a size of the search hits windows they want to retrieve by overwriting this method.
    static PrecisionAtK fromXContent​(org.elasticsearch.common.xcontent.XContentParser parser)  
    boolean getIgnoreUnlabeled()
    Gets the 'ignore_unlabeled' parameter.
    int getK()  
    int getRelevantRatingThreshold()
    Return the rating threshold above which ratings are considered to be "relevant" for this metric.
    java.lang.String getWriteableName()  
    int hashCode()  
    org.elasticsearch.common.xcontent.XContentBuilder toXContent​(org.elasticsearch.common.xcontent.XContentBuilder builder, org.elasticsearch.common.xcontent.ToXContent.Params params)  
    void writeTo​(org.elasticsearch.common.io.stream.StreamOutput out)  

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.elasticsearch.index.rankeval.EvaluationMetric

    combine

    Methods inherited from interface org.elasticsearch.common.xcontent.ToXContentObject

    isFragment

  • Field Details

  • Constructor Details

    • PrecisionAtK

      public PrecisionAtK​(int relevantRatingThreshold, boolean ignoreUnlabeled, int k)
      Metric implementing Precision@K.
      Parameters:
      relevantRatingThreshold - ratings equal or above this value will be considered relevant.
      ignoreUnlabeled - Controls how unlabeled documents in the search hits are treated. Set to 'true', unlabeled documents are ignored and neither count as true or false positives. Set to 'false', they are treated as false positives.
      k - controls the window size for the search results the metric takes into account

    • PrecisionAtK

      public PrecisionAtK​(boolean ignoreUnlabeled)

    • PrecisionAtK

      public PrecisionAtK()

  • Method Details

    • fromXContent

      public static PrecisionAtK fromXContent​(org.elasticsearch.common.xcontent.XContentParser parser)

    • writeTo

      public void writeTo​(org.elasticsearch.common.io.stream.StreamOutput out) throws java.io.IOException
      Specified by:
      writeTo in interface org.elasticsearch.common.io.stream.Writeable
      Throws:
      java.io.IOException

    • toXContent

      public org.elasticsearch.common.xcontent.XContentBuilder toXContent​(org.elasticsearch.common.xcontent.XContentBuilder builder, org.elasticsearch.common.xcontent.ToXContent.Params params) throws java.io.IOException
      Specified by:
      toXContent in interface org.elasticsearch.common.xcontent.ToXContent
      Throws:
      java.io.IOException

    • getWriteableName

      public java.lang.String getWriteableName()
      Specified by:
      getWriteableName in interface org.elasticsearch.common.io.stream.NamedWriteable

    • getRelevantRatingThreshold

      public int getRelevantRatingThreshold()
      Return the rating threshold above which ratings are considered to be "relevant" for this metric. Defaults to 1.

    • getIgnoreUnlabeled

      public boolean getIgnoreUnlabeled()
      Gets the 'ignore_unlabeled' parameter.

    • getK

      public int getK()

    • forcedSearchSize

      public java.util.OptionalInt forcedSearchSize()
      Description copied from interface: EvaluationMetric
      Metrics can define a size of the search hits windows they want to retrieve by overwriting this method. The default implementation returns an empty optional.
      Specified by:
      forcedSearchSize in interface EvaluationMetric
      Returns:
      the number of search hits this metrics requests

    • evaluate

      public EvalQueryQuality evaluate​(java.lang.String taskId, org.elasticsearch.search.SearchHit[] hits, java.util.List<RatedDocument> ratedDocs)
      Compute precision at k based on provided relevant document IDs.
      Specified by:
      evaluate in interface EvaluationMetric
      Parameters:
      taskId - an identifier of the query for which the search ranking is evaluated
      hits - the search result hits
      ratedDocs - the documents that contain the document rating for this query case
      Returns:
      precision at k for above SearchResult list.

    • equals

      public final boolean equals​(java.lang.Object obj)
      Overrides:
      equals in class java.lang.Object

    • hashCode

      public final int hashCode()
      Overrides:
      hashCode in class java.lang.Object