expression

The string expression to match against for scoring. This field supports placeholders in the format of {{references.key}} that will be replaced before matching. Regex is not supported for this expression. For example, users can define an ExactMatchScorer as follows: { "correctAnswerReward": 1.0, "wrongAnswerReward": -1.0, "expression": "{{references.concise_answer}}" } When evaluating the reward for each parsed autorater response, if the prompt references in the training/validation dataset has the following fields: `` { "example": ..., "references": { "concise_ansser": "Yes", "verbose_answer": "The answer is Yes" } } `` The above ExactMatchScorer will be replaced as follows for scoring: `` { "correctAnswerReward": 1.0, "wrongAnswerReward": -1.0, "expression": "Yes" } `` If the parsed autorater response is equal to the string "Yes", then the reward is 1.0, otherwise the reward is -1.0.