Reinforcement
Scores autorater responses by using exact string match reward scorer. This data type is not supported in Gemini API.
Properties
Assigns this reward score if the parsed response string equals the expression.
The string expression to match against for scoring. This field supports placeholders in the format of {{references.key}} that will be replaced before matching. Regex is not supported for this expression. For example, users can define an ExactMatchScorer as follows: { "correctAnswerReward": 1.0, "wrongAnswerReward": -1.0, "expression": "{{references.concise_answer}}" } When evaluating the reward for each parsed autorater response, if the prompt references in the training/validation dataset has the following fields: `` { "example": ..., "references": { "concise_ansser": "Yes", "verbose_answer": "The answer is Yes" } } `` The above ExactMatchScorer will be replaced as follows for scoring: `` { "correctAnswerReward": 1.0, "wrongAnswerReward": -1.0, "expression": "Yes" } `` If the parsed autorater response is equal to the string "Yes", then the reward is 1.0, otherwise the reward is -1.0.
Assigns this reward score if the parsed reward value does not equal the expression.