pythonCodeSnippet

The python code snippet as a string for evaluating rewards. The following is an example python code snippet that returns a reward 1.0 for a parsed response matching the user-provided reference answer in per prompt references map. `` def evaluate(example, response) -> float: response_str = response.get("parts", [])0 references = example.get("references", {}) if response_str == references.get("concise_answer"): return 1.0 return -1.0 `` Note: Reward output by the evaluate function is clipped to be within [-1, 1], i.e., reward = max(min(reward, 1.0), -1.0).