Skip to content

Some legacy evaluators do not support target output key #1217

@akshaylive

Description

@akshaylive

... such as LLM as a judge evaluator. Please fix.

Please refer to LegacyExactMatchEvaluator for a potential fix.

My suggestion is to create a def get_targeted_field(self, obj: Any) -> Any: in LegacyBaseEvaluator. Then, modify all evaluators to perform

actual_output = self.get_targeted_field(agent_execution.agent_output)
expected_output = self.get_targeted_field(evaluation_criteria.expected_output)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions