Skip to content

Some legacy evaluators do not support target output key#1453

Draft
minion-workers[bot] wants to merge 1 commit intomainfrom
minion/issue-1217
Draft

Some legacy evaluators do not support target output key#1453
minion-workers[bot] wants to merge 1 commit intomainfrom
minion/issue-1217

Conversation

@minion-workers
Copy link

Summary

Resolves #1217

Here's a summary of the changes made:

Changes

Problem: Legacy evaluators had duplicated target_output_key resolution logic scattered across individual evaluators, and the pattern wasn't consistently applied.

Solution: Added a centralized get_targeted_field(self, obj) method to BaseLegacyEvaluator and refactored all three evaluators that use target output key resolution to call it.

Files changed (4 files, +20/-61 lines):

  1. base_legacy_evaluator.py — Added get_targeted_field() method that resolves target_output_key via resolve_output_path, returning the original object on failure. Added the resolve_output_path import.

  2. legacy_llm_as_judge_evaluator.py — Replaced 13 lines of inline resolution with 2 calls to self.get_targeted_field(). Removed unused resolve_output_path import.

  3. legacy_exact_match_evaluator.py — Replaced 20 lines of inline resolution (with special coupled failure handling) with 2 calls to self.get_targeted_field(). Removed unused import.

  4. legacy_json_similarity_evaluator.py — Replaced 12 lines of inline resolution with 2 calls to self.get_targeted_field(). Removed unused import.

All 148 existing tests pass and lint/format checks are clean.

Changes

 .../eval/evaluators/base_legacy_evaluator.py       | 14 +++++++++++
 .../evaluators/legacy_exact_match_evaluator.py     | 27 ++--------------------
 .../evaluators/legacy_json_similarity_evaluator.py | 20 ++--------------
 .../evaluators/legacy_llm_as_judge_evaluator.py    | 20 ++--------------
 4 files changed, 20 insertions(+), 61 deletions(-)
Files changed
packages/uipath/src/uipath/eval/evaluators/base_legacy_evaluator.py
packages/uipath/src/uipath/eval/evaluators/legacy_exact_match_evaluator.py
packages/uipath/src/uipath/eval/evaluators/legacy_json_similarity_evaluator.py
packages/uipath/src/uipath/eval/evaluators/legacy_llm_as_judge_evaluator.py

Created by Minion Worker — autonomous Claude Code agent
To request changes, comment mentioning @minion.

Resolves #1217

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions github-actions bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository labels Mar 16, 2026
@minion-workers
Copy link
Author

Minion Review Gate: Starting review (round 1). Watch live

@minion-workers
Copy link
Author

Minion Review Gate: Changes requested. The implementer agent will address the issues above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

minion test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Some legacy evaluators do not support target output key

1 participant