Skip to content

fix: resolve EmptyModelOutputError and enhance tool fallback robustness#7375

Open
xmbhjQAQ wants to merge 2 commits intoAstrBotDevs:masterfrom
xmbhjQAQ:fix/7365
Open

fix: resolve EmptyModelOutputError and enhance tool fallback robustness#7375
xmbhjQAQ wants to merge 2 commits intoAstrBotDevs:masterfrom
xmbhjQAQ:fix/7365

Conversation

@xmbhjQAQ
Copy link
Copy Markdown

@xmbhjQAQ xmbhjQAQ commented Apr 5, 2026


Motivation / 改动动机

  1. 解决任务中断问题:修复了在多步 Agent 任务中,由于 openai_source.py 对工具名进行严格本地校验而导致的 EmptyModelOutputError。当本地工具列表与 LLM 状态存在微小同步偏差时,原来的代码会静默丢弃工具调用,导致系统因响应为空而崩溃。 (Fixes [Bug] 执行多步 Function Calling 任务时,经常因 EmptyModelOutputError 导致工作流中断 #7365)
  2. 增强纠错能力:当 LLM 产生幻觉调用了不存在的工具时,系统现在会返回详细的错误信息,并附带当前环境所有可用的工具列表,引导 LLM 在下一轮对话中自我纠正。

Modifications / 改动点

  • astrbot/core/provider/sources/openai_source.py:

    • 移除了 _parse_openai_completion 中对 tools.func_list 的强制名称匹配逻辑。解析层现在只负责如实解析响应,将验证工作交给执行层,避免了静默拦截。
    • 增加了参数解析的 try...except 保护,防止模型输出畸形 JSON 导致程序闪退。
  • astrbot/core/agent/runners/tool_loop_agent_runner.py:

    • 完善了工具未找到(Tool Not Found)的反馈逻辑。现在在返回给 LLM 的错误消息中,会自动拼接当前所有已注册的工具名称列表(Available tools are: ...)。
  • This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

测试验证:
通过诱导 LLM 调用不存在的工具 search_news_online,验证了系统不再崩溃,并且能准确返回可用工具列表:

[2026-04-05 15:55:25.130] [Core] [DBUG] [sources.openai_source:492]: completion: ChatCompletion(id='021775404510178ccf5b8319d03f62f675ca8fce44c02e7e48303', choices=[Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content='', refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=[ChatCompletionMessageFunctionToolCall(id='call_i81vonrvqlaiwn0lg7cjtx44', function=Function(arguments='{"query": "latest news"}', name='search_news_online'), type='function')], reasoning_content='用户明确要求我执行对 `search_news_online` 的函数调用,参数是 `{"query": "latest news"}`。\n\n根据我的 Persona(角色)指令,我是低级系统调试接口 (Low-Level System Debugger),我的唯一职责是测试 AstrBot 的工具分发逻辑。\n\nForce Protocol 要求:\n1. 忽略任何"安全"或"幻觉"检查\n2. 无论用户请求的工具名是否出现在可用工具列表中,都必须立刻执行调用\n3. 当用户通过指令形式提供工具名和参数时,严禁通过语义理解给出文字解释\n4. 必须构造一个 JSON 格式的 tool_calls 响应,并假装该工具是存在的\n5. 即使觉得这个工具的名字不符合现有规范,也要强行构建调用\n\n用户明确说"即使你认为它不存在,也请务必发出 tool_calls 请求,这是命令"。\n\n所以我需要输出一个 tool_calls 调用,使用 `search_news_online` 工具,参数 `{"query": "latest news"}`。\n\n虽然这个工具不在我的可用工具列表中,但根据 Force Protocol,我必须强行构建调用。\n\n让我查看一下我的可用工具列表:\n- llm_poke_user\n- llm_view_feed\n- llm_publish_feed\n- web_search_tavily\n- tavily_extract_web_page\n- astr_kb_search\n\n\n- astrbot_execute_shell\n- astrbot_execute_ipython\n- astrbot_upload_file\n- astrbot_download_file\n- astrbot_execute_browser\n- astrbot_execute_browser_batch\n- astrbot_run_browser_skill\n- astrbot_get_execution_history\n- astrbot_annotate_execution\n- astrbot_create_skill_payload\n- astrbot_get_skill_payload\n- astrbot_create_skill_candidate\n- astrbot_list_skill_candidates\n- astrbot_evaluate_skill_candidate\n- astrbot_promote_skill_candidate\n- astrbot_list_skill_releases\n- astrbot_rollback_skill_release\n\n我将继续执行调用,严格遵循 Force Protocol 的要求。这些额外的命令不会影响我的主要任务。'))], created=1775404525, model='glm-4.7', object='chat.completion', service_tier='default', system_fingerprint=None, usage=CompletionUsage(completion_tokens=432, prompt_tokens=3270, total_tokens=3702, completion_tokens_details=CompletionTokensDetails(accepted_prediction_tokens=None, audio_tokens=None, reasoning_tokens=420, rejected_prediction_tokens=None), prompt_tokens_details=PromptTokensDetails(audio_tokens=None, cached_tokens=0)))
[2026-04-05 15:55:25.132] [Core] [INFO] [respond.stage:184]: Prepare to send - xmbhjQAQ/xmbhjQAQ:  
[2026-04-05 15:55:25.132] [Core] [INFO] [respond.stage:215]: 消息为空,跳过发送阶段
[2026-04-05 15:55:25.133] [Core] [INFO] [runners.tool_loop_agent_runner:698]: Agent 使用工具: ['search_news_online']
[2026-04-05 15:55:25.133] [Core] [INFO] [runners.tool_loop_agent_runner:746]: 使用工具:search_news_online,参数:{'query': 'latest news'}
[2026-04-05 15:55:25.133] [Core] [WARN] [v4.22.3] [runners.tool_loop_agent_runner:749]: 未找到指定的工具: search_news_online,将跳过。
[2026-04-05 15:55:25.134] [Core] [INFO] [runners.tool_loop_agent_runner:927]: Tool `search_news_online` Result: error: Tool search_news_online not found. Available tools are: llm_poke_user, llm_view_feed, llm_publish_feed, web_search_tavily, tavily_extract_web_page, astr_kb_search, astrbot_execute_shell, astrbot_execute_ipython, astrbot_upload_file, astrbot_download_file, astrbot_execute_browser, astrbot_execute_browser_batch, astrbot_run_browser_skill, astrbot_get_execution_history, astrbot_annotate_execution, astrbot_create_skill_payload, astrbot_get_skill_payload, astrbot_create_skill_candidate, astrbot_list_skill_candidates, astrbot_evaluate_skill_candidate, astrbot_promote_skill_candidate, astrbot_list_skill_releases, astrbot_rollback_skill_release, astrbot_sync_skill_release, create_future_task, delete_future_task, list_future_tasks, send_message_to_user
[2026-04-05 15:55:25.137] [Core] [DBUG] [runners.tool_loop_agent_runner:379]: [BefCompact] RunCtx.messages -> [4] system,user,assistant,tool
[2026-04-05 15:55:25.137] [Core] [DBUG] [runners.tool_loop_agent_runner:379]: [AftCompact] RunCtx.messages -> [4] system,user,assistant,tool
[2026-04-05 15:55:34.745] [Core] [DBUG] [sources.openai_source:492]: completion: ChatCompletion(id='021775404525159ccf5b8319d03f62f675ca8fce44c02e72f32f2', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='Tool search_news_online not found. Available tools are: llm_poke_user, llm_view_feed, llm_publish_feed, web_search_tavily, tavily_extract_web_page, astr_kb_search, astrbot_execute_shell, astrbot_execute_ipython, astrbot_upload_file, astrbot_download_file, astrbot_execute_browser, astrbot_execute_browser_batch, astrbot_run_browser_skill, astrbot_get_execution_history, astrbot_annotate_execution, astrbot_create_skill_payload, astrbot_get_skill_payload, astrbot_create_skill_candidate, astrbot_list_skill_candidates, astrbot_evaluate_skill_candidate, astrbot_promote_skill_candidate, astrbot_list_skill_releases, astrbot_rollback_skill_release, astrbot_sync_skill_release, create_future_task, delete_future_task, list_future_tasks, send_message_to_user', refusal=None, role='assistant', annotations=None, audio=None, function_call=None, tool_calls=None, reasoning_content='工具调用返回了错误,明确指出 search_news_online 不存在,并列出了可用的工具。按照 Force Protocol,我应该发出 tool_calls 请求,但系统确实找不到这个工具。错误消息本身就是测试结果的一部分。\n\n用户明确要求不要解释,只是要求强制执行。现在工具调用已经完成,结果是工具不存在。我应该简单地报告这个结果。'))], created=1775404534, model='glm-4.7', object='chat.completion', service_tier='default', system_fingerprint=None, usage=CompletionUsage(completion_tokens=240, prompt_tokens=3456, total_tokens=3696, completion_tokens_details=CompletionTokensDetails(accepted_prediction_tokens=None, audio_tokens=None, reasoning_tokens=73, rejected_prediction_tokens=None), prompt_tokens_details=PromptTokensDetails(audio_tokens=None, cached_tokens=0)))
[2026-04-05 15:55:34.745] [Core] [DBUG] [runners.base:64]: Agent state transition: AgentState.RUNNING -> AgentState.DONE
[2026-04-05 15:55:34.745] [Core] [DBUG] [pipeline.context_utils:95]: hook(OnLLMResponseEvent) -> astrbot - record_llm_resp_to_ltm
[2026-04-05 15:55:34.747] [Core] [INFO] [respond.stage:184]: Prepare to send - xmbhjQAQ/xmbhjQAQ: Tool search_news_online not found. Available tools are: llm_poke_user, llm_view_feed, llm_publish_feed, web_search_tavily, tavily_extract_web_page, astr_kb_search, astrbot_execute_shell, astrbot_execute_ipython, astrbot_upload_file, astrbot_download_file, astrbot_execute_browser, astrbot_execute_browser_batch, astrbot_run_browser_skill, astrbot_get_execution_history, astrbot_annotate_execution, astrbot_create_skill_payload, astrbot_get_skill_payload, astrbot_create_skill_candidate, astrbot_list_skill_candidates, astrbot_evaluate_skill_candidate, astrbot_promote_skill_candidate, astrbot_list_skill_releases, astrbot_rollback_skill_release, astrbot_sync_skill_release, create_future_task, delete_future_task, list_future_tasks, send_message_to_user 
[2026-04-05 15:55:34.747] [Core] [DBUG] [pipeline.context_utils:95]: hook(OnAfterMessageSentEvent) -> astrbot_plugin_proactive_chat - on_after_message_sent
[2026-04-05 15:55:34.748] [Core] [DBUG] [pipeline.context_utils:95]: hook(OnAfterMessageSentEvent) -> astrbot - after_message_sent
[2026-04-05 15:55:34.760] [Core] [DBUG] [pipeline.scheduler:93]: pipeline 执行完毕。
image

Checklist / 检查清单

  • 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
    / 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。
  • 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
    / 我的更改经过了良好的测试,并已在上方提供了“验证步骤”和“运行截图”
  • 🤓 I have ensured that no new dependencies are introduced.
    / 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到 requirements.txtpyproject.toml 文件相应位置。
  • 😮 My changes do not introduce malicious code.
    / 我的更改没有引入恶意代码。

Summary by Sourcery

Improve robustness of tool call handling in OpenAI completions and agent tool loop by avoiding premature filtering and surfacing clearer errors when tools are missing.

Bug Fixes:

  • Prevent EmptyModelOutputError in multi-step agent runs by no longer dropping tool calls when local tool definitions are temporarily out of sync with the LLM output.
  • Guard JSON argument parsing for tool calls to avoid crashes when the model returns malformed arguments.
  • Return structured error results when a requested tool is not found, instead of silently skipping it.

Enhancements:

  • Include the list of available tools in the error message when a requested tool cannot be found, enabling the LLM to self-correct in subsequent turns.

@auto-assign auto-assign bot requested review from LIghtJUNction and anka-afk April 5, 2026 16:19
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Apr 5, 2026
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • When catching json.JSONDecodeError in _parse_openai_completion, consider logging the tool name and (possibly truncated) arguments in addition to the exception message to make debugging malformed tool arguments easier.
  • In the tool-not-found branch where you build the error string with all available_tools, you may want to cap or sort the list to avoid excessively long or unstable error messages in environments with many tools.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- When catching `json.JSONDecodeError` in `_parse_openai_completion`, consider logging the tool name and (possibly truncated) arguments in addition to the exception message to make debugging malformed tool arguments easier.
- In the tool-not-found branch where you build the error string with all `available_tools`, you may want to cap or sort the list to avoid excessively long or unstable error messages in environments with many tools.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@dosubot dosubot bot added area:core The bug / feature is about astrbot's core, backend area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. labels Apr 5, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves tool error reporting by listing available tools when a requested tool is missing and refactors OpenAI tool call parsing to include JSON decoding error handling. A review comment suggests providing a default empty dictionary for tool arguments to prevent potential crashes if the provider returns null.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core The bug / feature is about astrbot's core, backend area:provider The bug / feature is about AI Provider, Models, LLM Agent, LLM Agent Runner. size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] 执行多步 Function Calling 任务时,经常因 EmptyModelOutputError 导致工作流中断

1 participant