---
title: "Summary: Measuring and Reducing Unusable Replies of LLM Agents"
date: "2026-10-14"
type: paper-summary
tags: [ai, paper, agents, evaluation, usability]
source_paper: "http://arxiv.org/abs/2610.19659v1"
---
# Summary: Measuring and Reducing Unusable Replies of LLM Agents

Saved: 2026-10-14 22:35
Source: 2026-10-14_06-35-49Z_MeasuringandReducingUnusableRepliesofLLMAgents.md
Model: None

---

## Summary
This paper introduces the concept of "reply usability" to address a critical gap in current Large Language Model (LLM) agent evaluations, which typically focus on task completion rather than the requester's ability to understand and act upon the agent's output. The authors propose a novel metric using a "reader model" that simulates the requester's perspective to determine if a reply is interpretable, linguistically appropriate, and directly responsive to the request. By evaluating eight LLM models across seven families, the study reveals that a significant portion of agent replies are unusable, primarily due to references to unseen information or internal agent jargon. The paper demonstrates that explicit prompting and post-processing rewriting significantly reduce these failures, offering practical strategies to improve the reliability of LLM agents in real-world applications.

## Key Contributions
- **Definition of Reply Usability:** The authors formally define "reply usability" as the extent to which a requester can interpret an agent's reply based solely on information previously seen by the requester, ensuring the reply is in the correct language and directly answers the prompt.
- **Quantification of Failure Rates:** Through extensive testing, the study finds that 54% of 480 replies from eight different LLM models fail the usability test, with even the best-performing model failing 30% of the time, highlighting a widespread issue in current agent systems.
- **Identification of Failure Modes:** The research identifies that failed replies predominantly stem from two issues: referencing material the requester has not seen (context leakage) and using internal labels or jargon coined by the agent during its reasoning process.
- **Mitigation Strategies:** The paper demonstrates two effective mitigation techniques: appending explicit requirements to the prompt reduces failure rates from 52% to 43%, while using a secondary model to rewrite the reply further reduces failures to 36%, a result validated by human annotators.

## Methodology
The authors developed a "reader model" designed to simulate the position of a human requester. This model evaluates agent replies by checking three criteria: whether the reply can be interpreted using only the information the requester has already seen, whether it is in the requester's language, and whether it directly answers the original request. The validity of this automated metric was established by comparing its judgments against those of five human annotators, showing that the model's agreement with the majority of annotators falls within the range of agreement between individual annotators. The study then applied this metric to 480 replies generated by eight LLM models from seven different families. Finally, the authors tested mitigation strategies by modifying prompts to include explicit reply requirements and by implementing a secondary model to rewrite agent outputs before presenting them to the reader model.

## Results
The experimental results indicate a high prevalence of unusable replies across diverse LLM architectures. Specifically, 54% of the 480 tested replies failed the usability criteria, with the best-performing model still exhibiting a 30% failure rate. The analysis of these failures revealed that agents frequently rely on internal reasoning steps or intermediate data structures that are invisible to the user, leading to confusing or uninterpretable outputs. When the authors introduced explicit instructions in the prompt regarding how the reply should be structured and what information it should contain, the failure rate dropped to 43%. Furthermore, employing a secondary model to rewrite the agent's initial output into a user-friendly format reduced the failure rate to 36%. These improvements were confirmed through blind comparisons with human annotators, validating that the automated metrics align with human perceptions of usability.

## Significance
This research is significant because it shifts the evaluation paradigm for LLM agents from purely functional task completion to user-centric interpretability. As agents are increasingly delegated complex, multi-step tasks, the utility of the final output depends heavily on the user's ability to understand and trust the reply. By quantifying "reply usability" and identifying specific failure modes like context leakage and internal jargon, the paper provides actionable insights for developers. The proposed mitigation strategies—explicit prompting and post-processing rewriting—offer low-cost, effective methods to enhance agent reliability, ensuring that AI systems are not just capable but also communicative and accessible to human users.

## Related Concepts
- LLM Agents
- Reply Usability
- Reader Model
- Task Completion vs. Interpretability
- Context Leakage
- Internal Jargon/Labels
- Prompt Engineering
- Post-processing Rewriting
- Human-in-the-loop Evaluation