# Summary: 2026-08-06_14-04-17Z_WhenHistoryLies_EvaluatingandImprovingToolUseunder.md
Saved: 2026-08-06 20:45
Source: 2026-08-06_14-04-17Z_WhenHistoryLies_EvaluatingandImprovingToolUseunder.md
Model: None

---

## Summary  
Tool‑calling agents must infer the current task state from a long dialogue and its tool traces. When those traces become structurally valid but no longer authoritative—i.e., “history lies”—the model can make decisions that are correct under the original trajectory yet harmful in practice. The authors demonstrate this problem on Qwen3‑1.7B, where polluted histories cause 32 % of otherwise correct actions to flip and lead to entity reuse or interface misuse. Their contribution is a new benchmark (bench) together with a teacher‑student transfer method that restores reliable state policy without relying on clean history.

## Key Contributions  
- [Finding 1] Historical traces can hijack a model’s policy, causing incorrect decisions even when the original trajectory was correct.  
- [Finding 2] The bench benchmark isolates three failure modes—decision‑state errors, entity binding failures, and interface execution errors—by providing synchronized Original, Polluted, and Oracle State views with gold‑preserving interventions.  
- [Finding 3] A soft‑supervision teacher‑policy transfer raises the student’s Balanced Tool‑Use Accuracy from 66.3 % (Gold‑SFT) to 87.0 %, scaling consistently to larger models.

## Methodology  
The authors construct a paired benchmark where each interaction is presented in three synchronized views: the original clean history, a polluted version that retains structural validity but introduces misleading information, and an oracle view that reflects the true next action. Gold‑preserving interventions keep the system policy, current tools, latest request, and gold answer unchanged across these views. To train a student model, they use soft supervision on prefixes generated by the student, conditioning it on the teacher’s Oracle‑conditioned policy without ever exposing clean history.

## Results  
On Qwen3‑1.7B, the student achieves 87.0 % Balanced Tool‑Use Accuracy, outperforming Gold‑SFT (66.3 %), Oracle sequence distillation (82.3 %) and off‑policy token distillation (85.0 %). Scaling to larger models yields consistent gains: an 8B teacher lifts the 1.7B student to 91.9 %, while an 8B student reaches 93.0 %. The resulting policies also transfer to clean histories, unseen functions, external tool‑use benchmarks, and noisy multi‑hop QA tasks.

## Significance  
These findings establish history reliability as a distinct bottleneck in tool use and demonstrate that reliable‑state policy transfer is an effective, scalable solution. By decoupling the student from polluted histories through teacher supervision, agents can maintain correct behavior even when accumulated dialogue contains misleading traces.

## Related Concepts  
tool‑calling agents, accumulated dialogue, historical traces, state hijacking, oracle‑conditioned teacher policy, soft supervision, Balanced Tool‑Use Accuracy, multi‑turn histories, entity binding, interface execution.