# Summary: 2026-07-26_03-39-29Z_ReasoningorMemorization_CanLLMsUnderstandandGenera.md
Saved: 2026-07-27 20:14
Source: 2026-07-26_03-39-29Z_ReasoningorMemorization_CanLLMsUnderstandandGenera.md
Model: None

---

## Summary  
The paper investigates whether large language models can understand and generate Chinese xiehouyu riddles, which are traditional wordplay games that rely on subtle linguistic patterns. It creates novel xiehouyu to avoid data contamination and evaluates LLMs using multiple‑choice questions, free‑form explanations, and generation tasks. Findings reveal that frontier Chinese models memorize many low‑frequency riddles better than English‑centric ones, while human‑like reasoning remains limited in creation. The study suggests caution about attributing reasoning abilities due to data bias.

## Key Contributions  
- [Finding 1] Frontier Chinese LLMs achieve a mean delta accuracy of 23.6 % on MCQs for low‑frequency xiehouyu, indicating higher memorization than English‑centric models (5.1 %).  
- [Finding 2] For novel xiehouyu, Gemini 3.1 Pro reaches 92.6 % accuracy, surpassing the human average by roughly 24 %, suggesting strong pattern recognition.  
- [Finding 3] LLM‑generated xiehouyu receive lower human ratings for creativity than those created by humans, indicating limited creative fluency.

## Methodology  
The authors designed a set of existing low‑frequency xiehouyu and newly invented ones to test memorization versus reasoning. They measured delta accuracy between native speakers’ performance on existing vs novel riddles as a proxy for memorization. LLMs were evaluated using MCQs, free‑form explanations, and generation tasks; human participants rated the creativity of generated riddles.

## Results  
Frontier Chinese models had 23.6 % delta accuracy, while English‑centric models scored 5.1 %. Gemini 3.1 Pro generated novel xiehouyu with 92.6 % accuracy (human average ~78 %). LLM‑generated riddles were rated lower on creativity metrics than human‑created ones.

## Significance  
The study highlights that high‑performing Chinese LLMs may rely heavily on memorization rather than genuine reasoning, and that current claims of reasoning in language tasks must consider data biases. It also underscores the gap between model performance and human creative fluency.

## Related Concepts  
Xiehouyu (Chinese riddles), delta accuracy, low‑frequency word processing, large language models, creativity evaluation, memorization vs reasoning.