**Original paper:** [https://arxiv.org/abs/2608.12435v1](https://arxiv.org/abs/2608.12435v1)

# Summary: 2026-08-12_13-45-01Z_MARCH_ScalingRecurrentMemorywithContent_RoutedStat.md
Saved: 2026-08-13 22:24
Source: 2026-08-12_13-45-01Z_MARCH_ScalingRecurrentMemorywithContent_RoutedStat.md
Model: None

---

## Summary  
The paper introduces MARCH (Memory-Anchor Routing across Context History), a novel architecture designed to scale recurrent memory beyond fixed-size dimensions while maintaining computational efficiency over long sequences. By periodically caching cumulative recurrent-state checkpoints as state anchors and associating each with a compact, content-conditioned anchor key, MARCH enables a growing memory bank that adapts to increasing context lengths without sacrificing performance. The method preserves the native computation path of recurrent models by using attention-style aggregation over historical anchors at each token, effectively combining the strengths of transformers and recurrent networks. This approach addresses the trade-off between long-term recall and memory cost inherent in both transformer caches and fixed-state recurrent models.

## Key Contributions  
- [Finding 1] MARCH enables scalable state-space models by dynamically growing a memory bank through content-routed state anchors, eliminating the need for quadratic attention or linear key-value cache growth.  
- [Finding 2] The system maintains high recall on long-range dependencies by periodically storing cumulative recurrent states as checkpoints, ensuring earlier information is not lost despite autoregressive updates.  
- [Finding 3] Content-routed anchor keys allow fine-grained retrieval of specific historical states based on token-level content, improving the relevance and efficiency of memory access.

## Methodology  
MARCH operates by periodically generating state anchors that represent cumulative recurrent-state checkpoints across the context history. Each anchor is associated with a compact key derived from the current token’s content, enabling content-conditioned routing. At each new token, MARCH computes an anchor query to attend all causally available state anchors, and aggregates their outputs using attention-style summation. This allows the model to retrieve relevant historical information proportionally to its relevance, while only storing necessary checkpoints in memory. The architecture maintains a linear or sub-linear cost relative to context length, unlike transformers’ O(n²) attention or fixed-state models’ limited recall.

## Results  
After standard pretraining, MARCH consistently outperforms multiple linear attention variants across commonsense reasoning (e.g., Winograde), LongBench (long-context benchmark), and in-context retrieval tasks. The model achieves state-of-the-art results on these benchmarks while using significantly less memory than transformer-based alternatives with comparable context lengths. Ablation studies confirm that the content-routed anchor mechanism is critical for performance, as removing it degrades recall and efficiency. MARCH also demonstrates strong generalization across diverse tasks requiring long-term contextual understanding.

## Significance  
MARCH bridges a key limitation in recurrent models—fixed-size state—that hampers long-range memory—with the scalability challenge of transformers—quadratic attention cost. By introducing content-routed state anchors, it offers a principled solution that scales with context length while preserving efficiency and recall. This work advances the field by redefining how recurrent architectures can be extended beyond their traditional constraints, paving the way for more powerful and memory-efficient long-context AI systems.

## Related Concepts  
- State-space models  
- Recurrent neural networks (RNNs)  
- Transformers with attention mechanisms  
- Key-value caches in autoregressive decoding  
- Memory banks and checkpoints  
- Content-conditioned routing  
- Long-context retrieval