# Long-Context Language Models: Research Digest

> Early work showed models use long inputs unevenly. Current work targets better training recipes, retrieval and memory alternatives, and cutting the serving cost of long inputs.

- **Canonical:** https://agentflare.org/scholar/long-context-language-models-research-digest.html
- **Updated:** 2026-10-16
- **Category:** scholar
- **Full structured data:** `/scholar/long-context-language-models-research-digest.data.json` — $0.02 via AISA HTTP 402 (https://esa.aisa.one/api/v1/access/verify; agents set crawlerAutoPrice=true)

## Key data

- **Field:** long context language models
- **Foundational:** 10
- **Recent (arXiv):** 8
- **Updated:** 2026-10-16

Early work showed models use long inputs unevenly. Current work targets better training recipes, retrieval and memory alternatives, and cutting the serving cost of long inputs.

_…full analysis and the complete dataset are available to agents for $0.02 — fetch `/scholar/long-context-language-models-research-digest.data.json` (HTTP 402)._

## Sources

1. [Lost in the middle: How language models use long contexts](https://aclanthology.org/2024.tacl-1.9/)
2. [Loogle: Can long-context language models understand long contexts?](https://aclanthology.org/2024.acl-long.859/)
3. [A comprehensive survey on long context language modeling](https://arxiv.org/abs/2503.17407)
4. [Chain of agents: Large language models collaborating on long-context tasks](https://proceedings.neurips.cc/paper_files/paper/2024/hash/ee71a4b14ec26710b39ee6be113d7750-Abstract-Conference.html)
5. [How to train long-context language models (effectively)](https://aclanthology.org/2025.acl-long.366/)
6. [Retrieval meets long context large language models](https://proceedings.iclr.cc/paper_files/paper/2024/hash/d75f29006df67df084e6586f1cb8458c-Abstract-Conference.html)
7. [Longlora: Efficient fine-tuning of long-context large language models](https://proceedings.iclr.cc/paper_files/paper/2024/hash/211ab571cc9f3802afa6ffff52ae3e5b-Abstract-Conference.html)
8. [Longalign: A recipe for long context alignment of large language models](https://aclanthology.org/2024.findings-emnlp.74/)
9. [Long-context language modeling with parallel context encoding](https://aclanthology.org/2024.acl-long.142.pdf)
10. [Can long-context language models subsume retrieval, rag, sql, and more?](https://arxiv.org/abs/2406.13121)
11. [Scaling LLM Inference via Parallelism: Throughput-Latency Trade-off](https://arxiv.org/abs/2610.19405v1)
12. [Copy-Paste Language Models](https://arxiv.org/abs/2610.19247v1)
13. [Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules](https://arxiv.org/abs/2610.19210v1)
14. [MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantization](https://arxiv.org/abs/2610.18642v1)
15. [BELT : BEhavior-Aligned Low-Rank KV Cache Compression with Learned Top-k Rank Allocation](https://arxiv.org/abs/2610.17966v1)
16. [Virtual Head Attention](https://arxiv.org/abs/2610.17959v1)
17. [History Replays Itself: Unleashing Long-Context Capability in Linear Attention](https://arxiv.org/abs/2610.15715v1)
18. [Apertus 1.5: From Open Text Models to Open Multimodal Reasoners](https://arxiv.org/abs/2610.15542v1)

## Related

- [LLM Agents & Planning: Literature Digest](https://agentflare.org/scholar/llm-agents-planning-literature-digest.html)
- [Retrieval-Augmented Generation: Research Digest](https://agentflare.org/scholar/retrieval-augmented-generation-research-digest.html)
- [AI Alignment & Safety: Research Digest](https://agentflare.org/scholar/ai-alignment-safety-research-digest.html)
- [RLHF: Research Digest](https://agentflare.org/scholar/rlhf-research-digest.html)
- [Multimodal Foundation Models: Research Digest](https://agentflare.org/scholar/multimodal-foundation-models-research-digest.html)
- [Mechanistic Interpretability: Research Digest](https://agentflare.org/scholar/mechanistic-interpretability-research-digest.html)

---
_Part of AgentFlare, an agent-native data network powered by AISA. https://aisa.one/docs_