Robuta

Retrieval Augmented Generation (RAG) has emerged as a crucial technique for enhancing the accuracy of Large Language Models (LLMs) by incorporating external...

Wenhan Xiong, Barlas Oguz, Anchit Gupta, Xilun Chen, Diana Liskovich, Omer Levy, Scott Yih, Yashar Mehdad. Proceedings of the 2022 Conference of the North...

SmolLM3 is a powerful 3B parameter language model designed for efficient reasoning, long context understanding, and multilingual applications. Explore the...

Abstract page for arXiv paper 2406.10149: BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack

Acoustic processing requires integration over time. We have used in vivo intracellular recording to measure neuronal integration times in anesthetized rats....

During the inference phase of Large Language Models (LLMs) with long context, a substantial portion of GPU memory is allocated to the KV cache, with memory...

During the inference phase of Large Language Models (LLMs) with long context, a substantial portion of GPU memory is allocated to the KV cache, with memory...

French AI company Mistral has launched Codestral, a new coding model that delivers high coding performance with less computational overhead than existing...

Aligning large language models to handle instructions with extremely long contexts has yet to be fully investigated. Previous studies attempt to scale up the...

We present an approximate attention mechanism named `HyperAttention` to address the computational challenges posed by the growing complexity of long contexts...

Gefei Gu, Yilun Zhao, Ruoxi Ning, Yanan Zheng, Arman Cohan. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System...

The rapid ageing of populations around the world is creating complex challenges for national governments.

Move AI memory across platforms with Context Pack. Create portable memory packs from ChatGPT, Claude, and Gemini chats. Transfer your AI conversation history...

Apr 12, 2025 - New independent evaluations reveal that Meta's latest Llama 4 models - Maverick and Scout - perform well in standard tests but struggle with complex...

A Guide to Improving Long Context Instruction Following on Open Source Models

Join the discussion on this paper page

Abstract page for arXiv paper 2405.15318: Are Long-LLMs A Necessity For Long-Context Tasks?

Recent coverage of the grassroots uprising against the Taliban in Kandahar province fails to account for a long-running tribal feud.

PERK: Long-Context Reasoning as Test-Time Learning

LLM 0.24 is now available with new features to help take advantage of the increasingly long input context supported by modern LLMs. (LLM is my command-line...

NAEP - 2012 Long-Term Trend: Classroom Context: Coursetaking in Mathematics

Understanding documents with rich layouts and multi-modal components is a long-standing and practical task. Recent Large Vision-Language Models (LVLMs) have...

Processing structured tabular data, particularly large and lengthy tables, constitutes a fundamental yet challenging task for large language models (LLMs)....

Join the discussion on this paper page

State-of-the-art large language models (LLMs) are now claiming remarkable supported context lengths of 256k or even more. In contrast, the average context...

In this paper, we present a benchmark to pressure-test today's frontier models' multimodal decision-making capabilities in the very long-context regime (up to...

Though current long-context large language models (LLMs) have demonstrated impressive capacities in answering user questions based on extensive text, the lack...

The scaling of inference computation has unlocked the potential of long-context large language models (LLMs) across diverse settings. For knowledge-intensive...

Abstract page for arXiv paper 2503.01996: One ruler to measure them all: Benchmarking multilingual long-context language models

Abstract page for arXiv paper 2502.01549: VideoRAG: Retrieval-Augmented Generation with Extreme Long-Context Videos

Recent advances in long-context large language models (LLMs) make them commercially viable, but their standard attention mechanisms' quadratic complexity...

State-of-the-art large language models (LLMs) are now claiming remarkable supported context lengths of 256k or even more. In contrast, the average context...

As generative artificial intelligence (AI) applications become more prevalent, maintaining responsible AI principles becomes essential. Without proper...