Retrieval-augmented generation (RAG) has emerged as a solution to the limited knowledge horizon of large language models (LLMs). RAG combines “pre-trained parametric and non-parametric memory for language generation,” with the non-parametric memory typically retrieved from large collections of documents. RAG has been shown to dramatically improve the performance of LLMs on various question-answering and reasoning tasks. However, we argue that RAG often overwhelms the LLM with too much information, only some of which may be relevant to the task at hand. This contradicts Grice’s four maxims of effective communication, which state that the information provided should be “as much as needed, and no more” and that it should be “as clear, as brief” as possible. While these maxims were originally formulated in the context of human communication, we argue that they are also applicable in a RAG setting. We propose a simple, unsupervised method that injects pragmatic principles in retrieval augmented generation frameworks such as Dense Passage Retrieval (DPR) to enhance the utility of retrieved contexts. Our approach first identifies which sentences in a pool of documents retrieved by RAG are most relevant to the question at hand, cover all the topics addressed in the input question and no more, and then highlights these sentences within their context, before they are provided to the LLM, without truncating or altering the context in any other way. We show that this simple idea brings consistent improvements in experiments on three question answering tasks (ARC-Challenge, PubHealth and PopQA) using five different LLMs. It notably enhances relative accuracy by up to 19.7% on PubHealth and 10% on ARC-Challenge compared to a conventional RAG system.

Paper arxiv link: https://arxiv.org/abs/2502.17839