Why RAG Answers Fall Short and How Query Enrichment Can Help



❓Why do the users of RAG describe the answers as ‘correct but not useful’? How can the RAG pipeline be improved to avoid this?

😒 RAG user: ‘this answer is good, but it is not useful to me. It should have also mentioned a few more things’.

🤔 RAG Developer: ‘this looks perfectly good to me. I am myself amazed how my program is able to produce such a beautiful answer’.

One will encounter many such interactions while putting RAG into production. During the prototyping, when RAG starts producing correct answers, there is a feeling of awe and surprise. But when the actual users start using it for day-to-day use, they find the answers very basic. They say that they can find these answers reasonably quickly without the tool.

Why does this happen? 🤨

In my previous posts, I described how RAG works. It basically has three parts:

1️⃣Retrieve the material relevant to users' query.
2️⃣Augment the prompt with the material.
3️⃣Generate the answer using an LLM.

Each of the above parts can be improved to get better answers. Today we will see just one of the tricks: query enrichment 💸.

The query posed by the users is generally brief and devoid of details. For example, user may type the query ‘changes in strategy’ while reading an annual report.

The retrieval part takes this query and performs a vector comparison with the items in your database. Imagine that an important part of strategy appears on one of the pages, but it does mention much about the term ‘strategy’. The vector comparison will fail 👎to fetch this page.

In general, the shorter the query, the lesser is the possibility of correct retrieval. You might get one item, but miss others. Thus you get an answer that is correct, but does not contain details and nuances.

Since the problem is caused by a short query, there is a simple solution: make the query bigger😌. This is called query enrichment.

How can we make the query bigger such that it fetches more relevant items? There can be so many ways, but we will see two here.

🕐 Use Knowledge Graph: Create a knowledge graph of concepts and relationships. From the user query, extract the key terms and fetch related concepts from the KG. Convert the concepts and the connecting relationships into text. Add all this to the original query before retrieval.

🕑 Use Language Model: Prompt another language model about concepts related to the ones in the user query. Add the response from the LM to the query before retrieval.

This technique belongs to the general class of query pre-processing.

Can you think of more ways to enrich the query? Please add to the comments.

retrievalaugmentedgeneration
generatiaveai
genaiinproduction

By Devesh Rajadhyax
Co-Founder, Cere Labs

Comments

Post a Comment

Popular posts from this blog

Can language models reason?

Image
  Illustration by Rajashree Rajadhyax Large language models are an impressive technology that excels at providing answers based on vast knowledge of the world. However, they often struggle with reasoning and logic-based questions. To overcome this limitation, the Chain-of-Thought (CoT) method was introduced. This technique emulates how humans solve problems by reasoning step-by-step. Just as we break down complex problems into smaller, manageable parts for analysis and logical resolution, CoT guides language models to adopt a similar approach. By mimicking this natural, structured problem-solving process, CoT enhances the models’ ability to handle logic-driven tasks with greater accuracy and reliability. This approach, however, requires more time and computational effort because it involves processing more tokens . Tokens are the building blocks of text, like words or parts of words, that the model uses to understand and generate responses. Since the Chain-of-Thought method involv...
Read more

AI implementation myths

Image
  Visionary leaders recognize the importance of adopting cutting-edge technologies to enhance their business operations. This is a bold move, especially in the early stages when few businesses have embraced it. Generative AI is currently at such a juncture. Although AI has experienced several setbacks over the years, recent advancements in foundational models like large language models (LLMs) have reignited interest and investment in the field. Generative AI never fails to impress those who see it in action. At Cere Labs, my company, we have developed a Gen AI framework, and whenever I demonstrate it, the feedback is unanimously positive, with many stating it is "useful." However, despite this enthusiasm, there is hesitation when it comes to integrating this technology into business practices. Addressing certain misconceptions and concerns could help leaders more easily adopt Generative AI to enhance their operations. With my 8 years of experience assisting businesses in inco...
Read more

Homework 2.0: The science behind homework and why it still matters in the AI age!

Image
  Illustration by Rajashree Rajadhyax I recently came across an article about how teachers are coming up with methods to distinguish between homework done by students and that done by AI. This effort by teachers is not without cause; a recent case in Massachusetts saw a student penalized for using AI in their history paper, highlighting the need for such measures. I found it intriguing and couldn’t help but reflect on this shifting trend.  Homework! Does that word bring back memories? The race to finish it as fast as possible so you could finally go play! Holidays would’ve been so much better without it, don’t you think? I know I felt that way, and I’m pretty sure you did too. And let’s not forget those classmates with endless excuses for not doing it. The most common one? “I wasn’t feeling well!” 🤒 Some would even throw in a few coughs just to make it believable 😏. I was that kid on the opposite end. Just before a holiday, I’d be sneaking in some homework during lunch break...
Read more