As business users deepen their use of generative AI, they often find themselves frustrated by incorrect or irrelevant answers returned by a large language model. Augmenting LLM data with additional data sources can deliver more relevant results and minimize these frustrations. This technique is called Retrieval-augmented generation (RAG), and it allows LLMs to be combined with industry- and topic-specific data to improve the overall quality and relevance of generative AI results.
In this blog post, I’ll offer a quick introduction to RAG, explore its many benefits, and offer a look how IT organizations can use the Boomi platform to set up RAG implementations of their own.
What Is Retrieval-Augmented Generation (RAG)?
Retrieval-augmented generation is a technique that involves fetching useful business-specific context and then including that context with the prompt to an LLM. This technique helps the LLM provide a more accurate, business-specific, and useful reply to the user.
The retrieved context might come from a raw data set that complements the LLMs core training. Or it could even be synthetically generated information. An example design could synthetically generate questions around each chunk of ingested reference data, and then later try to match the user’s question with similar ones from this “hypothetical” question set.
Whichever approach is used — bundling a data set or bundling questions — RAG improves the quality of a generative AI LLM’s output. LLM users are more likely to be satisfied with the responses they receive, and they’re less likely to have to spend time trying out different phrasings or prompts to get the answers they’re looking for.
How does Retrieval-Augmented Generation Work?
Boomi’s low-code development platform is an ideal tool for building a RAG process to optimize the results an organization is getting from an LLM. With its drag-and-drop interface and easy-to-configure components, Boomi makes it easy for developers to build RAG implementations that optimize results for LLMs such as:
- OpenAI’s GPT models (e.g., GPT-3.5 and GPT-4, used in ChatGPT)
- Anthropic’s Claude models
- Google’s PaLM and Gemini models (both of which are used in Google’s Gemini conversational AI tool, known until recently as Bard.)
- Meta’s LLaMA models
The Boomi platform also supports setting up RAG processes for an organization’s proprietary LLMs.
Boomi even provides templates for RAG implementations, so that developers don’t have to start from scratch. Each template sets up the repeatable parts of a RAG process, while leaving the inputs and outputs of the process fully configurable, so customers can connect to the data and systems appropriate to their use cases.
One of Boomi’s RAG templates uses the Pinecone vector database to store and search relevant context in the form of vector embeddings. Embeddings are a machine-friendly representation of text and allow these designs to quickly and effectively fetch relevant information to include in our RAG prompts. Another template uses Anthropic Claude as the LLM, hosted on Amazon Bedrock, and Amazon Kendra enterprise search as the context-source. This second template might be of special interest to organizations that host their infrastructure on AWS and want more control over the data than just calling the OpenAI APIs affords.
The implementation also incorporates two key Boomi concepts for AI solutions:
Context Pipelines
A context pipeline collects and transforms data to optimize it for use by an AI application such as a generative AI LLM. Data teams can run context pipelines before users begin submitting prompts to a generative AI model, so that contextual data is ready for use when users submit their prompts. They can run the context pipeline just once, or they can run it periodically to update the context as new information or data sources become available.
Action Pipelines
An action pipeline is the real-time user engagement with a design like RAG. It can include connectivity to outside systems, retrieval of data, logic, and of course interactions with generative AI models and platforms. When a user executes an action pipeline, it takes advantage of the context already prepared by the context pipeline, so it can deliver results that are as relevant and useful as possible.
Learn more about Boomi’s template for RAG implementations here.
Benefits of Applying RAG to an LLM Use Case
By building a RAG process to complement an LLM, organizations can address important challenges that organizations are facing with their AI projects today.
RAG improves data accuracy and relevance, eliminating the need for multiple trial-and-error questions or prompts. RAGs improve the quality and relevance of responses from LLMs by providing business specific to the LLM that may not be in its base training. They also save users the trouble of submitting multiple queries or requests to an LLM to get the data or response they want.
RAG reduces AI hallucinations. RAGs reduce the risk of AI hallucinations in LLM responses. Hallucinations, also known as confabulations, are false answers generated by LLMs are they try to respond sensibly to a question or request. Hallucinations are common in LLMs. One study found that even with well-known LLMs, hallucinations make up between 3% and 27% of answers.
RAG enriches data while supporting data security and compliance. Using Boomi as an orchestration framework for generative AI designs provides logging and auditability that would be unavailable with employees simply using a generative AI end-product like ChatGPT. Furthermore, if the RAG design includes a model running with in-house infrastructure rather than as calls to an outside API such as Open AI, sensitive data can be kept off of third party platforms entirely, greatly reducing the risk of inadvertent data disclosures or regulatory violations.
Boomi’s Patent for RAG
Boomi was recently granted U.S. Patent No. 11,847,167, titled “System and Method for Generation of Chat Bot System with Integration Elements Augmenting Natural Language Processing and Native Business Rules.” This patent is for a technology solution built on the Boomi platform that creates a series of templatized processes and designs allowing Boomi customers to build hybrid chatbots that combine rules-based connectivity, natural language processing, Boomi logic, and AI technologies to triage and route requests coming into the chat bot. This technology helped lay the foundation for Retrieval Augmented Generation and demonstrates Boomi’s early innovations in applying AI to integration.
Interested in building your own RAG pipeline with Boomi? Read our Boomi Community article about our free Boomi RAG template.