RAG architecture: 5 patterns for enterprise AI

What is RAG (Retrieval-Augmented Generation)?

RAG (Retrieval-Augmented Generation) is an AI architecture in which a language model retrieves relevant information from external data sources whilst answering a question and uses this as context to generate a response.

A language model does not, by itself, have access to internal documents or up-to-date company figures. Without additional context, it generates answers based on its training data, with the risk of hallucination.

A simple RAG pipeline consists of three steps:

• Retrieval – the query is converted into an embedding vector. A similarity search then retrieves the top-k nearest chunks from a vector index.
• Augmentation – the retrieved chunks are combined with the original query and system instructions to form a single prompt.
• Generation – the LLM generates a response based on the composite context, possibly with a reference to the chunks on which the response is based.

That is the essence of it. The complexity of a production-ready RAG architecture lies primarily in everything surrounding it: how you build and keep the index synchronised, how retrieval is triggered, where session state is maintained, and how you version embeddings when switching models.

RAG architecture versus a traditional data platform

A traditional data platform is optimised for aggregation: for example, give me the total of all sales, using batch ETL and a lakehouse or data warehouse as the single source of truth.

GenAI workloads are optimised for something else. Here, you want to retrieve specific, relevant information quickly, using semantic rather than just SQL-based access, and with low latency.

This does not require replacing your data warehouse. It requires an additional layer alongside it: the AI execution plane.

Both layers can use the same underlying data, but approach it differently. Vector indexes sit alongside tables, and semantic search alongside SQL queries. The existing data infrastructure thus remains the foundation, whilst the AI execution plane makes data available in a different way to LLMs and AI applications.

5 RAG architectural patterns for enterprise AI

RAG is not a one-size-fits-all solution. Depending on how quickly a response needs to be available, whether a conversation requires memory, and whether the system needs to perform actions independently, you will end up with different architectural patterns.

The five patterns build partly on one another. The real-time patterns share the same foundation, with each subsequent pattern adding more statefulness or autonomy.

Which RAG pattern do you need?

The choice of a RAG architecture is primarily determined by the required speed, state and autonomy.

RAG pattern	Real-time	Memory	Tool usage	Typical application
Ephemeral RAG	No	No	No	Batch enrichment
Persistent Index Building	N/A	No	No	Up-to-date vector index
Online RAG	Yes	No	No	Enterprise search
Conversational RAG	Yes	Yes	No	Chatbots and assistants
Agentic RAG	Yes	Yes	Yes	Agentic workflows

Online RAG is suitable for standalone real-time queries. Conversational RAG makes more sense when conversation history forms part of the use case. Agentic RAG becomes relevant when the system needs to be able to consult multiple sources or tools independently.

Persistent Index Building forms the foundation for every real-time pattern – Online, Conversational and Agentic RAG. After all, without an up-to-date, searchable index, there is nothing to query reliably.

The most advanced pattern is not automatically the best. The right architecture is the simplest architecture that can reliably deliver the required capability.

Embedding versioning within a RAG architecture

Sooner or later, you’ll come up against this: you’ve indexed tens of thousands of documents using a particular embedding model, and six months later, a model emerges that performs noticeably better.

The temptation is simply to switch over. The problem is that old and new embeddings do not automatically lie in the same vector space. As a result, retrieval quality can become inconsistent without you realising it.

A robust solution is a blue/green deployment. You build a completely new index using the new embedding model, test it in parallel, and only switch over once the new index performs at least as well. The old index remains temporarily available in case a rollback is needed.

If that is too costly, you can migrate gradually: re-embed documents in batches and query both indexes temporarily.

A third option is to consciously accept the technical debt: continue using the old model for existing data and the new model only for new content. This is a compromise, but sometimes a deliberate architectural choice.

What helps in all cases is to tag embeddings from day one with the name and version of the embedding model. That single line of metadata makes any future migration considerably more manageable.

RAG monitoring: what should you log?

For debugging and evaluation, it is not enough simply to know that a RAG system has provided an answer. You want to be able to reconstruct why it provided that answer.

You should therefore record at least the following for each request:

• the query, user and session;
• which documents or chunks the retrieval returned;
• which final prompt was sent to the model;
• the response generated by the model;
• the latency of embedding, search and generation;
• token usage and costs.

For agentic systems, the full trace is added to this: which tools were called, in what order and with what results.

Without such a trace, a failing agent is difficult to debug. Tools such as LangSmith and MLflow Tracing are designed to provide insight into this decision tree.

RAG monitoring therefore operates on three levels simultaneously:

• System quality: uptime, latency and errors.
• AI quality: does the retrieval fetch relevant information, and does the model provide useful, substantiated answers?
• Costs: what is the cost of a query per user, session and use case?

All three are necessary to determine whether a RAG system not only functions technically but also runs reliably and efficiently in production.

Guardrails for agentic RAG

Agentic RAG requires more explicit boundaries than a traditional RAG pipeline. After all, the model itself determines which tools it uses and how many intermediate steps it performs.

Without boundaries, a session may continue to reason for too long, costs may rise, or an agent may access systems that fall outside its remit.

In a production environment, this means you need to consider, amongst other things:

• a maximum number of LLM calls per session;
• a strict timeout;
• human approval for sensitive actions;
• tool access per agent;
• output validation before a result reaches the user.

As soon as an agent is capable not only of generating text but also of carrying out actual actions, these guardrails become an integral part of the architecture.

Security and governance within enterprise RAG

Access control should take place at the point of retrieval, not merely as a retrospective filter.

If a user does not have rights to a document, that document must remain outside the retrieved context. A model should not receive the information at all.

This also makes governance an architectural issue. For example, who owns the vector store?

If the data platform team manages everything, this creates centralised governance, but may also create a bottleneck for the AI team. If the AI team manages the infrastructure independently, it can move faster, but the risk of data silos and duplicate storage increases.

A possible division of responsibilities is therefore:

• the data platform team retains ownership of the source data and catalogue;
• the AI team builds and manages vector stores that synchronise with these sources;
• both teams use the same catalogue for lineage and access management.

This ensures that governance remains centrally organised, whilst allowing AI teams the freedom to develop their own applications and retrieval layer.

From RAG pilot to production-ready RAG

None of the five RAG patterns is inherently better.

Agentic RAG is not the end point towards which every use case must evolve. Nor is Ephemeral RAG too simple to be taken seriously for batch enrichment.

Each pattern trades complexity for a specific capability: real-time response, conversational memory or tool usage. The right choice – and often the right combination of patterns – depends on what the use case actually requires, not on which pattern sounds the most advanced.

What distinguishes a RAG pilot that never leaves the lab from a RAG platform running reliably in production is rarely a larger model.

The difference lies in the surrounding architecture: orchestration, storage synchronisation, embedding versioning, monitoring, security and access management must be tailored to the actual nature of the problem. They must not be added only after something goes wrong.

Making a RAG solution production-ready therefore requires not only knowledge of LLMs, but also of data architecture, orchestration, security and monitoring. This is precisely where GenAI and data engineering converge.

At Blenddata, we design and build that infrastructure as part of our GenAI platform projects.