Vector RAG, Context RAG, Ontology RAG, and Graph RAG: four tools, not four competitors
“RAG” has quietly become shorthand for one specific pattern: chunk a document, embed it, store the vectors, retrieve the closest matches. That pattern works, but treating it as the whole story is why so many RAG systems plateau at “good enough for a demo.” In practice there are at least four distinct retrieval philosophies in production today, and they solve different problems. Knowing which one you actually need, before you start building, saves months of retrofitting.
1. Vector RAG
The baseline pattern: chunk documents, embed them, store the embeddings, and retrieve the top-K nearest matches by similarity.
- Best for: broad recall across large, unstructured corpora, PDFs, transcripts, support tickets, where you mostly need to find the right neighborhood of a document.
- Main weakness: chunks lose meaning at their boundaries. A chunk that just says “revenue grew 3%” is useless once it is separated from which company and which quarter it was talking about, and that kind of context loss is the single most common cause of RAG systems returning confidently wrong answers.
2. Context RAG
Context RAG fixes the boundary problem by generating context for each chunk before it is embedded, then reranking results after retrieval, instead of embedding raw chunks in isolation.
The clearest public evidence for this pattern comes from Anthropic’s own Contextual Retrieval research: combining contextual embeddings with contextual BM25 cut their top-20-chunk retrieval failure rate by 49%, and adding a reranking step on top of that pushed the reduction to 67%. That is a large jump for what is essentially a preprocessing change, not a new model or a new index.
- Best for: precision retrieval where being wrong is expensive, contracts, financial filings, policy documents.
- Main trade-off: more preprocessing cost and rerank latency, which matters if your query volume or corpus size is large.
3. Ontology RAG
Ontology RAG retrieves through the lens of an explicit domain model: entities, relationships, hierarchies, and constraints, rather than raw semantic similarity alone. A vector search can retrieve a clause; an ontology tells you what obligations, exceptions, and stakeholders that clause is actually tied to, and gives you a traceable path back to the source.
This is the pattern I reach for most often in enterprise BI and Text-to-SQL work, where the hard part is rarely finding a relevant table or column. It is knowing which hierarchy, which business rule, and which exception applies to a given question before you ever generate a query.
- Best for: compliance and governance workloads, and any system where “traceable to a source of truth” matters as much as “found the right answer.”
- Main trade-off: you have to build and keep the domain model current. An ontology that drifts out of sync with the underlying data is worse than no ontology at all.
4. Graph RAG
Graph RAG retrieves by traversing a knowledge graph, linking entities, walking multi-hop relationships, and pulling subgraphs relevant to the query, rather than ranking documents in isolation. Microsoft’s GraphRAG project is the best-known reference implementation, using community-level summaries of the graph to answer questions that span an entire dataset rather than any single document.
- Best for: multi-entity, cross-document, “global” questions, where the relationships between facts matter more than any individual passage.
- Main trade-off: the highest reasoning depth of the four, but also the most expensive to build and the most sensitive to a stale or incomplete graph.
The mental model
I think about the four patterns as a progression, not a ranking:
Vector finds relevant information. Context sharpens what gets retrieved. Ontology understands how that information connects to the rules of your domain. Graph reasons across those connections. None of them replaces the others; they answer different questions.
| Pattern | Retrieval unit | Best for | Main trade-off |
|---|---|---|---|
| Vector RAG | Embedding similarity | Broad recall on unstructured corpora | Loses meaning at chunk boundaries |
| Context RAG | Contextualized, reranked chunks | Precision retrieval where errors are costly | Extra preprocessing and rerank cost |
| Ontology RAG | Domain-model-guided retrieval | Compliance and governance, traceability | Requires maintaining an accurate domain model |
| Graph RAG | Multi-hop subgraph traversal | Cross-document, relationship-heavy questions | Graph construction and freshness overhead |
Examples in practice
Abstractions are easier to tell apart with a concrete question attached to each one.
Vector RAG. A support team loads five years of PDF manuals and past ticket transcripts into a vector store. A rep asks “how do we reset a unit that won’t power on after a firmware update,” and the system pulls back the passages closest in meaning, even though none of them use the rep’s exact wording. That is the pattern doing its job: broad recall with no manual tagging, which is exactly what “just find something relevant” needs.
Context RAG. A finance team asks their system “what was our Q2 revenue growth.” In plain Vector RAG, a chunk that just says “revenue grew 3%,” pulled from the Q3 filing, can rank as high as the real Q2 answer, because in embedding space the two sentences look almost identical. Context RAG prepends the surrounding filing, section, and quarter to each chunk before embedding it, then reranks, so the actual Q2 chunk wins instead of a numerically similar chunk from the wrong quarter.
Ontology RAG. A compliance question like “does this vendor contract require us to notify the customer before subcontracting” cannot be answered from one clause read in isolation. An ontology encodes that a subcontracting clause is linked to a notification obligation, which is linked to the customer relationship, so the system retrieves the whole obligation chain instead of a single paragraph, and can point back to exactly which clause each part came from.
Graph RAG. A team asks “what are the recurring failure modes across five years of incident reports.” No single report answers that question. Graph RAG builds a graph linking incidents, root causes, and affected systems, then summarizes at the community level, so it can answer a question that spans the entire corpus instead of retrieving the single closest report and missing the pattern entirely.
Where this actually matters
The systems I have seen hold up in production rarely commit to just one pattern. They layer them: vector search for broad candidate recall, a rerank pass for precision, and an ontology or graph layer reserved for the subset of questions that genuinely require reasoning across entities rather than a single lookup. The mistake is not picking the “wrong” pattern; it is building all four for a problem that only ever needed the first one, or trying to force a relationship-heavy question through plain vector search because that is the pattern you already had running.