Knowledge Graphs Are Making a Comeback — This Time Paired With LLMs

Knowledge graphs are making a comeback because they solve a specific weakness in LLM systems that plain vector search cannot: representing explicit relationships between entities — who reports to whom, which product depends on which component, which clause supersedes which — so a model can reason over structured connections instead of guessing at them from loosely related chunks of text.

Why Knowledge Graphs Fell Out of Fashion

Knowledge graphs had a previous wave of enterprise interest roughly a decade ago, and they fell out of favour for practical reasons: building and maintaining them required significant manual modelling effort, ontologies were brittle and expensive to update as the underlying data changed, and the query languages involved (SPARQL, Cypher) had a steep learning curve that limited who in an organisation could actually use them. Vector search and simpler document retrieval offered a much lower barrier to entry, and for a lot of use cases, "good enough" retrieval from unstructured text was genuinely good enough.

Why LLMs Changed the Calculus

Two things shifted. First, LLMs can now do much of the entity and relationship extraction work that used to require manual ontology design — reading unstructured documents and proposing the graph structure automatically, dramatically lowering the cost of building and maintaining a graph. Second, and more importantly, LLM-based systems exposed a real limitation of plain retrieval-augmented generation: vector search finds text that is semantically similar to a query, but it has no native way to represent that "Company A acquired Company B, which owns Product C" is a chain of facts a user might need connected, not just three loosely related paragraphs. Graph-based retrieval, paired with an LLM that can traverse and reason over the graph, handles exactly this class of multi-hop question that plain vector search answers poorly or not at all.

How a Graph Layer Actually Sits Alongside an LLM

In practice, "graph RAG" is not a replacement for vector retrieval so much as a complementary layer used alongside it. A typical implementation still uses vector search to find relevant starting points in a large document set, then uses the graph to expand from those starting points along explicit relationships — pulling in connected entities and facts that a pure text-similarity search would never surface because they are not phrased similarly to the original query. The LLM sits on top of both, using the graph structure to reason about how retrieved facts relate to each other before generating an answer. This hybrid pattern is now more common in production systems than either pure vector search or a graph-only approach used in isolation.

What "Graph RAG" Actually Improves

The practical benefit shows up specifically on queries that require connecting multiple facts, not on simple lookup questions where vector search already works fine. That distinction matters for deciding whether the added complexity of a graph layer is worth it for a given use case.

• Multi-hop questions — answers that require connecting two or more related facts across documents
• Entity disambiguation — distinguishing between two similarly named entities based on their actual relationships and context
• Explainability — a graph traversal path is a far clearer audit trail than "these text chunks were semantically similar"
• Consistency across a large, changing document set — a graph maintains relationships explicitly rather than re-deriving them per query
• Domain-specific reasoning — regulatory clause dependencies, organisational hierarchies, product bill-of-materials relationships

Enterprise knowledge management is where this pattern is showing up most concretely in the work we do. A large, changing body of internal documentation — policies, contracts, product specifications — naturally contains the kind of cross-referencing structure a graph represents well: this policy supersedes that one, this contract clause depends on that definition, this product component is used in these other products. Plain document search struggles precisely because those relationships are implicit in how humans read the documents, not explicit in the text itself, which is exactly the gap a graph layer is built to close.

Where Plain Vector Search Is Still the Right Answer

Graph-based retrieval adds real engineering and maintenance overhead — building the extraction pipeline, keeping the graph synchronised with source data, and managing the added system complexity. For straightforward document Q&A where questions are single-hop and the semantic similarity between question and answer is strong, plain vector search remains simpler, cheaper, and just as accurate. Reaching for graph RAG by default, the same way multi-agent architectures got over-applied, would repeat a familiar mistake — treating an approach that pays off for a specific class of problem as a universal upgrade rather than a targeted solution to a specific, observed failure mode.

The practical takeaway: evaluate whether your actual failure cases are multi-hop reasoning problems before investing in graph infrastructure. If they are, a graph layer paired with an LLM will measurably outperform plain retrieval. If your retrieval failures are really about document quality or chunking strategy, a knowledge graph will not fix that, and the added complexity will not be worth what it costs to maintain. Start by auditing a sample of your worst retrieval failures — the pattern usually points clearly to which category the problem actually belongs to.