Multi-Hop Reasoning: How It Works, Techniques & Use Cases

No record answers “Which customers are exposed to the supplier that just went offline?” Following a chain does: customer to order, order to part, part to supplier. Each link is a hop, and the answer exists only in their composition. That composition is where retrieval systems and language models fail expensively, because every step can be correct while the assembled answer is wrong.

This post defines multi-hop reasoning and the property that separates it from ordinary lookup, then works through what improves it in practice: knowledge graphs, GraphRAG, and agents.

What is multi-hop reasoning?

Multi-hop reasoning is answering a question that names its target indirectly, through a description that has to be resolved before the target can be looked up. Each step that resolves part of that description is a hop, and the result of an earlier hop, usually an entity, is what makes the next one possible. That intermediate result is often called the bridge entity.

Take “who approved the change that took down the service our checkout page depends on?” One hop resolves which service checkout depends on, the next finds the change deployed to it during the outage window, the third retrieves its approver. Searching on the question’s own wording surfaces nothing useful, because the identifiers that would match the final record only become known after two hops have resolved.

The indirection comes in two strengths. Sometimes the target is a stated fact the hops only make addressable, as the approver is recorded on the change once you know which change. Sometimes the target exists nowhere and the hops construct it, as no record lists which customers a supplier disruption reaches. Text benchmarks lean toward the first kind, including HotpotQA (2018) and its 113k Wikipedia-based questions; the questions organizations ask of their own data lean toward the second.

The property that matters either way is ordering dependency: the second hop’s query cannot be formulated until the first has resolved, which is why retrieving more documents at once does not help. But that dependency does not have to be paid while the question is being answered, and where it gets paid is what separates the approaches below.

How does multi-hop reasoning work?

How a system answers depends on whether the links the chain crosses are already recorded somewhere. When they are not, it has to find them as it goes, which means running a loop rather than a lookup:

Decompose. The question is broken into sub-questions with an explicit dependency structure: which can be answered now, and which wait on an answer that does not exist yet.

Resolve and bind. The first answerable sub-question runs against the knowledge source, and its result is bound in working state as the bridge entity the next step is conditioned on.

Reformulate and repeat. The next sub-question is rewritten with the bridge entity substituted in, turning an unanswerable query into an answerable one, until the chain reaches the original question or the system gives up.

The self-ask prompting method is a compact instantiation: the model explicitly asks itself follow-up questions and answers them before answering the original one. Because those questions surface as structured text, a search engine can answer them instead of the model’s own memory.

When the links are already declared, as edges in a graph schema, none of this loop survives as answer-time work. The whole chain goes into one query, and the order the loop had to discover one bridge entity at a time is worked out internally by the engine’s planner. The loop is the price of leaving the links undeclared.

Multi-hop reasoning vs. single-hop reasoning

Single-hop questions have an answer that exists somewhere, whole. Multi-hop answers have to be assembled.

The error behavior row is the one that decides architecture. A chain of four hops, each independently correct 90% of the time, is correct about 66% of the time end to end, and no component in that system looks broken during debugging. That arithmetic assumes every hop can be wrong, which is a property of how a hop is executed rather than of the question: a hop resolved by retrieval or inference carries an error rate, one resolved by traversing a declared relationship does not.

Key components of multi-hop reasoning

The same parts appear in every multi-hop system. What changes with the substrate is where they live: components you assemble yourself when the chain has to be discovered, and functions inside a query engine when the relationships are already modeled.

A knowledge source with addressable structure. It must support a follow-up conditioned on a specific entity, not only a request for text resembling a question. Entity-linked corpora, knowledge graphs, SQL tables, and tool APIs qualify; an undifferentiated pile of embedded text does not.

A planner. Something decides what the steps are and in what order they unblock each other. Discovered, it is a prompted model and the largest single source of failure; modeled, it is the query planner, and the same job stops producing errors.

Working state and verification. Bridge entities and partial paths persist across steps, and the answer is checked against the evidence that produced each one, catching the failure the table named: every hop defensible, the composition still wrong.

The discovered case makes all three your responsibility. The modeled case hands the first two to the engine and leaves you the schema that defines them.

Multi-hop reasoning in large language models

Language models perform some multi-hop reasoning internally, without being prompted to show their work. A 2024 study of latent multi-hop reasoning probed for this with prompts like “the mother of the singer of ‘Superstition’ is” and found it uneven: for certain relation types the latent pathway appeared in more than 80% of prompts, but a clear scaling trend with model size appeared for the first hop and not the second. That asymmetry is why techniques forcing the chain into the open work as well as they do: an intermediate result that exists as tokens in the context is one the model can condition on reliably, and one an engineer can inspect.

Longer context windows help less than they appear to. Placing a whole corpus in the prompt addresses coverage, but the bottleneck is composition, and the compositionality gap, which measures how often a model answers every sub-problem correctly yet still fails to produce the overall solution, does not narrow with model scale.

Multi-hop reasoning and knowledge graphs

A knowledge graph makes a hop a first-class operation. Entities are nodes, relationships are edges, and a chain of reasoning is a path expressed directly in the query language. The supplier question from the opening becomes a traversal:

MATCH (c:Customer)-[:PLACED]->(:Order)-[:CONTAINS]->(:Part)-[:SUPPLIED_BY]->(s:Supplier)
WHERE s.status = 'disrupted'
RETURN c.name AS customer, collect(DISTINCT s.name) AS suppliers

Hops are explicit, paths are returnable, and the schema bounds what can be traversed, so a relationship absent from the model cannot appear in a result.

That is also where the ordering dependency gets paid. Over text, hop two waits on hop one because the link between them exists only as prose that has to be found and read. Declared as an edge, the link is known before any question is asked. The traversal above is one query rather than four sequential steps, and the hops that compound error in a discovered chain are here just pattern elements resolved against declared relationships. The reasoning has not disappeared; it has moved to modeling time. Deciding that exposure runs from customer through order and part to supplier is the same decomposition the answer-time loop performs, done once by whoever writes the schema and reused by every question that follows.

 Commercial multi-hop questions are mostly the constructing kind. Security operations trace blast radius from a compromised credential through the identities, hosts, and services it reaches, a path query whose depth is not known in advance.

The obstacle in practice is operational. The entities these questions traverse already exist as tables in a warehouse or lakehouse, and standing up a separate graph database to hold a copy means maintaining an ETL pipeline whose lag becomes the graph’s staleness. PuppyGraph maps those existing tables into a graph through a user-defined schema; by default, queries run in place with no ingestion and no persistent duplicate dataset, so a traversal runs against current data. It compiles a graph query into a plan of node and edge operators that executes in its own distributed engine, issuing only simple projection and filter SQL to the sources; because the query is represented as graph operators end to end, the engine optimizes specifically for multi-hop traversals. The schema also functions as an enforced ontology: queries are validated against it before execution, and a reference to an entity or relationship that does not exist is rejected, and the rejection returns structured, LLM-readable feedback, so a broken chain can be repaired instead of quietly returning nothing.

Multi-hop reasoning and GraphRAG

GraphRAG applies the same idea to retrieval: it augments generation from a knowledge graph, instead of or alongside a vector index, so the context handed to the model carries relationships rather than a set of independently similar passages.

That matters because of a specific limitation of similarity search. Embedding retrieval returns passages resembling the query, and the document holding the second hop often resembles the original question very little, since it is about the bridge entity the question never names. MultiHop-RAG, a benchmark built for this shape, finds existing RAG systems inadequate on such queries. Microsoft’s GraphRAG work derives an entity knowledge graph from the source documents, then summarizes clusters of related entities into community summaries that are combined at query time; our GraphRAG architecture deep dive covers that pipeline in more detail. What these designs share is that the context handed to the model carries the relationship itself, so the second hop is already present instead of having to be found by resemblance.

Multi-hop reasoning in AI agents

For an agent, the hops span tools and turns rather than documents: query a graph, read a ticket, query the graph again with what the ticket revealed. The ordering dependency is identical and only the sources differ.

What changes is that an intermediate result becomes a premise. Once a wrong bridge entity enters working state, every later step is competently executed against the wrong subject, and the answer comes back coherent, well-cited, and false. When each generated query is validated against a schema before it runs, and an invalid reference returns structured feedback the agent can read, a broken hop becomes a repairable event rather than a silent detour. This is the self-correction loop ontology-driven agents rely on.

Benefits of multi-hop reasoning

The benefits below belong to multi-hop reasoning made explicit, as a query or a visible sequence of steps. A chain that stays latent inside a model reaches the first of them and not the rest.

Questions no record can answer become answerable. Exposure, blast radius, provenance, and influence are defined by chains rather than attributes, so a system limited to single-hop lookup cannot express them at all.

Reasoning becomes inspectable. An answer carrying its path is auditable in a way a generated paragraph is not. A reviewer can check each link and locate where a wrong answer went wrong, which is a hard requirement in fraud, security, and regulated domains.

Structure gets reused. Entities and relationships modeled once support any path a user later wants to trace, so the same machinery answers an open-ended family of questions.

All three follow from making the chain an explicit object rather than leaving it implicit in a model’s weights or a retriever’s ranking.

How to improve multi-hop reasoning in AI

Model the domain explicitly. Name the entities and relationships the questions traverse, as a schema or an ontology, before tuning retrieval. A hop is only well defined if the relationship it crosses is.

Make hops explicit rather than latent. Externalize each step as a sub-question, tool call, or query, and keep the intermediate result in context. Latent internal reasoning is where the second hop degrades, and it is also the part you cannot debug.

Evaluate on questions that resist shortcuts. A test set only measures multi-hop reasoning if an answer reachable without traversing the chain does not count. Build one over your own domain, then run a single-hop baseline against it: if that baseline scores well, the questions are not testing what you think they are.

PuppyGraph’s built-in AI assistant runs this loop in product form: it generates graph queries from a natural-language question against the enforced ontology, reads the structured rejection when a reference is invalid, and rewrites the query rather than returning a wrong answer.

Conclusion

Multi-hop reasoning answers questions that name their target indirectly, through a chain that has to be resolved before the target can be reached. Its defining constraint is ordering and its defining risk is compounding, since a chain of probabilistic steps is considerably less reliable than any step in it. Both costs fall as more of the chain is declared in advance rather than discovered at query time, which is why the durable improvements are architectural: model the domain, make the hops visible, and let the engine order them.

Try the forever-free PuppyGraph Developer Edition and book a demo with the team to see how openCypher and Gremlin queries traverse warehouse and lakehouse tables, with no graph-specific ETL, so a multi-hop question becomes one traversal over relationships the schema already declares.