What Is Graph Engineering? Definition and Patterns

A single agent loop has a ceiling. It runs inside one context window, works through its steps in sequence, and asks the same model that produced the work to judge whether the work is good. Teams that hit that ceiling tend to arrive at the same answer: run more than one loop, give each a narrower job, and connect them. The moment they do, the design question changes shape. Prompt wording and loop tuning become local decisions inside a larger structure that now has to be designed on its own terms.

That design work acquired a name in 2026, and the name spread faster than the ideas behind it. This guide covers what graph engineering means, how it relates to loop engineering, the components an agent graph is built from, the topologies that recur in production systems, the distinction between an execution graph and a knowledge graph, and the failure modes that appear once a system has more than one node.

What is graph engineering?

Graph engineering is the practice of designing the graph an agentic system executes in: the nodes that do the work, the edges that route work between them, and the shared state that travels along those edges. A node can be an LLM call, a deterministic function, a router, a verifier, or a human approval step. An edge encodes which node runs next, either as a fixed transition or as a decision made at runtime. State is the data structure that every node reads from and writes to as work moves through the system.

The term surfaced in AI-engineering writing in mid-2026, as a successor to loop engineering, and spread within days. It stuck because it named something teams were already doing. The structure itself is old: state machines, DAGs, and workflow engines have modeled work this way for decades, and LangChain’s own retrospective treats the vocabulary as the latest in a series that includes prompt engineering, context engineering, harness engineering, and loop engineering. What is genuinely different is the role the model plays inside the structure. In a conventional workflow engine, every transition is decided by code. In an agent graph, some nodes are deterministic and some are model calls, some edges are fixed and some are chosen by a model at runtime, and the engineering problem is deciding which is which. Code handles the routing that should be predictable; the model handles the steps that require interpretation.

Graph engineering vs loop engineering

The two are layers of the same design, not competing approaches. A loop is a graph whose path returns to an earlier node, so loop engineering describes one shape that graph engineering can express. At the graph level, the loop is one component in a larger system.

Reading down the last row is the fastest way to see what the shift costs. Loop engineering’s failures are failures of a single runaway process, and they are caught by a budget or a better check. Graph engineering’s failures are coordination failures, and they are caught by design decisions made before the system runs: how the work was divided, what each node is allowed to write, and what happens to the run when one branch fails. The practice moves from tuning the quality of one cycle to deciding which cycles exist and how they constrain each other.

Why graph engineering matters

Four ceilings push teams from one loop to several. Context exhaustion is the most common: a long research or migration task accumulates more intermediate material than a single context window holds, and quality degrades well before the hard limit. Serialized latency is the next, because a loop that processes ten independent subtasks in sequence takes ten times as long as it needs to. Missing isolation shows up when unrelated concerns share one context and interfere, so a retrieval failure in one subtask pollutes the reasoning on another. Self-grading is the subtlest: a model asked to evaluate its own output is a weak judge, and the loop has no other place to put a verifier.

Anthropic’s engineering writeup on its multi-agent research system, published in June 2025, puts numbers on both sides of the trade. Their multi-agent configuration, with one lead agent and parallel subagents, outperformed the equivalent single-agent setup by 90.2% on an internal research evaluation. The same writeup reports that multi-agent systems consume roughly fifteen times the tokens of a chat interaction, and that the architecture pays off only on tasks valuable enough to justify that. Their assessment of where it does not pay off is equally specific: domains where every agent needs the same context, or where subtasks depend heavily on each other, are poor candidates, and most coding work falls into that category because the subtasks are rarely independent.

That pairing is the argument for treating the graph as a design artifact rather than an emergent property. Fan-out buys parallelism and isolation at a real cost in tokens and coordination complexity, so the decision about which work becomes a separate node is an engineering decision with a price attached. Systems that grow nodes opportunistically pay the cost without reliably getting the benefit.

Core components of an agent graph

Whatever framework a team uses, the same four pieces have to be specified.

Nodes are the units of work, and they are not all model calls. A node can wrap an LLM invocation, but it can equally be a deterministic function that parses output, a router that classifies an input and picks a branch, a verifier that checks work against explicit criteria, or a human checkpoint that pauses the run for approval. LangGraph’s documentation makes the point plainly: nodes “can contain an LLM or just good ol’ code.” Deciding which nodes need a model is the first real design choice, and the cheapest reliability win is usually converting a model call that was only ever doing bookkeeping into a function.

Edges carry routing, and control transfer has more than one meaning. A fixed edge means one node always follows another. A conditional edge means a function or a model inspects the current state and selects the next node. The distinction that matters most in practice is who keeps control after the transfer. The OpenAI Agents SDK separates agents-as-tools from handoffs on exactly this line: with agents-as-tools, a manager calls a specialist for a bounded subtask and retains control of the conversation, while a handoff routes control to the specialist, which owns the remainder of the turn. Picking the wrong one produces systems that either bottleneck on a manager or lose the thread when a specialist takes over and never gives it back.

Shared state is a contract, and concurrent writes need a rule. State is the structure that travels the graph, and once two nodes can run at the same time, the question of what happens when both write to the same field becomes unavoidable. LangGraph handles this with a reducer per state key, where the default replaces the value and custom reducers accumulate, for instance by appending to a list. The general requirement holds regardless of framework: every field that parallel branches can touch needs a defined merge behavior, and fields without one are a source of silent data loss.

Control means checkpoints, budgets, and termination. Agent graphs run long enough that failures during a run are normal rather than exceptional, so the system needs to persist progress at boundaries and resume from them rather than restarting. It also needs limits that are not stopping conditions inside a single loop: a budget per node, a ceiling on how many subagents a fan-out may spawn, and terminal nodes that end the run. Anthropic’s multi-agent research writeup describes early versions of their system spawning fifty subagents for simple queries, which is what an unbounded fan-out looks like in production.

A framework supplies defaults for all four, and the defaults are where unexamined designs come from. Where the node boundaries fall, which transitions are fixed, how concurrent writes merge, and what bounds a run are decisions a team makes whichever tool holds them.

Common agent graph topologies

Most production systems are assembled from a small set of recurring shapes. Anthropic’s Building Effective Agents, published in December 2024, named most of them well before the graph vocabulary arrived, which is the clearest evidence that the practice predates its label.

‍Shapes where code fixes the path. A sequential pipeline (prompt chaining) decomposes a task into ordered steps, each consuming the previous step’s output, with programmatic checks between them. Parallel fan-out and gather splits work into independent branches that run at once and merges the results, in either of two forms: sectioning, where each branch handles a different subtask, and voting, where the same task runs several times and the results are compared. Both shapes are predictable, cheap to debug, and the right default whenever the decomposition is known in advance.

Shapes where a model chooses the path. Routing classifies an input and sends it to a specialized branch, which lets each branch be optimized separately instead of one prompt trying to cover every case. Orchestrator-worker goes further: a lead node decomposes the task at runtime, delegates the pieces to workers, and synthesizes their results. This is the shape behind Anthropic’s research system, where the lead agent spawns subagents that explore different aspects in parallel, each with its own context window. It suits work whose subtasks cannot be enumerated ahead of time, and it is where fan-out costs concentrate.

Shapes that fold a loop back into the graph. Evaluator-optimizer pairs a node that produces work with a separate node that critiques it, looping between the two. Keeping them as distinct nodes is what makes the verifier independent, which is the property a single loop cannot provide. Hierarchical or nested graphs, which came from the frameworks rather than that original pattern set, let an entire subgraph act as one node in a parent graph. That is how large systems stay comprehensible: the parent expresses the high-level flow, and each node expands into its own topology.

A related decision cuts across all of these: how much of the graph is fixed before the run and how much forms during it. A fully static topology is predictable, auditable, and easy to test, but it cannot express work whose shape depends on what the system discovers. LangChain’s retrospective puts the working compromise as systems that “mix known structure with runtime variability.” The same retrospective notes that agent graphs are usually not DAGs, because production systems retry failed tool calls, ask users for missing information, revise answers after validation, and pause for human input before resuming. The practical position is to fix the parts of the topology that carry policy, cost, and approval, and to leave runtime flexibility where the work genuinely varies.

Execution graphs versus knowledge graphs

An execution graph and a knowledge graph share a word and almost nothing else, which is why nearly every explainer on the topic opens with a disclaimer.

An execution graph models control and state. Its nodes are units of work, its edges are permitted transitions, and its lifetime is a single run. It answers the question of which component acts next and what information it receives. A knowledge graph models a domain. Its nodes are entities, its edges are relationships between them, and it persists across runs as a description of what exists in the business. It answers questions about how things are connected. The two are designed by different people for different reasons, and a system can use one, both, or neither.

They do meet at a specific place. Every node in an execution graph that touches enterprise data has to express a query, and the execution graph has no opinion about whether that query is meaningful. Routing decides when a node reads data; the data model decides what a read can correctly say. That second question is where multi-node systems accumulate a class of error the graph itself cannot catch.

Grounding the nodes on a semantic data layer

The failure that survives every syntactic check is the query that runs and returns the wrong thing. A generated query can be valid SQL, join real tables, and still ask a question the schema does not answer: an entity that does not exist under that name, a relationship inferred from a column that means something else, a metric aggregated across a grain that makes it meaningless. Nothing in the execution graph rejects this. The node returns rows, the edge fires, and the next node reasons over a confident, wrong answer.

Multi-node systems make this worse in two ways. Each node that gets its own tools and its own schema description is another place for the model’s understanding of the data to drift, so a graph with eight data-touching nodes maintains eight approximations of the same schema. And because results flow along edges rather than back to a human, a semantic error introduced in one branch is synthesized into a final answer several nodes later, by which point its origin is hard to locate.

PuppyGraph addresses this at the data layer rather than the orchestration layer. It defines a graph schema over existing tables in SQL databases, warehouses, and lakehouses, and answers openCypher and Gremlin queries against it, so that schema functions as an enforced ontology: every query is validated against that model of entities, relationships, and properties before it executes, and references to entities or relationships that do not exist are rejected with structured, LLM-readable feedback that explains the violation in the domain’s own terms. In graph-engineering vocabulary, that rejection is a signal an edge can route on, so an invalid query becomes a transition back to the node that produced it, and the bad data never travels forward. One ontology shared by every node also replaces the per-node schema approximations with a single contract. Because the tables stay in the warehouse, lake, or open table format where they already live, there is no separate graph store to keep in sync with the data the rest of the system already trusts.

 PuppyGraph does not orchestrate agents, define topologies, or run workflows. It is the layer a node queries, which is why it sits under the graph rather than inside it.

Observability and governance for agent graphs

Operating a graph is different from operating a loop, and most of the difference comes from there no longer being a single transcript to read. A loop produces one linear history. A graph produces a trace per node and per edge, with branches that ran concurrently and a merge point where their results combined. Debugging means reconstructing which path the run actually took, which is why the routing decisions themselves need to be recorded, not just the node outputs.

Evaluation attaches at node boundaries rather than to the system as a whole. A node with a defined input and output contract can be tested in isolation, which turns an otherwise opaque system into components with measurable behavior. Anthropic’s multi-agent research writeup reports getting useful signal from samples as small as twenty queries when effect sizes are large, and using an LLM judge against an explicit rubric, with human review reserved for the edge cases automation misses.

Cost and policy both become graph properties. Token spend attributes to nodes, so the expensive part of a system is identifiable rather than aggregate, and budgets can be set where the spending happens. Policy is the more consequential case: an approval requirement expressed as an instruction in a prompt is a suggestion, while the same requirement expressed as a human checkpoint node with an incoming edge is a structural constraint the run cannot route around. Making the topology explicit is what makes an agentic system auditable at all, because the set of paths a run could have taken is written down rather than inferred after the fact.

Failure modes and best practices

The failure modes worth designing against are specific and mostly not model failures.

State contention occurs when parallel branches write the same field without a defined merge rule, and it presents as results that silently disappear. Fan-out cost compounds quietly, since each additional worker multiplies token spend whether or not it contributed. Error propagation turns one bad branch into a corrupted synthesis, because downstream nodes have no way to know a predecessor’s output was wrong. Non-deterministic routing makes reproduction hard, since a run that fails once may take a different path the next time, and unlinked traces make the failed path slow to reconstruct. And over-engineering is the most common of all: a graph built where one loop would have sufficed, carrying coordination costs for parallelism the workload never needed.

The practices that follow are each an answer to one of those. Start with the simplest topology that works and add nodes only when a specific ceiling is being hit, which is the same advice Anthropic gives about agentic systems generally: increase complexity only when simpler solutions fall short. Make edges explicit rather than leaving transitions to a model that could pick anything, and reserve model-decided routing for decisions that genuinely need judgment. Keep the verifier a different node from the producer, since independence is the whole point of separating them. Type the shared state and define a merge rule for every field a parallel branch can write. Budget per node as well as per run. Checkpoint at node boundaries so a long run resumes rather than restarts. And give the nodes that read data one governed model to read against, so a routing decision downstream is made on results that mean what the graph thinks they mean.

Conclusion

The object being designed has moved up a level twice in two years: from the wording of a single call, to the shape of the loop that keeps calling, to the topology of the system those loops run in. Graph engineering is the name that landed on the third step, and the practice it names was in place before the name arrived. Nodes, edges, and shared state are where a multi-agent system’s correctness, cost, and auditability are now decided, and they are decided before the system runs. None of that structure helps if the nodes reason over data they have misunderstood.

Try the forever-free PuppyGraph Developer Edition and book a demo with the team to see how openCypher and Gremlin queries traverse warehouse and lakehouse tables, with no graph-specific ETL, so every node in an agent graph queries one enforced semantic model instead of its own approximation of the schema.