Build the Pipeline, Not the Agent

We published a piece a few weeks ago about a tool called Agentless, which has no agent in it, and which outscored every open-source agent framework it was measured against on SWE-bench Lite at about 70 cents a problem. The argument landed. The question we got back was the practical one: fine, so what do we build instead?

This is that. Six steps that turn one workflow into a fixed-path pipeline, in the order the decisions actually arrive.

Each step has three parts. Run this is the action. What good looks like is the default with its reasoning attached. Change it if is the condition that moves the default. If your team is past the basics, the third part is the one for you: every default here is a starting position, and a team that knows why a default sits where it does can move it deliberately.

One thing this guide will not give you is a number for how many tools is too many. We went looking. The figures in circulation trace back to vendor blogs citing “recent academic studies” and naming none, and the nearest real paper measures something else entirely and points the other way. They are in our do-not-use ledger now. Where no sourced number exists, we say so and give you the reasoning instead.

Step 1: Write the path down before you write any code

Most teams start by choosing a framework. That is choosing the answer before stating the problem, and it is the step where the real decision gets skipped.

Run this. Take the one workflow you picked and write its steps as a numbered list, in plain language, the way you would explain it to a new hire. Then mark each step with one of two labels: specifiable, meaning you could write down the rule, or judgment, meaning you genuinely could not.

Our test for which label applies: two engineers who know the workflow, asked separately to write the rule for that step, would produce the same rule. If they would, it is specifiable. That is the same two-experts-agree test we use for eval tasks, reused here, and it is our default rather than a benchmark.

What good looks like. Most steps come back specifiable. That is the expected result, not a sign you did it wrong. The value of this step is that it converts a vague sense of “this workflow needs AI” into a list where the AI-shaped part is usually two or three lines out of fifteen.

Change it if you cannot write the list at all, in which case you have found something more useful than an architecture decision. A workflow nobody can enumerate is not ready for automation of any kind, and no amount of model autonomy will supply the specification you do not have.

Step 2: Fix the phases

Agentless runs three fixed phases: find the code, propose a fix, validate the fix. The model never decides its own next action, never plans, and gets no complex tools. That structure is most of why it worked.

Run this. Group your labeled steps into phases that always run in the same order. The spine that generalizes from Agentless is locate, produce, validate: gather the inputs the work needs, generate the candidate output, check it before it lands. Write the phases as actual control flow, so the order is a property of your code rather than a hope about the model’s behavior.

That generalization is ours. Agentless built its three phases for GitHub issue repair, not as a universal shape, and we are extending it.

What good looks like. A validate phase that can fail closed. In Agentless the validation is running the tests, which is a hard signal. Yours might be a schema check, a rules engine, a diff against the prior version, or a human in the loop on a sample. What matters is that the pipeline has a place where an output can be rejected without a human noticing it first.

Change it if your workflow genuinely branches, and then fix the branch too. A conditional is still structure. What you are avoiding is the model choosing the sequence at runtime, not the sequence having more than one shape.

Step 3: Put judgment only where you could not hardcode

This is the step where teams buy autonomy they did not need, and it is worth being precise about what the evidence says on each side.

Run this. Go back to your list from step 1 and implement every step marked specifiable as ordinary code. Then, for the steps marked judgment, start with exactly one of them handed to the model, and get the pipeline working end to end before you hand over a second.

What good looks like. One judgment step to start. That is our default, and the reasoning is that each judgment step is a place the run can diverge, divergences compound, and starting at one is what makes the cost comparison in step 6 readable at all.

The evidence on structure being worth more than teams expect is unusually clean, because it comes from a test designed to prove the opposite. The common argument for skipping structure is that hand-built scaffolding gets eaten by the next model release. A controlled comparison on the GAIA benchmark ran three scaffolds against five models with three attempts per question and pre-registered exactly that hypothesis. It was rejected. Scaffold choice alone moved measured accuracy by as much as 28 percentage points within a single model, and the most capable model tested gained the most from structured scaffolding on the harder problems.

We state that one carefully, because it is a single-author preprint that has not been peer reviewed and whose author names four confounds in his own data. It is enough to retire the claim that structure is about to become unnecessary. It is not enough to prove the reverse.

Change it if you can show on your own eval that a second judgment step improves the outcome per dollar. That is the whole test, and step 6 is where you run it. The burden of proof sits with the complexity, not with the simple version.

Step 4: Give each step only the tools it can use

Run this. For every step that calls the model, pass only the tools that step could plausibly need. Not the workflow’s tools. That step’s tools.

What good looks like. A short menu per step, and no number attached to “short.” Part of why Agentless worked is that a smaller action space is easier to be right in, and Anthropic’s own engineering guidance says to find the simplest solution possible and increase complexity only when needed, going as far as “this might mean not building agentic systems at all.” The same post warns that framework abstraction layers “obscure the underlying prompts and responses” and “make it tempting to add complexity when a simpler setup would suffice.”

We would like to give you a threshold here and we are not going to, because the thresholds in circulation did not survive being checked. The direction holds without them: every tool definition you pass is context spent and one more wrong turn available.

Change it if a step genuinely needs breadth, such as an open-ended research step. Then the honest move is to accept that this step is the expensive one and measure it separately, rather than letting its tool menu become the default for every other step in the pipeline.

Step 5: Keep writes single-threaded

If a second model is in your design, this step decides what it is allowed to do.

Run this. Name the one component that owns changes to each system of record. Everything else in the pipeline reads and advises, but does not write.

What good looks like. Advisors, not actors. Cognition published a post called “Don’t Build Multi-Agents” and then walked it forward in April 2026 into a sharper rule earned in production: multi-agent systems work best when writes stay single-threaded and the additional agents contribute intelligence rather than actions. Their working example is a review agent with clean context that catches about two bugs per pull request, 58% of them severe. The reason it works is fresh context, not parallelism.

That is the most portable rule in this area. Extra models are allowed to know things. They are not allowed to do things.

The failure modes on the other side are catalogued. A UC Berkeley team built a taxonomy of 14 failure modes across three categories from 150 traces, backed by more than 1,600 annotated traces across seven frameworks, and their framing of the field is that performance gains on popular benchmarks “are often minimal.”

Change it if you have a genuine parallel workload with no shared state, such as scoring a thousand independent records. Parallelism is not the problem. Concurrent writes to the same system of record are.

Step 6: Price the run, not the result

This is the gate on steps 3 and 5, and it is the step almost nobody runs.

Run this. Before you keep the more complex version, give the simple version the same budget and compare. Same tasks, same eval, same dollars.

What good looks like. A comparison per dollar rather than per result. Anthropic published what its multi-agent research system consumes: an agent uses about 4x the tokens of a chat interaction, and a multi-agent system about 15x. The same post reports that the multi-agent setup beat single-agent Claude Opus 4 by 90.2% on research evaluations. Read one line further and the picture changes: token usage by itself explains 80% of the performance variance on their BrowseComp evaluation, with three factors explaining 95%.

So a large share of what looks like an architecture win is a spending win. You can buy a good deal of that improvement by letting one model think longer, which means the honest comparison gives the simple version the same budget and sees what survives.

Anthropic is also direct about where the trade stops working: multi-agent systems fail on tasks that need shared context, on work with heavy interdependencies, and on most coding. That is a vendor telling you to buy less of its product, which is worth reading twice.

Change it if the model changes. A cheaper or stronger model can flip the arithmetic that justified the complex version, in either direction. Re-run the comparison on any model change, which is our default and follows from the same logic that makes a golden regression set worth keeping.

The order is the argument

Run them in sequence, because each is load-bearing for the next. You cannot fix phases you have not enumerated, so 1 gates 2. “Judgment only where you could not hardcode” is meaningless until the hardcodable steps are named, which is what step 1 produces. And 6 is the gate on 3 and 5, because added complexity stays only if it wins per dollar.

Two things this guide is not saying. It is not saying agency never pays: Anthropic’s research system is a real case where it did, and they published the cost. And it is not saying frameworks are worthless, only that choosing one is step 7, not step 1.

Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027, citing cost and unclear business value, from a poll of more than 3,400 organizations. That is a forecast rather than a measurement, and we would not build a case on it. But it is pointed at the same thing the measured sources are: the expensive part of these projects is rarely the part doing the work.

The version that costs 70 cents a task is not a worse version you settle for. On the one benchmark where somebody checked, it was the better one.