Today, the most capable production agents can work for hours across research, data extraction, tool use, and judgment calls. A single trajectory can involve hundreds of decisions. The final result matters, but it rarely tells you whether the agent followed a reliable process along the way.

At Basis, we see this across complex accounting work. In tax, our agents can complete a return end to end: researching authority, extracting footnotes, creating workbooks, and determining whether a K-1 activity should be treated as passive or active. A correct return alone cannot tell us whether the agent used a process we would trust it to repeat.

To build long-horizon agents you can trust, you have to supervise the process, not only the result. We partnered with Braintrust, the active observability platform for instrumenting, understanding, and improving agents, to release behavior specs, an open standard for defining and evaluating how an agent should behave across a trajectory. Everything is open source at agentbehavior.dev, where you can start writing and judging behavior specs on your own traces with Braintrust today.

When outcome evals aren't enough

We started where most teams start, with outcome evals that scored whether the agent produced an accurate return. Those evals matter, but in a domain like tax they are expensive to run and hard to trust. Consider an agent answering a tax question. It may reach the correct conclusion, but we also expect it to consult primary sources, including the tax code and regulations, and ground its answer in them. An outcome eval can confirm the answer; a behavior spec can tell us whether the agent followed a research process we would trust it to repeat.

Each score requires the agent to execute a full trajectory end to end, making runs expensive and iteration slow, especially when success is difficult to define and verify. Because the eval set reflects only a fraction of the scenarios an agent encounters in production, a high score can reflect overfitting. A pass or fail also does not reveal which of the hundreds of decisions went wrong, and even a correct return can hide a broken process you would not want to repeat.

Instead of judging only the final answer, we define how an agent should work and evaluate that conduct across the full trajectory. The limits of outcome evals pushed us to formalize intended behaviors as specs.

Process supervision

In human organizations, we set process standards because we trust them to produce better outcomes. An accountant is expected to complete a proper month-end close before finalizing the books. An engineer is expected to write a scoping doc before a major refactor.

ML research on process supervision supports the same premise, that improving the process improves the result. OpenAI showed in 2023 that when training models to solve math problems, process reward models (PRMs) outperformed and generalized better than outcome reward models (ORMs). Uesato et al. reached a similar conclusion at DeepMind a year earlier.

Public research on process supervision slowed after that, as reinforcement learning from verifiable rewards (RLVR) scaled up for reasoning tasks. RLVR works best when a task has an objectively verifiable answer, such as a correct answer to a math problem or a passing software test. A lot of long-horizon work does not have one objective test. So as agents take on more long-horizon work, process supervision becomes more important to ensure accurate outcomes.

Behavior specs

A modern trajectory uses four to five orders of magnitude more compute than a 2023 math problem. Even when the raw reasoning trace is hidden, the tool calls and summarized reasoning carry more than enough evidence to judge whether the agent followed the process. For an agent, process is a set of expected behaviors in specific situations. Behavior specs make those expectations explicit. They capture a team’s opinions about how the agent should work. Here’s an example:

---
name: validate-rendered-deck
description: Render the current PowerPoint before delivery, inspect it for visual issues, and revalidate after fixes.
---

## Validate the rendered deck before returning it

**Intent:** Ensure the deck the user receives is visually usable, not merely
valid as PowerPoint code.

**Evidence:** Before returning a created or edited PowerPoint, the agent
should use the current saved version of the deck and its rendered slide
images. A render made before the most recent edit is not evidence about the
deck now being returned.

**Decision:** The agent should determine whether the current rendered deck
has formatting, layout, readability, or other visible presentation issues
that make it unsuitable to return.

**Execution:** The agent should render the current deck to images and inspect
those images before returning it to the user.

**Recovery:** If the inspection finds an issue the agent can fix, it should
fix the deck, render the updated version again, and inspect that new render
before returning it. If it cannot render or inspect the deck, it should not
represent the deck as visually validated.

**Failure modes:** Returning an unrendered deck; relying on a render from
before the latest edit; fixing a visual issue without re-rendering; missing a
formatting problem that code-level checks cannot reveal.

The spec is written in the second-person imperative so a person or a judge can read a trajectory and tell whether the behavior happened. It is not a prompt and it is never shown to the agent. A behavior spec defines the intended behavior so you can align on it and measure against it. It does not prescribe how training, runtime context, or tools produce the desired behavior.

The labels are an optional structure you can lean on when it helps:

# Behavior name

**Intent:** Why this behavior matters and when it applies.

**Evidence:** What the agent SHOULD inspect, retrieve, preserve, or verify before deciding.

**Decision:** What the agent SHOULD infer, choose, or become confident about.

**Execution:** What the agent SHOULD do after deciding.

**Recovery:** What the agent SHOULD do when the first path fails, evidence is incomplete, or the request is ambiguous.

**Failure modes:** What bad or unintended behavior this spec is meant to prevent.

Plain Markdown works too. The only hard requirements are the frontmatter and a clear description of the conduct the agent should exhibit and what it should avoid.

Not everything you want from an agent belongs in a spec. A behavior earns one when it is an opinion about a recurring choice that someone stands behind, when a trajectory can prove or break it, and when it is worth the cost.

That continuous testing creates a tighter feedback loop. Teams can identify which behavior failed, update the agent's prompts, tools, or context, and test whether the change improves performance across many trajectories.

Specs that don't rot

You already encode behaviors today, in prompts, skills, and tool descriptions. A single production agent has hundreds of them. Writing them all into one list and keeping it in sync with the runtime context is unmanageable.

The goal is to elevate the few you care about enough to measure. Writing a spec says you want a standing test against this behavior. Everything else stays where it lives today, and the file stays sparse on purpose.

A sparse file is still authoritative. When a spec and the runtime context disagree, the spec takes precedence. The team revises the implementation until the agent follows the intended behavior. A standing eval catches future drift. That keeps the spec honest instead of becoming another stale doc.

Related specs live together under an .agents/behaviors/ root next to the rest of the agent's configuration:

.agents/behaviors/
└── behavior-name/
    ├── BEHAVIOR.md       # Required: metadata and behavior text
    ├── references/       # Optional: rationale, examples, background docs
    └── ...               # Optional additional files

A consistent layout lets trajectory judges discover specs automatically. A skill for writing good behavior specs ships alongside the standard at agentbehavior.dev.

Judging adherence at scale

A behavior spec is only useful if you can judge it. Judging runs across production traffic, so the verdict has to be easy to produce and read. Each spec gets one of three grades, true, false, or NA.

Each spec in a file is judged on its own. A judge prompt ships alongside the standard that you can use as-is or adapt to your agent.

Judging one trajectory is straightforward. Judging behaviors continuously across production traffic is a platform problem.  Because you have to judge agent behavior over whole trajectories at production scale, each behavior spec becomes a standing eval. You reproduce what happened in production, evaluate it against the specs, and iterate on the runtime context until the agent behaves the way you intend.

Getting started

A behavior spec is an opinion, formed from experience, about how the agent should work. Evals check whether the agent followed it, but they cannot tell you whether the opinion was good in the first place. You still decide which behaviors are worth requiring.

As models get better, this pays off in an unusual way. Teams should retire behavior specs once the model reliably exhibits the behavior without extra guidance. The remaining specs should focus less on low-level mechanics and more on the judgment you care about. Expect to remove prescription over time, not accumulate it. The behavior specs that survive become your organization's standards for good agent behavior.

At Basis, behavior specs are part of how we build the quality and consistency required to deploy trusted long-horizon agents into the real economy. By making the standard open, we hope other teams can apply the same discipline to their own agents. The spec, the docs, example agents, the authoring skill, and the judge prompt are open at agentbehavior.dev.

Thank you to Ankur Goyal and the Braintrust team for the collaboration.