AI Agent Harnesses: Why the Same Model Produces Different Results

Two AI coding systems can use the same model, receive the same task, and produce very different costs, completion times, and engineering outcomes.

The difference is often the AI agent harness: the system around the model that controls context, tools, memory, delegation, retries, validation, recovery, and human approval.

That distinction matters because organizations still tend to evaluate agentic AI as if the model name determines the result. The model matters, but it is only one component. The harness decides how that capability is used—and how much waste accumulates around it.

At Chrysent, this is the design premise behind OPI, our working local-first software engineering harness, and OPI Forge, the self-hosted software engineering platform available in private beta.

What is an AI agent harness?

An AI agent harness is the operational layer that turns a language model into a working system.

A model can interpret instructions and generate an answer. A harness gives it access to context, tools, memory, execution environments, policies, reviewers, and recovery mechanisms. It also decides when the model should continue, delegate, retry, stop, or ask a human to decide.

In practical terms:

Agent = model capability + harness design

For software engineering, the harness can determine:

• which repository context enters the prompt;
• which tools and actions are available;
• how work is divided into bounded tasks;
• which model handles each task;
• how findings and decisions are preserved;
• what tests and reviews are mandatory;
• how failures are retried or escalated;
• when human authority is required; and
• what evidence is needed before work is considered complete.

A strong model inside a weak harness can be expensive, repetitive, and difficult to trust. A disciplined harness can make the same model more efficient, more consistent, and easier to govern.

The evidence: same model, different economics

The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI makes this effect measurable.

In a Writer-authored controlled study, the researchers held the tasks and models constant while replacing a conventional production loop with a more disciplined agent harness. Across 22 enterprise-agent tasks and six models, the paper reports:

• 38% fewer tokens per task;
• 41% lower cost per task; and
• 44% lower median latency.

Aggregate task quality moved from 0.78 to 0.81. At this sample size, we view that as approximate quality parity rather than proof of a material quality improvement.

The study is meaningful, but it should be interpreted carefully. It is vendor-authored, includes one of the vendor's own models, uses a small task set, and relies partly on LLM-based judging. Its strongest and most defensible conclusion is not that one harness is universally superior. It is that orchestration is a major efficiency variable even when the underlying model does not change.

That is the harness effect:

Same model. Same task. Different orchestration. Different economics.

Where agentic AI wastes tokens

Most token waste is invisible to the person waiting for the result. It accumulates inside the operating loop.

A poorly designed agent may repeatedly:

1. replay a long conversation history on every turn;
2. expose a large tool catalog when only one tool is relevant;
3. load entire files because it cannot identify the useful symbols;
4. send every specialist's full output back into the parent context;
5. retry failures without changing the conditions that caused them;
6. poll while waiting instead of resuming from an event or checkpoint; and
7. reconstruct old decisions after an interruption because no durable state exists.

This is sometimes described as token maxing: buying more capability by spending more tokens, even after additional context stops improving the outcome.

The problem is not that context is bad. The problem is that irrelevant, duplicated, and poorly routed context is expensive.

How OPI changes the shape of the work

OPI is Chrysent's working local-first harness for agent-assisted software engineering. We use it to develop our own platform and tooling. A self-hosted, single-developer OPI option is available today as part of our enterprise offering.

It does not ask one model to behave like an entire engineering organization inside one endless conversation. It organizes delivery into bounded work, specialized roles, review gates, durable artifacts, recovery points, and human-controlled decisions.

The goal is straightforward:

Give each model the right job, the right evidence, the right boundary, and the right gate.

1. Keep durable work outside the active chat

Plans, findings, implementation notes, reviews, validation results, and handoffs do not need to remain permanently inside the model's active context.

OPI treats these outputs as durable artifacts. The orchestrating process can inspect a compact status first and retrieve deeper detail only when that detail changes the next decision.

This reduces the need to repeatedly purchase the same history with every model call.

2. Route on summaries before reading full reports

A specialist's output may contain thousands of tokens, but the next routing decision may require only a few facts:

• Did the work pass?
• Is there a blocker?
• Is a follow-up required?
• Does another reviewer need to inspect it?
• Is the full report necessary now?

Summary-first routing reserves detailed context for the cases where it is actually useful.

3. Bound the task before choosing the model

Not every task requires the strongest model, the largest context window, or the longest reasoning budget.

Routine inspection, formatting, summarization, and narrow checks can often use faster or less expensive models. Architecture, security analysis, difficult debugging, or cross-system synthesis may justify stronger models and more reasoning effort.

OPI treats model selection as a routing decision—not a religion.

4. Resume instead of rebuying history

Long-running work gets interrupted. Sessions compact. Networks fail. Machines restart. Humans return later.

When the chat transcript is the only source of truth, recovery requires rereading and reconstructing what the system already knew. OPI is designed around checkpoints, preserved decisions, and durable progress so work can resume from state rather than restart from memory.

Continuity is not only a productivity feature. It is part of the token economics.

5. Make review part of execution

A coding assistant often treats “done” as the moment it stops generating text.

OPI uses a governed software delivery loop. Work can move through requirements, design, implementation, review, validation, operational checks, evidence, handoff, and closure. Reviews return verdicts and follow-ups. Failed gates route work back to the appropriate step.

The model did not suddenly become more intelligent. The system became more disciplined.

Our internal same-model demonstration

Our own internal evidence points in the same direction, although it should not be treated as a universal benchmark.

In one small Python coding task, we ran the same underlying model—claude-opus-4-8—through OPI and Claude Code. The variable was the harness.

In that run, OPI used about 3.7× fewer input tokens and cost about 71% less, while producing an equivalent or better artifact in our review.

The caveats matter: this was an internal test, on one small task, with a sample size of one. It demonstrates that harness overhead can dominate the cost profile of a task. It does not prove that every OPI task will achieve the same reduction.

That is why we do not optimize for the lowest raw token count in isolation. We optimize for less wasted context per accepted result.

Why larger context windows do not solve the problem

A larger context window increases capacity. It does not create judgment.

A 256k-token or one-million-token window can hold more information, but it does not decide:

• which information is relevant;
• what should remain outside the conversation;
• which tool is appropriate;
• which task should be delegated;
• when a retry is likely to help;
• which review is mandatory; or
• whether the output is safe to accept.

If the harness treats the context window as a storage bin, a larger window can make bad habits more expensive. The result is often longer prompts, more repeated history, and a larger bill—not better engineering.

The better approach is deliberate context engineering:

Keep durable work outside the chat. Retrieve the right evidence at the right time. Give each task only the context it needs.

Better process can improve model performance

Cost is only half of the story.

Software engineering is not a single act of code generation. It is a sequence of decisions, constraints, implementations, reviews, tests, operational checks, and approvals.

A raw model can generate plausible code. A governed harness can ask whether that code:

• satisfies the accepted requirement;
• respects architectural and security constraints;
• includes sufficient tests;
• introduces operational risk;
• drifted beyond the agreed scope;
• has evidence supporting closure; and
• requires a human decision before proceeding.

This is why specialized AI workers are useful only when the system around them is disciplined. More agents do not automatically create better results. Roles, boundaries, evidence, and gates are what make specialization useful.

Human authority belongs inside the harness

OPI is not built around the assumption that software delivery should become fully autonomous.

Some decisions belong to people. Product scope, architecture obligations, security posture, and material testing strategy should not change silently because an agent found a convenient implementation path.

Agents can investigate, recommend, implement, review, and surface tradeoffs. The harness should preserve explicit human authority where the decision changes what the organization is building or the risk it is accepting.

That boundary is essential for enterprise trust.

Code intelligence narrows the context further

Harness discipline becomes more powerful when the agent can retrieve precise code context.

Large repositories are hostile to naive agents. Without a structured map, an agent may read many files, repeat searches, miss dependencies, and spend most of its context discovering where the work begins.

Chrysent Axis Code is the code intelligence substrate that can give agents a repository map through exact search, semantic search, symbol lookup, dependency relationships, repository-scale navigation, and MCP access.

OPI Forge is the software delivery harness. Axis Code is the code intelligence layer. Together, they attack different parts of the same problem:

The harness narrows the work. The code graph narrows the context.

Better retrieval does not guarantee fewer tokens on every task. Sometimes it reveals more relevant work and correctly causes the agent to spend more. The honest objective is not fewer tokens at any cost. It is better accepted outcomes per dollar.

The metric that matters: cost per accepted outcome

Token count is a bill, not a measure of value.

A cheap run that fails review is expensive. A larger model can be economical if it avoids repeated attempts. A smaller model can be ideal when the task is narrow and well bounded. A better retrieval layer may use more tokens because it found a risk that a cheaper run would have missed.

The useful questions are:

• Did the work satisfy the requirement?
• Did it pass review and validation?
• Did it avoid regressions?
• Did it preserve security and operational constraints?
• Did it produce inspectable evidence?
• Can the next session understand and continue the work?
• How much did the accepted result cost?

That is cost per accepted outcome.

OPI is designed to improve that ratio—not simply to make the token counter smaller.

From OPI to OPI Forge

OPI is the working local-first harness we use today. OPI Forge is the self-hosted software engineering platform that brings the same operating model to private-beta teams.

The private-beta product includes:

• governed agentic software delivery;
• specification-driven workflows;
• specialized AI workers;
• review and validation gates;
• durable project memory;
• model and provider flexibility;
• code intelligence integration;
• human-controlled decision points; and
• auditable delivery artifacts.

For an individual developer, a local harness can be powerful. For a team, the same system also needs shared visibility, policy, reporting, managed configuration, and operational support.

That is the role OPI Forge fills for private-beta teams.

Frequently asked questions

What is an AI agent harness?

An AI agent harness is the operational software around a language model. It manages context, tools, memory, execution, orchestration, retries, validation, governance, and human escalation so the model can perform multi-step work reliably.

Why can the same AI model produce different results?

The model may be identical, but the harness can change what context it receives, which tools it can use, how tasks are decomposed, how failures are handled, and what validation is required. Those choices affect cost, latency, consistency, and the final result.

Do larger context windows reduce AI agent costs?

Not automatically. Larger windows allow more information to be sent, but they do not determine whether that information is relevant. Without disciplined context selection, larger windows can increase repeated input and total cost.

What is the difference between OPI and OPI Forge?

OPI is Chrysent's working local-first software engineering harness and the foundation of the product. OPI Forge is the self-hosted platform for teams that need shared workflows, governance, visibility, policy, and operational support, available in private beta.

The harness sets more than the token bill

The Writer paper makes a claim we think the industry should take seriously:

The harness sets the token economics of agentic AI.

We agree.

We would add one more point:

The harness also sets the engineering discipline of agentic AI.

Cost and competence are linked. A harness that sends less irrelevant context also gives the model a clearer job. A harness that routes bounded work to specialists reduces confusion. A harness that preserves artifacts improves continuity. A harness that requires review improves accountability. A harness that selects models per task can improve both cost and capability.

The next wave of AI software engineering will not be won only by larger models or longer context windows.

It will be won by systems that know how to use them.

That is what OPI is built for.

And that is what OPI Forge offers to private-beta teams.