Agentic coding tools comparison

The measurement rig behind Claude Code Is Way More Token-Hungry Than OpenCode. We Measured Exactly How Much, Systima's study of harness token overhead in agentic coding tools.

It measures what a coding-agent harness sends to the model API before and around your prompt; system prompts, tool schemas, injected scaffolding, prompt-cache behaviour, and the multipliers real configurations stack on top (instruction files, MCP servers, framework templates, subagents).

How it works

harness (Claude Code / OpenCode / any Anthropic-compatible CLI)
  -> logging proxy (captures request payloads + response usage)
    -> model endpoint

The proxy records two things per request. The exact JSON payload the harness emitted (system blocks, tool schemas, messages), which is ground truth for what the harness sends. And the usage block the API returned (input tokens, cache writes, cache reads, output tokens), which is ground truth for what was metered.

Layout

Path	Purpose
rig/proxy.mjs	Capturing HTTP proxy. Forwards to UPSTREAM_URL, writes one JSON capture per request, optionally appends each record to a tamper-evident audit chain
rig/run-lane.sh	Runs one isolated benchmark lane (fresh config dirs, labelled captures)
rig/analyse.mjs	Aggregates captures into results/results.json (per-lane payload sizes, token totals, cache columns)
rig/ingest-audit.mjs	Replays all captures into a SHA-256 hash-chained audit log via @systima/aiact-audit-log
fixtures/	Seed files for the tasks (e.g. notes.md for the file-summarise task)
qbench/	Quality benchmark; a seeded ten-assert test suite plus reference implementation
results/results.json	Aggregated results from the July 2026 study (all 58 lanes, 273 captured requests)

Quick start

Requires Node 20+, plus the harnesses you want to measure (claude, opencode).

# 1. Start a lane; the runner boots the proxy, runs the harness against it, and shuts it down
export ANTHROPIC_API_KEY=sk-ant-...
sh rig/run-lane.sh my-first-lane claude "Reply with exactly: OK"

# 2. Inspect the captured payloads
ls captures/my-first-lane/

# 3. Aggregate
node rig/analyse.mjs

Lane types; claude (Claude Code), claude0 (Claude Code with --tools ""), opencode (OpenCode), opencode0 (OpenCode; give it a project opencode.json containing {"tools": {"*": false}} in the lane workdir).

Isolation

The floor numbers only mean something if the harness runs clean. run-lane.sh points CLAUDE_CONFIG_DIR, XDG_CONFIG_HOME, and XDG_DATA_HOME at fresh directories under the bench, so your real MCP servers, memory, and settings stay out of the measurement. Multiplier lanes then add one variable at a time (an instruction file in the workdir, an .mcp.json passed explicitly, and so on).

Measuring through a gateway

If your traffic passes through a local LLM gateway rather than the API directly, calibrate before trusting metered figures. Send a bare one-line request through the same chain, note the cold cache-write total, and set it as the envelope constant:

GATEWAY_ENVELOPE_TOKENS=6200 node rig/analyse.mjs

Payload-level figures come from captured request bodies and need no calibration; a gateway cannot affect them. Check the servedModel field in the captures too; our gateway silently substituted model snapshots, which we only caught because we were logging at the boundary.

Quality benchmark

qbench/ holds the seeded test suite used to check that token differences were not quality differences. Each lane gets a fresh workspace with a uniquely named ten-assert test file (tests-template.js, instantiated per lane with a nonce) that the harness must satisfy but cannot see the grading of; reference-impl.js verifies the suite's ground truth. Hash the test files before and after each lane to prove the harness did not modify its own exam.

Audit chain

Every capture can be replayed into a tamper-evident, SHA-256 hash-chained audit trail:

npm install @systima/aiact-audit-log ai @aws-sdk/client-s3
AUDIT_DIR=./audit-logs node rig/ingest-audit.mjs
npx aiact-audit-log verify --dir ./audit-logs --system-id harness-token-bench

This is the same mechanism the library provides for EU AI Act Article 12 logging. The study's own 273-record chain verifies end to end.

What is and is not published here

results/results.json contains the full aggregated dataset from the study; per-lane, per-request payload sizes, token counts, and cache columns. The raw captures are retained privately and are not in this repository, because they embed complete request payloads; harness system prompts, real-world MCP tool schemas, and personal configuration context. The rig lets you regenerate equivalent captures for your own setup in minutes.

Caveats

One machine, one pair of harness versions (Claude Code 2.1.207, OpenCode 1.17.18), July 2026. Harness prompts change frequently, and as the study found, vary by model tier; treat the numbers as a snapshot and the method as the durable artefact. Full caveats in the blog post.

Licence

MIT