Why — Agentic Dynamics

Systems are constantly evolving. Scaling agents means measuring that evolution.

Agentic Dynamics is the empirical study of how AI agents behave, adapt, interact, recover, and produce outcomes across changing tasks, environments, workflows, and time. This repository is its founding instrument and corpus.

“The system shapes the agent — and the agent reshapes the system. Read that again.”
What — the field

A measurement question became Agentic Dynamics.

Field map of Agentic Dynamics: five disciplines — AI and agentic systems, FinOps and cloud economics, software engineering and architecture, systems and reliability engineering, and experimental science — surround a core that measures how context, complexity, and uncertainty shape agent decisions and propagate into reliability, cost, and value.
Five disciplines feed one core: how context, complexity, and uncertainty shape agent decisions — and propagate into reliability, cost, and value.

Agentic Dynamics is the empirical study of how AI agents behave, adapt, interact, recover, and produce outcomes across changing tasks, environments, workflows, and time. This instrument currently examines cost, verification, maintainability, recovery, and five-session progression in coding-agent workflows. Routing and long-horizon value remain questions for controlled experiments, not conclusions inferred from a price sheet.

How — the question scaled

One question became an N x M measurement problem.

The original question compared one model's cost and quality under degraded input. Answering it properly required N linked sessions, because each output becomes the next session's codebase, and M measurement angles, because passing tests cannot stand in for verification, maintainability, cache economics, routing risk, or long-horizon cost. The corpus is the product of that multiplication, not a larger benchmark.

N — linked sessions

  • Session 1: greenfield build from spec
  • Session 2: feature addition
  • Session 3: integration (auth, async, persistence)
  • Session 4: refactor (repository pattern, layering)
  • Session 5: cross-cutting (rate limiting, pagination)

Controlled variation

  • Clean — unmodified specifications
  • Early degrade — the first spec is corrupted, cascading through later sessions

M — measurement angles

  • AST diffs, conventions, and maintainability per commit
  • Maintainability cost and long-horizon scalability
  • One-way/two-way-door routing and verification depth
  • Cache economics, recovery, tests, and independent review

215 stories across 3 codebases (CRUD API, static site generator, notification service) and 2 tiers, run against 7 models. Every measurement is provenance-tagged; a signal that was not captured is published as null with its coverage, never a fabricated zero. Collection creates one N x M burden; analysis creates a second across models, conditions, tasks, tiers, commits, and story positions. More sessions without analysis are volume, not information.

How — from events to policy

Instrument, derive, write policy, grid, campaign.

Instrument — capture events
Session 1–5 traces, tokens, cache reads, cost, tests, and code changes
Derive — turn events into information
Static analysis, reviews, verification depth, recovery, and cost per accepted outcome
Write policy — consume measured information
Control rules use only fields the instrument actually produces
Grid — test policy as an arm
Compare policy choices across controlled factor combinations
Campaign — change one variable and repeat
Use the remaining uncertainty to select the next measurement
The instrument cycle
Instrument to derive to write policy to grid to campaign and back to instrument. Derive is the only path into policy; unmeasured requirements block it. THE LOAD-BEARING RULE TO MAKE POLICIES, WE NEED INFORMATION INSTRUMENT → DERIVE → WRITE POLICY → GRID → CAMPAIGN → REPEAT 01 / INSTRUMENTINSTRUMENTledger events: quality · grit · confidence 02 / DERIVEDERIVEmeasurement rules → named information 03 / WRITE POLICYWRITE POLICYcontrol rules consume measured fields 04 / GRIDGRIDtest policy as an arm 05 / CAMPAIGNCAMPAIGNchange one factor · emit next spec emit fields produce · the only path assign as arm compare arms next spec unmeasured requirement blocks policy
Figure H1. The instrument cycle — Instrument → Derive → Write policy → Grid → Campaign, returning to Instrument. Derive is the only path into policy, and a control rule whose requirements are not measured cannot be written.
[P] method structureref: svg-marker-flow.html
What — the evidence

The unmeasured gap became visible.

The supporting corpus contains 1,067 instrumented story sessions. Four findings made the gap concrete.

Finding 1

Price and verification depth do not move together in this corpus.

OpenAI spans a 41× price range — Luna at $0.09 and Sol at $3.75 — and both write ~10 tests per story. Claude spans 2.9× — Haiku at $1.59 and Sonnet 5 at $4.58 — and both write ~120. All pass their tests. In these observed tasks, the 10× verification gap appears between vendors rather than between price tiers; the result describes this corpus, not a universal purchasing rule.

Finding 2

Later sessions cost more across the five-session arc.

The measured story cost roughly doubles from its greenfield session ($0.16) to its cross-cutting session ($0.34), while tokens grow from 18.7K to 34.9K. Every session inherits a larger codebase, so per-session cost understates the observed maintenance arc by ~2×. Generalizing that mechanism beyond the measured arc remains modeled.

Finding 3

Cache behavior separates token volume from session cost.

The cheapest model (Flash) and the most expensive (Sol) both send ~47K tokens per session — Flash at $0.07, Sol at $3.75. Luna hits a 98% cache-hit rate and sends just 6.3K. Token volume, cache policy, and test thoroughness appear as three separate knobs in this corpus; their separation does not establish that any one predicts the other two.

Finding 4

Degradation can leak across sessions, and recovery can be measured.

Corrupting the session-1 spec (early_degrade) lowers the success rate to 86% (versus 97% clean) and raises the cascade rate (1.5% → 2.5%). Degradation raises risk at the margin; it does not dominate the outcome. Cells labelled bad_seed/early_degrade that carried no instrumented test verdict received no actual perturbation and are counted as clean (docs/data_integrity_findings.md treatment rule 1), so the arms above are the corrected ones.

What — the durable-value gap

Correctness is not verification.

A model can pass every test it writes and still leave a system unverified. The gap is measured by test thoroughness:

Tests per story: Luna 7 · Terra 9 · Sol 13 · Flash 34 · Pro 34 · Haiku 117 · Sonnet 122

All seven models pass their own tests at roughly equal rates, but the tests differ by an order of magnitude. Luna ships ~7 tests that confirm the happy path. Sonnet 5 ships ~122 that probe edge cases — duplicate usernames, whitespace titles, missing bearer tokens, idempotent notifications. Both codebases "work." One has a much broader observed verification surface.

This is the durable-value gap. Correctness asks whether code passes the tests you have. Verification asks whether you have the tests that would catch the failures that matter. In this corpus, that safety net varies by vendor family, while price alone does not explain the difference.

Evidence Instrument Framework Story GitHub Related Work