Databricks playbook · Agentic Dynamics instrument

Two Methods, Convergent Questions

In July 2026, Databricks published their playbook for reducing AI coding costs at scale — based on surveys of Stripe, Coinbase, Uber, and Ramp. The sources reached related operational conclusions through different evidence: Databricks contributes enterprise survey evidence, while this instrument contributes 227 historical controlled stress tests across 8 models and 3 architectures.

Read the original Databricks post →
Scope labels
[X] external
Databricks' survey findings are external context — quoted with their source and date (Aug 2026 blog), never used as causal support for a corpus finding.
[M]/[C] instrument
This instrument's measurements are scoped to the 227 historical controlled experiments + the 1,067-session story corpus; they are not enterprise deployment data.
Anti-SaaS guard
Related-work comparison only — no product framing, no conversion language.

The Efficiency Frontier and Grit

Databricks defines the Efficiency Frontier as "the set of models that have the best price point for a given level of intelligence." We define Grit as Ground-Truth Integrity — how well a model maintains correctness when given degraded, contradictory, or incomplete instructions.

Complementary concepts examined through different methods. The Efficiency Frontier concerns price for a level of intelligence; Grit concerns integrity under degraded input.

Databricks

Their evidence [X] external

Surveys of Stripe, Coinbase, Uber, and Ramp. Directional estimates. "Public benchmarks do a poor job of indicating real-world performance on coding tasks." (source: Databricks blog, Aug 2026 — [X] external context, not corpus evidence.)

This instrument

Our evidence

227 historical controlled experiments across 8 models and 3 architectures. 10 calibrated perturbation operators. Every measurement provenance-tagged [M] measured, [C] computed, [H] heuristic, [X] external.

Lever 1: Shift to Efficient Models

Databricks

Their finding [X] external

"Rapidly adopting newer, more efficient models delivers the largest cost wins." ~50% savings. "The efficiency frontier is advancing far faster than the intelligence frontier."

This instrument

Our measurement

$0.02/session (DeepSeek) vs $1.08/session (Claude). 69× measured cost gap. Same generated token volumes (~11K tokens/session). The gap is pricing architecture, not capability. Head-to-head correctness across overlapping task types when regrouped shows different model-rankings across perturbation classes — no single provider dominates.

Where they complement each other: Their survey focused on price/intelligence tradeoffs from enterprise deployment data. Our controlled experiments measure integrity under stress — revealing that a model can be cheap and smart on a benchmark yet collapse under inverted constraints. Grit captures something the Efficiency Frontier wasn't designed to measure.

Lever 2: Dynamic Request and Task Routing

Databricks

Their finding [X] external

"Internal results suggest that our AI Gateway Smart Router is able to consistently reduce average task cost by more than 30%." Request-level, task-level, and escalation routing.

This instrument

Our measurement

Grit-routed strategy: 95% correctness at $0.12/session. Per-task routing decision table from 17 task types. 14/17 → DeepSeek default. 3/17 → Claude escalation. DeepSeek-only beats Claude-only on BOTH cost AND correctness.

Where they complement each other: Their routing strategy is based on task complexity taxonomy from real-world deployment data. Our Grit-gated routing adds stress-resilience to the equation — measuring flail rate under perturbation to ensure cheaper models don't collapse when instructions degrade.

Lever 3: Visibility, Tripwires, and Progressive Budgets

Databricks

Their finding [X] external

"Instead of a hard user spending cap, most companies are adopting a more nuanced and progressive approach." Visibility, spend gates, downshifting, suspension. ~10% savings.

This instrument

Our measurement

WOC = 0.90 — measured first-pass success rate across 227 historical experiments. Rule 7: Budget Ceiling (Tmax = Budget / Cjob). Rule 8: Cascade Rule (escalate only on Grit failure). Their "downshifting" = our cascade. Their "spend gates" = our WOC alerts.

Where they complement each other: Their approach handles reactive budgeting — detecting and responding to overspend in real-time from live deployment data. This instrument adds forecasting: the Snowball Rule (N²) and EPM Horizon model multi-year cost trajectories, projecting costs before they compound.

Lever 4: Reducing Token Overhead

Databricks

Their finding [X] external

"Simple tuning of our harness and caching settings led to an almost 50% reduction in the number of generated tokens." Compaction, less chatty harnesses, cache tuning. ~10% + 50% token reduction.

This instrument

Our measurement

Explanation Tax: Claude flails on 11% of sessions (zero code, narration only) vs DeepSeek 8%. When models succeed, DeepSeek carries negligible narration overhead (0.0% penalty), Claude carries 8.5%. Cache Tax: $3.75/Mtok writes vs $0.14/Mtok reads — 26.8× pricing asymmetry. Claude writes 20.0K cache tokens/session at premium rates. DeepSeek reads 223K at negligible cost. Exact same insight. Calibrated measurements.

Where they complement each other: Their surveys and experiments identified that token overhead exists. Our stress tests measured why it happens — the model's architectural default to conversational mode under perturbation — making it actionable at the model selection level as well as the tool tuning level. If you're tuning your tools to reduce verbosity, you're working against the model's training distribution. Routing to a model with lower narrative overhead may be more cost-effective.

Where Each Adds Unique Insight

Databricks brings industry scale — real-world deployment data from companies processing millions of AI workflows daily, validated against enterprise budgets. Their surveys capture operational patterns at a breadth that controlled experiments alone cannot match: adoption lifecycles, organizational friction, and the full cost of change management.

The Agentic Dynamics instrument contributes methodology that enterprise surveys can't provide: controlled reproducibility, perturbation stress testing that reveals model fragility invisible on clean benchmarks, provenance-tagged measurements at the session level, and an open-source instrument deployable against any workload.

Perturbation stress testing

Measures Grit through 10 calibrated degradation operators — revealing that a model performing well on benchmarks may fail entirely under degraded instructions.

Model behavior under degraded input

Manifold vs semantic perturbation classification. Some models handle logical corruption (94% correctness). Others collapse under linguistic distortion (escape=0.76).

Compounding and forecasting

Snowball Rule (N²): codebase growth compounds quadratically. EPM Horizon: energy prices inflate compute costs. Survival Horizon: how many sessions before bankruptcy?

Attractor basin topology

The shape of a model's failure space. Wide-shallow basins explore efficiently. Narrow-deep basins recover expensively. Collapsed basins flail. Measured, not theorized.

Autonomous workforce model

Batch economics (50% discount, 72-hour horizon). Escalation cascades. SLA buffers. Workforce capacity modeling — projecting autonomous agent workloads beyond individual developer use.

Reproducible evidence

Provenance-tagged measurements [M]/[C]/[H]/[X], open-source under Apache 2.0, inspectable down to individual session transcripts. Every measurement is independently verifiable.

Where they converge: Databricks and this instrument examine four related operational areas with different scopes. Their surveys captured operational patterns at enterprise scale — from companies processing millions of AI coding workflows daily. Our 227 historical controlled experiments provide calibrated measurements under controlled stress. Together they offer complementary evidence: enterprise survey findings and provenance-tagged, reproducible instrument results that anyone can test against their own workloads.

Read the original Databricks playbook →
See the full experimental evidence →
View the Full Evidence →