Skip to content

T68b Budget Enforcement Validation

T68b Budget Enforcement Validation

Scope

T68b turns budgets.maximumCostUsd, budgets.maximumTokens, and budgets.maximumDurationMs from fields that were parsed, bounded, digested and then ignored into an enforced control. A per-run meter accumulates the usage.updated events every qualified driver already emits, prices them through a versioned table, and stops the run through the executor’s existing cancel-and-checkpoint path at a configured share of the declared ceiling. Consumed spend is sealed into the Run Capsule beside the declared ceiling.

59 cases across four suites, against a declared minimum of 18.

Suite Cases
tests/unit/budget-meter.test.mjs 17
tests/integration/run-scoped-budget.test.mjs 21
tests/integration/run-capsule-budget-evidence.test.mjs 9
tests/fault-injection/budget-enforcement-faults.test.mjs 12

Deterministic gates

Both gates ran on a clean checkout detached at the implementation revision, dispatched through full-validation.yml rather than on a developer machine. The uploaded artifact of each run records the revision and the profile it ran.

Command Result
pnpm gate:quick PASS — run 30670820650
pnpm gate:security PASS — run 30670822421

No profile is a superset of another, so each is listed with what it ran:

Profile Stages
gate:quick format:check, lint, typecheck, test:unit, test:agent-readiness
gate:security format:check, lint, typecheck, build, test:unit, test:architecture, test:qualification, test:security, test:fault

External attestation matters here for a reason this task discovered. An earlier gate:security run of a different task passed on a developer machine and failed on a clean runner, because that machine happened to have the qualified driver binaries installed. Evidence that depends on whose machine produced it is not portable evidence.

Adequacy matrix

Anchored in .specs/features/budget-enforcement/spec.md.

Criterion Requirement Assertion
BUD-01 Usage events accumulate per run; no events means zero budget-meter.test.mjs — accumulation across events, and a meter with no events reporting zero cost and zero tokens
BUD-02 Price-table version recorded; unknown model fails closed budget-meter.test.mjsVES_BUDGET_MODEL_UNKNOWN; run-capsule-budget-evidence.test.mjs seals priceTableVersion
BUD-03 Cost threshold stops through cancel plus checkpoint, recoverable budget-enforcement-faults.test.mjs — a usage flood stops at the threshold with a budget-exceeded checkpoint carrying the meter snapshot, the driver cancelled, the worktree cleaned and coordination released
BUD-04 Duration threshold stops with the same semantics budget-enforcement-faults.test.mjs — a silent driver is stopped by the executor’s own timer, armed from the run’s remaining duration
BUD-05 Capsule carries declared versus consumed and the price-table version run-capsule-budget-evidence.test.mjs — declared-versus-consumed seals and survives verification; a capsule without budget evidence still seals, so older runs stay valid
BUD-06 Thresholds outside (0, 100] and non-positive ceilings rejected budget-meter.test.mjsVES_BUDGET_INVALID on out-of-range thresholds and non-positive ceilings; negative and fractional usage counts rejected as VES_BUDGET_USAGE_INVALID

Discrimination sensor

Each mutation was applied to the implementation, the four suites re-run, and the source restored.

Mutation Criterion Result
An unpriced model returns instead of failing, so an unknown model runs free BUD-02 KILLED
The threshold compares against the raw ceiling rather than its configured share BUD-03 KILLED
Accumulation drops input tokens and counts only output BUD-01 KILLED
The duration branch of the threshold evaluation is removed BUD-04 KILLED
The budget outcome stops winning over the driver’s own cancellation report BUD-03 KILLED
The capsule accepts a negative consumed amount BUD-05 KILLED
The repair loop rebuilds the meter per attempt instead of resuming it BUD-03 KILLED

The last one is the defect this task shipped and then corrected. The meter was originally constructed inside a single execute() call while the gate repair loop calls the executor up to five times, so every attempt received a fresh threshold and a declared run ceiling could be spent once per attempt. Neither T68b nor T68c could have caught it alone: one has no loop, the other has no meter. It was filed as #124 and fixed in the revision this report names, which is why 46d22d8 and not the original 43050df is the implementation revision.

Non-shallow checks

  • A stop is recoverable, never a process kill: the executor cancels the driver and checkpoints stage budget-exceeded with the meter snapshot, and the worktree and coordination lease are released on that path.
  • An unpriced model fails closed rather than running at silent zero cost, which is the difference between a budget and a suggestion.
  • A resumed ledger is validated as untrusted input. Winding consumption backwards on resume would buy a fresh ceiling, so negative, fractional, and infinite values are rejected.
  • Provider usage figures are recorded as claims, not as verified billing facts. Reconciliation against a billing API is explicitly out of scope in the specification, and this report does not imply it.

Verdict

T68b is complete for its declared scope. Six of six acceptance criteria have file-and-assertion evidence, both declared gates pass on the implementation revision through external runs, and every mutation in the sensor was killed.

What this report does not assert: independent verification, or recorded human acceptance. docs/qualification/REPORT-CONTRACT.md is explicit that neither is a field here, and this repository has one collaborator, so independence for the maintainer’s own work is not obtainable by configuration - a limitation docs/merge-governance.md states rather than implies.