Two screenshots became a failing data contract

The bug report was two images.

The first showed a card in an app:

Region: SOUTH
Waiting: 92,785+
Invited: 534+

The second was a photograph of the source dashboard:

WAITING | SOUTH | 93,698
INVITED | SOUTH | 975

No stack trace. No query. No pipeline log. Just four numbers that should have described the same thing and did not.

This is the kind of problem an agent can make much faster or confidently make worse. “The data is stale” is an easy diagnosis. So is “clear the cache.” The useful move was to turn the screenshots into a contract before touching code.

Normalize the claim

Two visible numbers are comparable only when their dimensions match. Write the claim as one row:

{
  "month": "06/2026",
  "region": "SOUTH",
  "status": "INVITED",
  "population": "PROGRAM_A",
  "app": { "value": 534, "qualifier": "floor" },
  "source": { "value": 975, "qualifier": "exact" }
}

Then check the hidden dimensions: refresh month, region, status, population, stock versus flow, category filters, and whether each value is exact, censored, a floor, or an estimate.

Many apparent data bugs are label bugs. Here the dimensions matched. The disagreement was real.

Trace one value backwards

Use a fixed route:

label
component prop
API or read model
cache
query
stored row
load mapping
parsed artifact
raw export
source dashboard

At every boundary, record the value and dimensions. Do not jump from UI to database and call the pipeline stale.

The component received 534. The formatter added +, so presentation was not changing the magnitude. The read model produced 534 by summing exact category rows. The database contained them. The loader and parser preserved them. The raw export contained detailed rows, many with <20.

The source dashboard's 975 appeared only when the category dimension was removed.

That was the first divergence. The pipeline ingested:

month x region x status x category

The app needed a separately published table at:

month x region x status

The app was not reading the wrong row. The necessary row had never entered the system.

The shape of the discrepancy was evidence

The waiting discrepancy was about one percent. The invitation discrepancy was much larger because more detailed invitation cells were censored.

If cache staleness were the cause, the statuses might lag similarly. If the wrong region were selected, both could be far apart. A discrepancy that grows with the proportion of censored cells points toward aggregation grain.

The screenshots contained that clue before anyone opened the repository.

Make the screenshots executable

The regression suite should preserve the relationship, not the private product domain:

test('coarse exact total covers the detailed floor', () => {
  const coarse = parsePublishedCount('975')
  const detailedFloor = 534

  assert.doesNotThrow(() => assertCoarseTotalCoversDetail(coarse, detailedFloor))
})

Pin suppression too:

assert.deepEqual(parsePublishedCount('<20'), {
  kind: 'censored',
  exact: null,
  lower: 0,
  upper: 19,
  source: '<20',
})

The screenshot-to-contract playbook captures the investigation pass. The aggregation-grain lab makes the core semantics executable.

Split the agent roles

I use four passes:

1. The investigator has read-only access and produces competing hypotheses.
2. The builder implements the smallest source-to-screen vertical slice.
3. The browser tester gets only the expected visible behavior and deployed URL.
4. The release reviewer searches the production diff for a sequence where data and code arrive in the wrong order.

These roles can use the same model. Separation still helps because each pass has a different objective and smaller context.

The first question stops premature debugging. The backward trace locates the defect. The executable invariant prevents recurrence.

The screenshots were not weak evidence. They were an incomplete specification.