Benchmax

PDF extractionUnclaimed property list · synthetic › Dispute

Dispute 003 · accepted · produces gold v3 · decided 2026-09-07

Unclaimed property list: owner names wrapped onto two lines were truncated to the second line

Fields: owners[137].name, owners[164].name. Raised by consensus_test.py, 2026-09-07 (every completed system disagreed with gold the same way). Decided by Benchmax review, 2026-09-07 (page evidence; founder sign-off pending).

Gold before and after

Fieldv2v3also accepted
owners[137].nameMAUDIENEMCCART JENNIFER MAUDIENE
owners[164].nameCENTERMACK ALFORD CORT CENTER

Effect on scores · this document

Systembeforeafter
Codex · GPT-6 Astra · medium effort0.0000.000
Codex · GPT-5.6 Sol · medium effort0.9790.979
Claude Code · Claude Opus 5 · medium effort0.0320.032
Claude Code · Claude Sonnet 5 · medium effort0.0000.000

Regraded from saved outputs. No configuration was rerun.

Original gold

Field Gold v2 What every completed system returned
owners[137].name MAUDIENE MCCART JENNIFER MAUDIENE
owners[164].name CENTER MACK ALFORD CORT CENTER

Evidence

Page 2 of the list. The name column wraps long names onto a second line. The extracted text reads, in order:

MCCART JENNIFER MAUDIENE 317 N STOCKTON

and

MACK ALFORD CORT CENTER PO BOX 220

In both places the second line is the continuation of the owner's name, printed in the same column and directly above the owner's address. The upstream annotation kept only the second line. Two of the four systems completed this document (the other two timed out); both returned the full two-line name.

Decision

Set owners[137].name to MCCART JENNIFER MAUDIENE and owners[164].name to MACK ALFORD CORT CENTER. No field-rule change.

Rationale

The schema asks for the owner's name as printed. A name that the layout wraps is still one name; the wrap is a rendering artifact, and the address that follows each entry shows where the record boundary actually is. The two cells are the only ones in 80,177 where every system agreed against gold on this document, which is the signature of a parsing error in the annotation rather than a model error. This document is tagged synthetic upstream; the fix is to the annotation, not the document.

Effect on scores

Two cells of 80,177. Under v3 the two completed systems each gain two matched cells; the timed-out systems remain at zero. The change is identical for every system that submitted, so no ranking moves.

Source: 003-unclaimed-list-wrapped-owner-names.md and its gold patch. To contest this decision, open a new dispute referencing it.