PDF extraction › Customs entry summary › Dispute
Dispute 004 · accepted · produces gold v3 · decided 2026-09-07
Customs entry summary: the relationship indicator printed at the start of the second and third invoices
Fields: line_items[36].relationship_indicator, line_items[59].relationship_indicator. Raised by consensus_test.py, 2026-09-07 (every system disagreed with gold the same way). Decided by Benchmax review, 2026-09-07 (page evidence; founder sign-off pending).
Gold before and after
| Field | v2 | v3 | also accepted |
|---|---|---|---|
| line_items[36].relationship_indicator | null | Y | |
| line_items[59].relationship_indicator | null | Y |
Effect on scores · this document
| System | before | after |
|---|---|---|
| Codex · GPT-6 Astra · medium effort | 0.995 | 0.997 |
| Codex · GPT-5.6 Sol · medium effort | 0.995 | 0.997 |
| Claude Code · Claude Opus 5 · medium effort | 0.995 | 0.997 |
| Claude Code · Claude Sonnet 5 · medium effort | 0.995 | 0.997 |
Regraded from saved outputs. No configuration was rerun.
Original gold
| Field | Gold v2 | Every system |
|---|---|---|
line_items[0].relationship_indicator (line 001) |
Y |
Y |
line_items[36].relationship_indicator (line 037) |
null |
Y |
line_items[59].relationship_indicator (line 060) |
null |
Y |
All other line items: null in gold and in every system's output.
Evidence
The entry covers three commercial invoices. On the form, column 32.C ("Relationship") is printed once per invoice, on the invoice header line that precedes the invoice's first line item:
- Page 1, above line 001: the first invoice's header carries
Y. - Page 5, above line 037: "Invoice Number 002/US202203-0004" with
Yin column 32. - Page 7, above line 060: "Invoice Number 003/US202203-0005" with
Yin column 32.
The three placements are printed identically. The upstream annotation assigned
the first Y to line 001 and left lines 037 and 060 null.
Decision
Set line_items[36].relationship_indicator and
line_items[59].relationship_indicator to Y, matching the treatment of
line 001. No field-rule change.
Rationale
Gold must treat the same printed pattern the same way within one document.
Either the indicator printed on an invoice header belongs to that invoice's
first line item, in which case all three first items carry Y, or it belongs
to no line item, in which case line 001 should also be null. The upstream
annotation chose the first reading for invoice 1; this dispute applies that
reading consistently. Every system read it the same way, which is evidence
about the page, not about the models.
Effect on scores
Two cells of 1,254. Every system gains two matched cells under v3; all four move from 0.995 to 0.997 on this document, so no ranking moves.
Source: 004-customs-relationship-indicator-first-item-of-each-invoice.md and its gold patch. To contest this decision, open a new dispute referencing it.