Benchmax

PDF extractionCustoms entry summary › Dispute

Dispute 004 · accepted · produces gold v3 · decided 2026-09-07

Customs entry summary: the relationship indicator printed at the start of the second and third invoices

Fields: line_items[36].relationship_indicator, line_items[59].relationship_indicator. Raised by consensus_test.py, 2026-09-07 (every system disagreed with gold the same way). Decided by Benchmax review, 2026-09-07 (page evidence; founder sign-off pending).

Gold before and after

Fieldv2v3also accepted
line_items[36].relationship_indicatornullY
line_items[59].relationship_indicatornullY

Effect on scores · this document

Systembeforeafter
Codex · GPT-6 Astra · medium effort0.9950.997
Codex · GPT-5.6 Sol · medium effort0.9950.997
Claude Code · Claude Opus 5 · medium effort0.9950.997
Claude Code · Claude Sonnet 5 · medium effort0.9950.997

Regraded from saved outputs. No configuration was rerun.

Original gold

Field Gold v2 Every system
line_items[0].relationship_indicator (line 001) Y Y
line_items[36].relationship_indicator (line 037) null Y
line_items[59].relationship_indicator (line 060) null Y

All other line items: null in gold and in every system's output.

Evidence

The entry covers three commercial invoices. On the form, column 32.C ("Relationship") is printed once per invoice, on the invoice header line that precedes the invoice's first line item:

The three placements are printed identically. The upstream annotation assigned the first Y to line 001 and left lines 037 and 060 null.

Decision

Set line_items[36].relationship_indicator and line_items[59].relationship_indicator to Y, matching the treatment of line 001. No field-rule change.

Rationale

Gold must treat the same printed pattern the same way within one document. Either the indicator printed on an invoice header belongs to that invoice's first line item, in which case all three first items carry Y, or it belongs to no line item, in which case line 001 should also be null. The upstream annotation chose the first reading for invoice 1; this dispute applies that reading consistently. Every system read it the same way, which is evidence about the page, not about the models.

Effect on scores

Two cells of 1,254. Every system gains two matched cells under v3; all four move from 0.995 to 0.997 on this document, so no ranking moves.

Source: 004-customs-relationship-indicator-first-item-of-each-invoice.md and its gold patch. To contest this decision, open a new dispute referencing it.