Benchmax

PDF extractionByline Bancorp earnings report › Dispute

Dispute 002 · accepted · produces gold v2 · decided 2026-09-06

Byline Bancorp Q2 2023: headline rounding rule and pre-tax income labeled as operating income

Fields: total_revenue, net_income, operating_income, operating_margin. Raised by Benchmax agent-track review, AGENT_FINDINGS.md. Decided by Benchmax review (Rishabh Jain), 2026-09-06.

Gold before and after

Fieldv1v2also accepted
total_revenue90.45790.590.457
net_income26.10726.126.107
operating_income35.339null
operating_margin39.1null

Effect on scores · this document

Systembeforeafter
Codex · GPT-6 Astra · medium effort0.7141.000
Codex · GPT-5.6 Sol · medium effort0.7141.000
Claude Code · Claude Opus 5 · medium effort0.7860.929
Claude Code · Claude Sonnet 5 · medium effort0.7141.000

Regraded from saved outputs. No configuration was rerun.

Original gold

Field Gold v1 Source cited by gold
total_revenue 90.457 Page 22, non-GAAP reconciliation, "Total revenues 90,457" (thousands)
net_income 26.107 Page 18, five-quarter summary, "Net income 26,107" (thousands)
operating_income 35.339 Page 18, "Income before provision for income taxes 35,339"; gold rule marks it gold_unknowable
operating_margin 39.1 Computed as 35.339 / 90.457; gold rule marks it gold_unknowable

Evidence

Page 3, "Second Quarter 2023 Highlights", the deck's headline financial-highlights slide, prints "Revenue of $90.5 million, up 19%" and a "Net Income & EPS" tile of "$26.1 million". Page 18 prints the line "Income before provision for income taxes 35,339" and "Net income 26,107" in thousands. Page 22 prints "Total revenues 90,457" inside a non-GAAP reconciliation table. No line on any page is labeled operating income, operating profit, or income from operations.

The schema's own global convention (2) reads: "a currency metric presented as a rounded headline figure on a highlights/summary slide (e.g. '$2.1B') is captured as that rounded value (2100), not recomputed from reconciliation detail." The operating_income description reads: "Do NOT confuse with: ADJUSTED operating profit/margin, EBITDA, gross profit, pre-tax earnings, or net income." The operating_margin description defines it as GAAP operating_income / total_revenue and says to use null when no GAAP operating figure is shown.

Decision

Rationale

Gold v1 contradicts the schema it is paired with in two ways. It takes the reconciliation figures where the schema says to take the headline figures, and it fills operating_income with a line the schema excludes by name. The schema is the instruction the extractor is graded against; when gold and schema disagree, the schema governs and gold is corrected. Accepting the precise figures as alternates recognizes that a reader who reports 90.457 has not misread the document, only applied a different rounding convention than the one the schema asked for. Pre-tax income is different in kind from operating income for a bank, and the schema author anticipated exactly this confusion, so it is not made acceptable.

Effect on scores

Four of 14 cells. Under v1 every configuration scored 0.7143 or 0.7857 on this document (Opus used the precise figures). Under v2 a configuration that reported the headline figures and null operating fields scores 1.0; one that reported the precise figures and null operating fields also scores 1.0. A configuration that reported 35.339 as operating income is still marked wrong on that cell under v2. This dispute moves scores in the same direction for every configuration and is the largest single gold correction in the pilot.

Source: 002-byline-headline-rounding-and-operating-income.md and its gold patch. To contest this decision, open a new dispute referencing it.