PDF extraction › Byline Bancorp earnings report › Dispute
Dispute 002 · accepted · produces gold v2 · decided 2026-09-06
Byline Bancorp Q2 2023: headline rounding rule and pre-tax income labeled as operating income
Fields: total_revenue, net_income, operating_income, operating_margin. Raised by Benchmax agent-track review, AGENT_FINDINGS.md. Decided by Benchmax review (Rishabh Jain), 2026-09-06.
Gold before and after
| Field | v1 | v2 | also accepted |
|---|---|---|---|
| total_revenue | 90.457 | 90.5 | 90.457 |
| net_income | 26.107 | 26.1 | 26.107 |
| operating_income | 35.339 | null | |
| operating_margin | 39.1 | null |
Effect on scores · this document
| System | before | after |
|---|---|---|
| Codex · GPT-6 Astra · medium effort | 0.714 | 1.000 |
| Codex · GPT-5.6 Sol · medium effort | 0.714 | 1.000 |
| Claude Code · Claude Opus 5 · medium effort | 0.786 | 0.929 |
| Claude Code · Claude Sonnet 5 · medium effort | 0.714 | 1.000 |
Regraded from saved outputs. No configuration was rerun.
Original gold
| Field | Gold v1 | Source cited by gold |
|---|---|---|
total_revenue |
90.457 | Page 22, non-GAAP reconciliation, "Total revenues 90,457" (thousands) |
net_income |
26.107 | Page 18, five-quarter summary, "Net income 26,107" (thousands) |
operating_income |
35.339 | Page 18, "Income before provision for income taxes 35,339"; gold rule marks it gold_unknowable |
operating_margin |
39.1 | Computed as 35.339 / 90.457; gold rule marks it gold_unknowable |
Evidence
Page 3, "Second Quarter 2023 Highlights", the deck's headline financial-highlights slide, prints "Revenue of $90.5 million, up 19%" and a "Net Income & EPS" tile of "$26.1 million". Page 18 prints the line "Income before provision for income taxes 35,339" and "Net income 26,107" in thousands. Page 22 prints "Total revenues 90,457" inside a non-GAAP reconciliation table. No line on any page is labeled operating income, operating profit, or income from operations.
The schema's own global convention (2) reads: "a currency metric presented as
a rounded headline figure on a highlights/summary slide (e.g. '$2.1B') is
captured as that rounded value (2100), not recomputed from reconciliation
detail." The operating_income description reads: "Do NOT confuse with:
ADJUSTED operating profit/margin, EBITDA, gross profit, pre-tax earnings, or
net income." The operating_margin description defines it as GAAP
operating_income / total_revenue and says to use null when no GAAP operating
figure is shown.
Decision
total_revenue: expected becomes 90.5 per the schema's rounding rule; 90.457 is retained as an acceptable alternate, because it is also a correct figure printed in the document.net_income: expected becomes 26.1; 26.107 retained as an alternate, same reasoning.operating_income: expected becomes null. 35.339 is pre-tax income, which the schema explicitly excludes. It is not accepted as an alternate.operating_margin: expected becomes null, following from the above.
Rationale
Gold v1 contradicts the schema it is paired with in two ways. It takes the
reconciliation figures where the schema says to take the headline figures, and
it fills operating_income with a line the schema excludes by name. The
schema is the instruction the extractor is graded against; when gold and
schema disagree, the schema governs and gold is corrected. Accepting the
precise figures as alternates recognizes that a reader who reports 90.457 has
not misread the document, only applied a different rounding convention than
the one the schema asked for. Pre-tax income is different in kind from
operating income for a bank, and the schema author anticipated exactly this
confusion, so it is not made acceptable.
Effect on scores
Four of 14 cells. Under v1 every configuration scored 0.7143 or 0.7857 on this document (Opus used the precise figures). Under v2 a configuration that reported the headline figures and null operating fields scores 1.0; one that reported the precise figures and null operating fields also scores 1.0. A configuration that reported 35.339 as operating income is still marked wrong on that cell under v2. This dispute moves scores in the same direction for every configuration and is the largest single gold correction in the pilot.
Source: 002-byline-headline-rounding-and-operating-income.md and its gold patch. To contest this decision, open a new dispute referencing it.