PDF extraction › Document
energy · short document · 21 gold cells · real-source
Pepco residential bill
Gold v3 · imported · labels from llamaindex/ExtractBench · Source PDF ↗ · 3 pages
| System | v3 | Execution | Rows | Pass / fail / missing / extra |
|---|---|---|---|---|
| Codex · GPT-6 Astra · medium effort | 1.000 | completed | 1 / 1 | 21 / 0 / 0 / 0 |
| Codex · GPT-5.6 Sol · medium effort | 1.000 | completed | 1 / 1 | 21 / 0 / 0 / 0 |
| Claude Code · Claude Opus 5 · medium effort | 1.000 | completed | 1 / 1 | 21 / 0 / 0 / 0 |
| Claude Code · Claude Sonnet 5 · medium effort | 1.000 | completed | 1 / 1 | 21 / 0 / 0 / 0 |
Pages



Fields · gold v3 against every system
| Field | Gold · gpt-6-astra · gpt-5.6-sol · claude-opus-5 · claude-sonnet-5 |
|---|---|
| account_number | 0123 4567 890 wrong? |
| amount_due | 176.31 wrong? |
| bill_date | May 11, 2015 wrong? |
| billing_period_end | May 5, 2015 wrong? |
| billing_period_start | April 4, 2015 wrong? |
| customer_name | PEPCO CUSTOMER wrong? |
| invoice_number | 200000088589 wrong? |
| payment_due_date | Jun 1, 2015 wrong? |
| previous_balance | 287.32 wrong? |
| service_address | 123 9TH ST NW WASHINGTON DC 20012 wrong? |
| service_number | 0012 3456 7890 7001 6762 64 wrong? |
| total_current_charges | null wrong? |
| total_delivery_charges | 21.01 wrong? |
| total_supply_charges | 155.3 wrong? |
| utility_provider | Pepco wrong? |
meters 1 row
| # | meter_number | service_type | current_reading | previous_reading | usage_quantity | usage_unit |
|---|---|---|---|---|---|---|
| 0 | 1ND344415274 wrong? | electric wrong? | 57043 wrong? | 55628 wrong? | 1415.0 wrong? | kWh wrong? |
Values are shown after the scorer's normalization. Green: matches gold under the pinned scorer, including per-field normalizers and accepted alternates. Red: differs. Grey: absent on one side.
Extraction schema sent to every system · 43 descriptions contain a literal gold value
The upstream schema descriptions include examples that equal this document's answers in 43 places. All systems received the same schema.
{
"description": "A residential utility bill (electric and/or gas) from a single provider. Covers single-meter bills (Pepco electric, PPL electric) and dual-commodity bills (We Energies electric+gas). Top-level fields are flat bill-level identifiers and totals; per-meter usage is the repeated list `meters[]`.",
"properties": {
"account_number": {
"description": "The customer\u0027s utility account number. Aliases: \u0027Account Number\u0027, \u0027Account number\u0027, \u0027Account #\u0027. Format: VERBATIM as printed, preserving internal spaces and dashes (e.g. We Energies \u00270123456789-00003\u0027, Pepco \u00270123 4567 890\u0027). Do NOT confuse with: invoice_number (Pepco \u0027200000088589\u0027), service_number, the long barcode/scanline digit string at the bottom of a payment stub, or the meter number. Location: top summary table or the \u0027Account number:\u0027 label near the customer name; the payment stub repeats it.",
"type": "string"
},
"amount_due": {
"description": "The total amount due for this bill by the due date, as a number (strip \u0027$\u0027 and commas; e.g. We Energies 246.40, Pepco 176.31). Aliases: \u0027Amount Due\u0027, \u0027Total amount due by \u003cdate\u003e\u0027, \u0027Total Current Balance\u0027. Do NOT confuse with: the higher late-payment amount (Pepco \u0027amount due after Jun 1\u0027 179.06), previous_balance, total_current_charges (when it differs), or a single service subtotal. Location: top \u0027Amount Due\u0027 box or the \u0027Total amount due by \u003cdate\u003e\u0027 summary line.",
"type": "number"
},
"bill_date": {
"description": "The date the bill was issued/printed, VERBATIM as printed. Aliases: \u0027Bill Date\u0027, \u0027Bill Issue date\u0027, \u0027Statement Date\u0027. Examples: \u002702/03/2026\u0027, \u0027May 11, 2015\u0027. RULE: return null if the bill prints no explicit bill/issue/statement date \u2014 some bills (e.g. PPL) show only a Due Date and a billing period and have NO issue date; in that case do NOT substitute the due date, the billing-period end, the \u0027balance as of\u0027 date, or a meter-read date. Do NOT confuse with payment_due_date, billing_period_start/end, or the next-meter-read date. Location: top summary table (\u0027Bill Date\u0027) or the \u0027Bill Issue date:\u0027 label.",
"type": [
"string",
"null"
]
},
"billing_period_end": {
"description": "End date of the service/billing period this bill covers. Aliases: \u0027Bill Period\u0027 (second date), \u0027for the period\u0027 (second date), \u0027Usage from\u0027 (second date). Format: VERBATIM as printed; e.g. We Energies \u002702/02/2026\u0027, Pepco \u0027May 5, 2015\u0027. Do NOT confuse with bill_date, payment_due_date, or the next-meter-read date. Location: the \u0027Bill Period: X to Y\u0027 / \u0027for the period X to Y\u0027 line \u2014 take the SECOND (later) date.",
"type": "string"
},
"billing_period_start": {
"description": "Start date of the service/billing period this bill covers. Aliases: \u0027Bill Period\u0027 (first date), \u0027for the period\u0027 (first date), \u0027Usage from\u0027 (first date). Format: VERBATIM as printed; e.g. We Energies \u002701/05/2026\u0027, Pepco \u0027April 4, 2015\u0027. Do NOT confuse with bill_date or billing_period_end. Location: the \u0027Bill Period: X to Y\u0027 / \u0027for the period X to Y\u0027 line \u2014 take the FIRST (earlier) date.",
"type": "string"
},
"customer_name": {
"description": "The name of the customer/account holder the bill is addressed to. Aliases: \u0027Customer Name\u0027, \u0027Service to\u0027, \u0027Bill to\u0027. Format: verbatim as printed (e.g. We Energies \u0027JOHN Q CUSTOMER\u0027, Pepco \u0027PEPCO CUSTOMER\u0027). Do NOT confuse with the utility_provider name or the remittance payee. Location: the addressee block near the service address or above the service-address label.",
"type": "string"
},
"invoice_number": {
"description": "The bill/invoice number uniquely identifying this statement, ONLY if the bill prints an explicit \u0027Invoice Number\u0027/\u0027Bill Number\u0027 label. Example: Pepco \u0027200000088589\u0027 (printed as \u0027Invoice Number: 200000088589\u0027 on the coupon tear line). Format: verbatim digits as printed. Do NOT confuse with account_number, service_number, or an archive/page/system tag \u2014 in particular the We Energies \u0027WEC_PDF_Out_Archive 00001\u0027 string is an internal archive tag, NOT an invoice number, so We Energies invoice_number is null. Return null when no field is explicitly labeled invoice/bill number. Location: near the \u0027Please tear on the dotted line\u0027 coupon line or the bill header.",
"type": [
"string",
"null"
]
},
"meters": {
"description": "One entry per metered service on the bill. Single-meter electric bills have one entry (Pepco: one electric meter); dual-commodity bills have one electric + one gas entry (We Energies: meter NZT0123456 electric + meter 0WG0123456 gas). Aligned by meter_number.",
"items": {
"properties": {
"current_reading": {
"description": "The most recent (ending) meter reading for this period, as a number (drop any leading zeros\u0027 formatting; the numeric value). Aliases: \u0027Current Reading\u0027, \u0027Actual Reading \u003clater date\u003e\u0027. Example: We electric 40705, We gas 1464, Pepco 57043 (printed \u0027057043\u0027). Do NOT confuse with previous_reading (the earlier/beginning reading) or usage_quantity (the difference). Return null if no numeric reading is printed. Location: the meter-reading rows/table for this meter \u2014 the later of the two read dates (\u0027Current Reading\u0027 column).",
"type": [
"number",
"null"
]
},
"meter_number": {
"description": "The meter identifier for this metered service. Aliases: \u0027Meter\u0027, \u0027Meter Number\u0027, \u0027Meter No\u0027. Format: VERBATIM alphanumeric as printed (e.g. We Energies \u0027NZT0123456\u0027 / \u00270WG0123456\u0027, Pepco \u00271ND344415274\u0027). Do NOT confuse with account_number, service_number, or the rate/schedule code (e.g. \u0027Rg1\u0027, \u0027Residential-R\u0027). Location: the per-service block (\u0027Meter \u003cid\u003e\u0027) or the \u0027Meter Number\u0027 column of the usage table.",
"type": "string"
},
"previous_reading": {
"description": "The prior (beginning) meter reading for this period, as a number (the numeric value). Aliases: \u0027Previous Reading\u0027, \u0027Actual Reading \u003cearlier date\u003e\u0027. Example: We electric 40036, We gas 1385, Pepco 55628 (printed \u0027055628\u0027). Do NOT confuse with current_reading (the later reading) or usage_quantity. Return null if not printed. Location: the meter-reading rows/table for this meter \u2014 the earlier of the two read dates (\u0027Previous Reading\u0027 column).",
"type": [
"number",
"null"
]
},
"service_type": {
"description": "The commodity this meter measures. \u0027electric\u0027 for kWh electric service, \u0027gas\u0027 for natural-gas service (therms/CCF), \u0027water\u0027 for water. Infer from the service section heading (\u0027Electric Service\u0027, \u0027Gas Service\u0027, \u0027Residential Electric Service\u0027, \u0027Use (kWh)\u0027). Do NOT guess: choose by the section the meter appears under and its usage unit. We Energies meter NZT0123456 is electric; meter 0WG0123456 is gas.",
"enum": [
"electric",
"gas",
"water"
],
"type": "string"
},
"usage_quantity": {
"description": "The billed consumption for this meter over the period, as a number. For electric use the billed kWh; for gas use the billed therms (the unit the gas charges are calculated on), NOT the raw CCF reading difference. Example: We electric 669 (kWh), We gas 83.9 (therms \u2014 the \u002779 CCF x 1.062 BTU = 83.9 Therms\u0027 converted value, NOT 79), Pepco 1415 (kWh). Do NOT confuse with current_reading/previous_reading or the raw reading difference when a meter constant/CCF\u2192therms conversion applies. Location: the \u0027Total Use\u0027/\u0027KWH\u0027/\u0027Therms Used\u0027/\u0027Total Difference\u0027 figure for this meter.",
"type": "number"
},
"usage_unit": {
"description": "The unit of usage_quantity. \u0027kWh\u0027 for electric, \u0027therms\u0027 for gas billed on therms, \u0027CCF\u0027 only if the gas bill is billed on CCF with no therms conversion, \u0027gallons\u0027 for water. Must match the unit usage_quantity is expressed in. For We Energies gas, usage_quantity is 83.9 therms so usage_unit=\u0027therms\u0027 (NOT \u0027CCF\u0027, even though the raw reading difference is 79 CCF).",
"enum": [
"kWh",
"therms",
"CCF",
"gallons"
],
"type": "string"
}
},
"required": [
"meter_number",
"service_type",
"usage_quantity",
"usage_unit"
],
"type": "object"
},
"type": "array"
},
"payment_due_date": {
"description": "The date payment is due. Aliases: \u0027Due Date\u0027, \u0027Total amount due by\u0027, \u0027Payment Transfer Date\u0027 (for autopay-withdrawal bills). Format: VERBATIM as printed; e.g. Pepco \u0027Jun 1, 2015\u0027 (from \u0027Total amount due by Jun 1, 2015\u0027). Do NOT confuse with bill_date or billing_period_end, and do NOT use the later late-payment date (\u0027after Jun 1, 2015\u0027). For We Energies (autopay) use the \u0027Payment Transfer Date\u0027 \u002702/25/2026\u0027. Location: top summary box (\u0027Due Date\u0027/\u0027Payment Transfer Date\u0027) or the \u0027Total amount due by \u003cdate\u003e\u0027 line.",
"type": "string"
},
"previous_balance": {
"description": "The balance carried from the prior bill before this period\u0027s payments/charges, as a number. Aliases: \u0027Previous Balance\u0027, \u0027Balance from your last bill\u0027. Example: We Energies 277.79, Pepco 287.32. Do NOT confuse with amount_due, balance_forward ($0.00 after payment), or total_current_charges. Use the TOP-LEVEL bill summary value, NOT a per-service sub-summary\u0027s \u0027Balance from last bill\u0027. Return null if not printed. Location: the \u0027Activity Since Last Bill\u0027 / \u0027Summary of your charges\u0027 / \u0027Billing Summary\u0027 block.",
"type": [
"number",
"null"
]
},
"service_address": {
"description": "The address where utility service is delivered. Aliases: \u0027Service Address\u0027, \u0027Your service address\u0027, \u0027Service to\u0027. Format: full street address as printed, single line, including city/state/ZIP if shown (e.g. We Energies \u0027123 MAIN ST NEENAH WI 54956-1802\u0027, Pepco \u0027123 9TH ST NW WASHINGTON DC 20012\u0027). Do NOT confuse with the utility\u0027s remittance/PO-Box mailing address or the regulator addresses. Location: under the \u0027Service Address\u0027/\u0027Your service address\u0027 label near the customer name.",
"type": "string"
},
"service_number": {
"description": "The service / point-of-delivery (POD) number distinct from the account number, ONLY if printed. Aliases: \u0027service number\u0027, \u0027Service No\u0027, \u0027POD ID\u0027. Example: Pepco \u00270012 3456 7890 7001 6762 64\u0027 (printed as \u0027Residential-R - service number ...\u0027 on the \u0027Details of your Electric Charges\u0027 page). Format: verbatim including spaces. Do NOT confuse with account_number or meter_number. Return null if the bill shows no separate service number (e.g. We Energies prints none \u2192 null). Location: the per-service detail section header.",
"type": [
"string",
"null"
]
},
"total_current_charges": {
"description": "The total NEW charges for the current billing period, taken VERBATIM from a single line that the bill explicitly labels as the combined current-period charges. Accepted labels: \u0027Total Current Charges\u0027, \u0027Total Current Balance\u0027 (We Energies), or an equivalently-labeled single combined-charges total. Example: We Energies \u0027Total Current Charges $246.40\u0027 -\u003e 246.40. RULE: return null if the bill shows no such single combined-current-charges line \u2014 do NOT compute it by summing delivery + supply, and do NOT substitute the amount due or a single-service subtotal. Pepco (separate \u0027New electric distribution charges\u0027 + \u0027New WGL Energy Svcs supply charges\u0027, no combined current-charges line) -\u003e null. PPL (separate \u0027Total Delivery Charges\u0027 + \u0027Total Supply Charges\u0027, no combined current-charges line) -\u003e null. Do NOT confuse with amount_due, previous_balance, total_delivery_charges, or total_supply_charges. Location: the charges-summary / billing-summary block.",
"type": [
"number",
"null"
]
},
"total_delivery_charges": {
"description": "Total distribution/delivery charges billed by the regulated utility for the current period, as a number, ONLY when the bill cleanly separates delivery from supply. Aliases: \u0027Total Delivery Charges\u0027, \u0027Total Electric Delivery Charges\u0027, \u0027New electric distribution charges\u0027. Example: Pepco 21.01 (\u0027New electric distribution charges - Pepco\u0027). Return null when the bill does NOT split delivery vs supply: We Energies shows commodity totals (Electric Service Total + Gas Service Total) with no delivery/supply split \u2192 null. Do NOT use a commodity subtotal (e.g. We Energies gas distribution line) as the delivery total. Location: the delivery/distribution charges subtotal.",
"type": [
"number",
"null"
]
},
"total_supply_charges": {
"description": "Total supply/generation charges (often billed for a competitive/third-party supplier) for the current period, as a number, ONLY when the bill cleanly separates supply from delivery. Aliases: \u0027Total Supply Charges\u0027, \u0027supply charges\u0027, \u0027Generation \u0026 Transmission Charges\u0027. Example: Pepco 155.30 (\u0027New WGL Energy Svcs supply charges\u0027). Return null when the bill does NOT split supply vs delivery (bundled): We Energies is bundled (no delivery/supply split) \u2192 null. Do NOT use a commodity subtotal (e.g. We Energies \u0027Gas Supply Service\u0027 line) as the supply total. Location: the supply/generation charges subtotal.",
"type": [
"number",
"null"
]
},
"utility_provider": {
"description": "Name of the utility company that issued the bill. Aliases: \u0027utility\u0027, \u0027company\u0027, the brand printed beside the bill title and in the payment remittance address. Examples: \u0027We Energies\u0027, \u0027Pepco\u0027, \u0027PPL Electric Utilities\u0027. CASING: use the Title-Case brand exactly as it appears in the running body text and the remittance/return-payment block; do NOT copy the stylized all-lowercase logo wordmark (e.g. the Pepco logo renders \u0027pepco\u0027 but the company is \u0027Pepco\u0027). Do NOT confuse with the competitive generation supplier (e.g. \u0027WGL Energy Svcs\u0027 on Pepco supply pages; the separate supplier referenced on PPL) \u2014 that is the third-party supplier, not the issuing distribution utility. Location: bill header/title and the \u0027make payable to\u0027 / remittance block.",
"type": "string"
}
},
"required": [
"utility_provider",
"account_number",
"customer_name",
"service_address",
"billing_period_start",
"billing_period_end",
"payment_due_date",
"amount_due",
"meters"
],
"title": "UtilityBill",
"type": "object"
}