Benchmax

PDF extraction › Document

healthcare · short document · 83 gold cells · real-source

Medicaid remittance advice

Gold v3 · imported · labels from llamaindex/ExtractBench · Source PDF ↗ · 1 page

Systemv3ExecutionRowsPass / fail / missing / extra
Codex · GPT-6 Astra · medium effort 0.964 completed 7 / 7 80 / 3 / 0 / 0
Codex · GPT-5.6 Sol · medium effort 0.964 completed 7 / 7 80 / 3 / 0 / 0
Claude Code · Claude Opus 5 · medium effort 0.964 completed 7 / 7 80 / 3 / 0 / 0
Claude Code · Claude Sonnet 5 · medium effort 0.964 completed 7 / 7 80 / 3 / 0 / 0

Pages

Medicaid remittance advice page 1
Page 1

Fields · gold v3 against every system

claims 3 rows

#claim_numberpatient_namerecipient_idpatient_account_numbermedical_record_numberclaim_from_dateclaim_to_datetotal_submittedtotal_paidtotal_patient_responsibilityclaim_statusreason_codes
0 5079171505699 wrong? DAVIS JANE wrong? 90000000A95001 wrong? 039634 wrong? null wrong? 060107 wrong? 061107 wrong? null wrong? null wrong? null wrong? APPROVES wrong? 0469 wrong?
1 5006170703899 wrong? JOHNSON MA wrong? 90000000A95001 wrong? 039305 wrong? null wrong? 040307 wrong? 040707 wrong? null wrong? null wrong? null wrong? DENIES wrong? 0036 wrong?
2 5033172401899 wrong? JONES DAVID wrong? 90000000A95001 wrong? 039357 wrong? null wrong? 041607 wrong? 042307 wrong? null wrong? null wrong? null wrong? SUSPENDS wrong? 0602 wrong?

payments 1 row

#check_numbercheck_amountra_datepayer_namepayee_namepayee_provider_numberpayee_npiclaim_type
0 39248026 wrong? null wrong? 01/01/08 wrong? CA MEDI-CAL wrong? CHILDREN'S HOSPITAL wrong? 0123456789 wrong? null wrong? MCARE CROSSOVER wrong?

service_lines 3 rows · differences

#claim_numberline_numberdate_of_service_fromdate_of_service_toprocedure_codemodifiersrevenue_codeunitscharge_amountallowed_amountpaid_amountline_statusreason_codes
0 5079171505699 wrong? null wrong? 060107 wrong? 061107 wrong? null wrong? null wrong? null wrong? null
claude-sonnet-5716
wrong?
null wrong? 716
gpt-6-astranull
gpt-5.6-solnull
claude-opus-5null
wrong?
null wrong? null wrong? 0469 wrong?
1 5006170703899 wrong? null wrong? 040307 wrong? 040707 wrong? null wrong? null wrong? null wrong? null
claude-sonnet-5696
wrong?
null wrong? 696
gpt-6-astranull
gpt-5.6-solnull
claude-opus-5null
wrong?
null wrong? null wrong? 0036 wrong?
2 5033172401899 wrong? null wrong? 041607 wrong? 042307 wrong? null wrong? null wrong? null wrong? null
claude-sonnet-5696
wrong?
null wrong? 696
gpt-6-astranull
gpt-5.6-solnull
claude-opus-5null
wrong?
null wrong? null wrong? 0602 wrong?

Values are shown after the scorer's normalization. Green: matches gold under the pinned scorer, including per-field normalizers and accepted alternates. Red: differs. Grey: absent on one side.

Extraction schema sent to every system · 8 descriptions contain a literal gold value

The upstream schema descriptions include examples that equal this document's answers in 8 places. All systems received the same schema.

{
  "$defs": {
    "Claim": {
      "properties": {
        "claim_from_date": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "First date of service for the claim. LABEL: \u0027DATES OF SERVICE\u0027 (from). Format MM/DD/YY[YY] or MMDDYY as printed. Null if absent. Transcribe EXACTLY as printed \u2014 do NOT add or remove slashes. The CA RAD service-date columns are headed literally \u0027MMDDYY\u0027 (six bare digits, no slashes); MS ENVISION prints slashed dates."
        },
        "claim_number": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Unique payer claim control number for this adjudicated claim. LABEL ALIASES: MS ENVISION \u0027TCN\u0027 (Transaction Control Number, ~17 digits); CA Medi-Cal RAD \u0027CLAIM CONTROL NUMBER\u0027/\u0027CCN\u0027 (~13 digits). Extract verbatim incl leading zeros. Do NOT confuse with patient_account_number or recipient_id."
        },
        "claim_status": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Adjudication status of the claim. MS: explicit \u0027STATUS\u0027 column (\u0027PAID\u0027/\u0027DENY\u0027). CA: the SECTION the claim sits under (\u0027APPROVES\u0027,\u0027DENIES\u0027,\u0027SUSPENDS\u0027,\u0027ADJUSTMENTS\u0027). Verbatim/uppercase. Null if indeterminable."
        },
        "claim_to_date": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Last date of service for the claim (\u0027DATES OF SERVICE\u0027 to). Same format. Null if absent. Transcribe EXACTLY as printed \u2014 do NOT add or remove slashes. The CA RAD service-date columns are headed literally \u0027MMDDYY\u0027 (six bare digits, no slashes); MS ENVISION prints slashed dates."
        },
        "medical_record_number": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Medical record number for the claim if printed (MS \u0027MED REC NO\u0027; CA \u0027MEDICAL RECORD NUMBER\u0027); else null."
        },
        "patient_account_number": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Provider\u0027s internal patient account/control number. LABEL ALIASES: MS \u0027PAT ACCT NUM\u0027; CA \u0027PATIENT CONTROL NUMBER\u0027. This is the PROVIDER\u0027s number, NOT the payer claim control number. Null if absent."
        },
        "patient_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Patient/recipient name as printed (typically LAST FIRST [MIDDLE]). LABEL ALIASES: MS \u0027BENEFICIARY NAME\u0027; CA \u0027RECIPIENT NAME\u0027. Verbatim as one string."
        },
        "reason_codes": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Claim-level denial/adjustment reason code(s) if printed at the claim row (CA \u0027RAD CODE\u0027; MS claim-level \u0027EXCEPTION CODES\u0027). State-specific 4-digit codes, space-separated if multiple, verbatim. Null if codes are only at the service line."
        },
        "recipient_id": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Member/recipient Medicaid identifier. LABEL ALIASES: MS \u0027MEDICAID ID\u0027; CA \u0027RECIPIENT MEDI-CAL ID NO.\u0027. Verbatim. Do NOT confuse with claim_number or patient_account_number."
        },
        "total_paid": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "description": "Total paid amount for the whole claim. LABEL ALIASES: MS \u0027TOT PAID AMT\u0027; CA \u0027PAID AMOUNT\u0027 total. Numeric. Null if blank."
        },
        "total_patient_responsibility": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "description": "Total patient responsibility for the claim. LABEL ALIASES: MS \u0027PAT RESP AMT\u0027; CA patient liability \u0027PAT LIAB\u0027/share-of-cost. Numeric. Null if absent."
        },
        "total_submitted": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "description": "Total billed/submitted amount for the whole claim. LABEL ALIASES: MS \u0027SUBMITTED AMT\u0027; CA claim \u0027TOTAL CHARGES\u0027/\u0027BILLED AMOUNT\u0027 total. Numeric. Null if not printed."
        }
      },
      "required": [
        "claim_number"
      ],
      "type": "object"
    },
    "Payment": {
      "properties": {
        "check_amount": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "description": "Total dollar amount of this remittance/warrant if printed as a single check/warrant total; else null. Numeric, no $ or commas."
        },
        "check_number": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Remittance/warrant identifier for this payment block. LABEL ALIASES: MS ENVISION \u0027REMITTANCE\u0027 (on the RA banner); CA Medi-Cal RAD \u0027WARRANT NO\u0027/\u0027WARRANT NO.\u0027. Extract verbatim incl leading zeros. Do NOT confuse with claim_number (TCN/CCN) or provider number."
        },
        "claim_type": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Claim/section type label printed in the header if present. CA \u0027CLAIM TYPE\u0027 (e.g. \u0027OUTPATIENT\u0027,\u0027INPATIENT\u0027,\u0027LONG TERM CARE\u0027,\u0027MCARE CROSSOVER\u0027); MS provider-type banner (e.g. \u0027INPATIENT\u0027,\u0027VISION AND HEARING\u0027). Else null."
        },
        "payee_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Name of the provider/facility receiving payment. MS: the SERVICE PROVIDER NAME / provider on the banner. CA: the \u0027TO:\u0027 provider name (e.g. \u0027ABC PROVIDER\u0027,\u0027CHILDREN\u0027S HOSPITAL\u0027). Do NOT put the provider NUMBER here."
        },
        "payee_npi": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "National Provider Identifier of the payee if printed (MS \u0027NPI NUMBER\u0027, 10 digits); else null. Do NOT confuse with payee_provider_number."
        },
        "payee_provider_number": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Provider number of the payee. LABEL ALIASES: MS \u0027PROVIDER NO\u0027; CA \u0027PROVIDER NUMBER\u0027. Digits, verbatim. Do NOT confuse with payee_npi."
        },
        "payer_name": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Primary payer/program name in the RA header. MS: \u0027MISSISSIPPI ENVISION MMIS\u0027 (use this system name, not the \u0027DIVISION OF MEDICAID\u0027 line). CA: \u0027CA MEDI-CAL\u0027. Verbatim."
        },
        "ra_date": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Date printed on the remittance advice header. LABEL ALIASES: MS \u0027DATE\u0027; CA \u0027DATE\u0027. Format MM/DD/YY or MM/DD/YYYY as printed. Do NOT confuse with a claim\u0027s DATES OF SERVICE."
        }
      },
      "required": [
        "check_number"
      ],
      "type": "object"
    },
    "ServiceLine": {
      "properties": {
        "allowed_amount": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "description": "Allowed/payable amount for the line. LABEL ALIASES: CA \u0027PAYABLE CHARGES\u0027/\u0027ALLOWED AMOUNT\u0027/\u0027MEDI-CAL ALLOWED\u0027. MS RA may not print a separate allowed -\u003e null."
        },
        "charge_amount": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "description": "Billed/charge amount for the line. LABEL ALIASES: MS \u0027LN SUBM AMOUNT\u0027; CA \u0027TOTAL CHARGES\u0027/\u0027BILLED AMOUNT\u0027 (row). Numeric. Null if absent."
        },
        "claim_number": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "The parent claim\u0027s control number (TCN/CCN) that THIS service/charge line belongs to. Must equal the claim_number of its claim. Used to group lines under claims."
        },
        "date_of_service_from": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Service line from-date. \u0027DATES OF SERVICE\u0027 on the line. MM/DD/YY[YY] or MMDDYY verbatim. Null if absent. Transcribe EXACTLY as printed \u2014 do NOT add or remove slashes. The CA RAD service-date columns are headed literally \u0027MMDDYY\u0027 (six bare digits, no slashes); MS ENVISION prints slashed dates."
        },
        "date_of_service_to": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Service line to-date. Null if absent or same single date. Transcribe EXACTLY as printed \u2014 do NOT add or remove slashes. The CA RAD service-date columns are headed literally \u0027MMDDYY\u0027 (six bare digits, no slashes); MS ENVISION prints slashed dates."
        },
        "line_number": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Line item number within the claim if printed (MS \u0027LINE\u0027 = 1,2,3...). CA RAD charge rows usually have no explicit line number -\u003e null."
        },
        "line_status": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Line adjudication status if printed (MS \u0027LN STATUS\u0027 = \u0027PAID\u0027/\u0027DENY\u0027). Null if absent."
        },
        "modifiers": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Procedure modifier(s) if present (MS \u0027M1 M2 M3 M4\u0027; CA \u0027PROCED CODE MODIFIER\u0027). Join multiple with a space. Null if none."
        },
        "paid_amount": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "description": "Paid amount for the line. LABEL ALIASES: MS \u0027LN PAID AMOUNT\u0027; CA \u0027PAID AMOUNT\u0027 (row). Numeric. Null if blank."
        },
        "procedure_code": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Procedure/service code on the line. LABEL ALIASES: MS \u0027PROC\u0027; CA \u0027PROC CODE\u0027/\u0027PROCED CODE\u0027; institutional CA \u0027ACCOM CODE\u0027 (accommodation/revenue). Core code only, verbatim. Null if absent."
        },
        "reason_codes": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Service-line denial/adjustment/exception reason code(s). LABEL ALIASES: MS \u0027EXCEPTION CODES\u0027 (4-digit, e.g. \u00270132 0439\u0027); CA \u0027RAD CODE\u0027 (4-digit, e.g. \u00270401\u0027). State-specific reason codes, NOT national CARC/RARC. Space-separated if multiple, verbatim. Null if none."
        },
        "revenue_code": {
          "anyOf": [
            {
              "type": "string"
            },
            {
              "type": "null"
            }
          ],
          "description": "Revenue code if printed (MS \u0027REVCD\u0027). Null if absent."
        },
        "units": {
          "anyOf": [
            {
              "type": "number"
            },
            {
              "type": "null"
            }
          ],
          "description": "Units/quantity for the line. LABEL ALIASES: MS \u0027LINE UNITS\u0027; CA \u0027QTY\u0027/\u0027DAYS OR VISIT\u0027. Numeric. Null if absent."
        }
      },
      "required": [
        "claim_number"
      ],
      "type": "object"
    }
  },
  "properties": {
    "claims": {
      "description": "Every adjudicated claim shown in the document, keyed by TCN(MS)/CCN(CA). Do not drop or merge claim rows.",
      "items": {
        "$ref": "#/$defs/Claim"
      },
      "type": "array"
    },
    "payments": {
      "description": "All remittance/warrant payment header blocks in the document (one per REMITTANCE/WARRANT NO).",
      "items": {
        "$ref": "#/$defs/Payment"
      },
      "type": "array"
    },
    "service_lines": {
      "description": "Every service/charge line across all claims, each carrying its parent claim_number. Do not drop lines.",
      "items": {
        "$ref": "#/$defs/ServiceLine"
      },
      "type": "array"
    }
  },
  "required": [
    "payments",
    "claims",
    "service_lines"
  ],
  "title": "StateMedicaidRemittanceAdvice",
  "type": "object"
}