PaperlayerView exact revision on arXiv

Source-linked AI summary

Estimating Uncoded Crash Factors with Tabular Foundation and System One Models: Kumo Tabular and Jev

Amir Rafe, Subasish Das

arXiv:2610.10321v1stat.APcs.LGstat.ME

TL;DR

Road safety counts rely on coded crash fields, although narratives can document factors those fields omit, and sampled narrative reads alone do not yield statewide counts. The paper combines table and narrative readers with human judgments and a sampling design; narratives exceed coded fields for several factors, and a probability-sampled human check agrees with all fifteen estimates within their margins.

  • Problem

    Coded fields and narratives can differ, but sampled narrative reads need a sampling design and human reference to support statewide counts of what narratives state.

    Source: §1 · pp. 1–2

  • Method

    The system uses Kumo Tabular on every coded record, Jev on two probability samples of narratives, human recalibration, a multiwave estimator, and a second human check.

    Source: §1 · p. 2, §3 · p. 4, §4.1 · p. 8

  • Results

    Narratives document more injury crashes than coded fields for five factors, the human check agrees with all counts within sampling margins, and the re-read list finds discordance 7 to 58 times as often as random reading.

    Source: §7 · pp. 18–19

  • Takeaways & Limitations

    Two calibrated readers of different views and a sampling design provide a safety office population counts, a discordance map, a validated re-read list, and a reading budget with tier-by-tier uncertainty.

    Source: §7 · p. 19

    The study covers one state; Jev is a closed commercial model with undisclosed training data, and extrapolations to crashes without usable narratives are model-based.

    Source: §7 · p. 19

Abstract

from arXiv · show

Road safety programs count the coded fields of police crash records, while the officer's narrative, which often records factors the fields omit, is rarely read. A safety office thus cannot tell how much its counts miss or where to review. This study develops and evaluates a system that joins both views of the 5,601,890 Texas crashes from 2017 to 2025 into population estimates with stated validity. An in-context tabular foundation model, Kumo Tabular, reads the coded record of every crash, a calibrated System One model, Jev, reads the narratives of two probability samples, and human judgments recalibrate its probabilities. A multiwave predict-then-debias estimator joins the three tiers, and a second human tier drawn with recorded probabilities checks the estimates by design. For hydroplaning, medical episodes, fatigue, animals, and phone use, the narrative documents more injury crashes than the coded field, 15,074 against 7,340 for phone use, and the human check agrees with all fifteen estimates within its margin. A re-read list ranked by Kumo Tabular finds confirmed discordance 7 to 58 times as often as random reading. At the planning cost of human coding, one further round of human judgments would cut the root mean square relative half-width from 22.0 to 16.2 percent, against 21.2 for reading every narrative. Two calibrated readers of different views, joined by a sampling design, give a safety office counts, a discordance map, a validated re-read list, and a reading budget, with Kumo Tabular reading the table at 15 times the speed of TabPFN 3.5.

1 Introduction

The paper addresses the lack of a statewide, design-based way to compare coded crash fields with what narratives document. It joins tabular and narrative readers with human judgments to estimate population counts and locate confirmed disagreement.

1 Introduction

  • Coded fields and narratives can record different factors for the same crash, so coded-field counts alone cannot reveal how much the narrative documents.A sampled model reading cannot become a statewide count without a sampling design and a human reference for what the narrative states.Source: §1 · pp. 1–2
  • No prior study linked both views of the same crash under one sampling design to estimate statewide discordance, map it, validate re-read lists, and allocate reading budgets.These are the quantities the authors identify as necessary for deciding which fields to revise, records to review, and where to spend the next reading budget.Source: §1 · p. 2
  • The study evaluates the system on Texas crashes from 2017 to 2025 and asks how well coded records predict narrative factors, where views disagree, and how readers compare.Its questions also cover factor counts, re-read-list performance, and future reading-budget allocation.Source: §1 · p. 2
  • The paper estimates Texas crash populations by combining a table reader for every crash, narrative reads from probability samples, and human judgments that recalibrate the narrative model.A multiwave predict-then-debias estimator joins the tiers, while a second probability-sampled human tier checks estimates by design.Source: §1 · p. 2

2 Background

Prior crash-data, narrative-model, inference, and tabular-model research each addresses only part of the problem. The paper identifies a gap in joining the record’s two views under one sampling design for population estimates and validated discordance review.

2 Background

  • Police crash records under-report casualties, and factor accuracy varies with what officers can establish; Texas coding rules can leave some evidence in narratives without making the coded field erroneous.Prior studies compare police records with hospital, investigator, or test records, and one proxy definition found nearly three times as many sleep-related involvements as police reports.Source: §2 · p. 3
  • Narrative-model studies generally report sample agreement or prevalence, while two Texas studies report statewide counts; none uses the coded record as a second reader.Agreement with coded labels need not match expert agreement, and earlier Arkansas results found attribute differences exceeded model differences [5].Source: §2 · p. 3
  • Prediction-powered inference corrects model bias with trusted measurements, and multiwave extensions handle adaptive samples, but prior methods do not use two machine readers of different views as nested phases.Related approaches include cross-prediction, active inference, and design-based supervised learning with probability-sampled gold subsets.Source: §2 · p. 3
  • Tabular foundation models predict rows from labeled context, but prior crash applications use tabular models for prediction rather than inference; benchmarks also report weaker performance under temporal and grouped shifts.The study comparison identifies no prior work that links both record views under a sampling design, maps confirmed discordance, and validates a re-read list on a probability sample.Source: §2 · pp. 3–4

3 Data and design

The analysis defines a geolocated, sufficiently long-narrative population and keeps crashes outside it separate. Its design combines table predictions for all crashes, sampled narrative-model reads, and human judgments.

3 Data and design

  • The narrative population contains 5,018,080 geolocated Texas crashes with narratives longer than 40 characters, or 89.6 percent of the 5,601,890 crashes.All estimates refer to this population or a domain within it; crashes outside it are reported separately.Source: §3 · p. 4
  • Outside the narrative population, 484,462 crashes had narratives but no coordinates and 99,332 had empty or short narratives; these crashes had a 0.17 percent fatal share versus 0.64 percent inside.The excluded groups nevertheless include 997 fatal crashes.Source: §3 · p. 4
  • The table reader receives 97 crash, unit, and person features; identifiers, coordinates, and narrative text are excluded, and dates are limited to year.Missing cells remain missing.Source: §3 · p. 4
  • The design uses table predictions for every crash, narrative-model reads from probability samples of 150,000 and 79,510, and 2,309 usable human judgments on 400 narratives.Two or three coders, blinded to model output, reached a consensus label for each narrative and factor; a second human tier checks estimates using recorded probabilities.Source: §3 · p. 4

4 Methods

The methods combine coded-record probabilities for every crash, sampled narrative reads recalibrated with human judgments, and a multiwave estimator; a second probability-sampled human tier checks the estimates.

4.1 Readers, estimator, and human check

  • Kumo Tabular reads coded records, Jev reads sampled narratives, and a human tier recalibrates Jev’s probabilities before the estimator joins the views.Kumo Tabular uses a 60,000-row context; Jev is run with a typed schema and gating rule.Source: §4 · p. 5, §4.1 · pp. 6–7

4 Methods

  • Wave 1 is a simple random sample of 150,000 narratives from U; wave 2 samples the remaining narratives using the table reader’s discordance score.Wave-specific inverse-probability weights are combined with fixed shares.Source: §4.1 · p. 7

4.1 Readers, estimator, and human check

  • The multiwave estimator anchors on the table reader across the population and corrects its bias using weighted measured narratives.Its variance-tuned parameter is restricted to [0,1], and the intervals include uncertainty from human recalibration and map selection.Source: §4.1 · p. 8
  • A second Poisson-sampled human tier checks estimates using narratives not judged in phase 2, with recorded inclusion probabilities and sampling after estimates are fixed.The correction targets the human criterion where the narrative reader’s probability is at least 0.05; below that, the share is at most 0.6% except for wrong-way driving and vehicle defects.Source: §4.1 · p. 8

4.2 Second wave, outputs, and table-reader evaluation

  • The second wave targets narrative surplus, defined as a human-confirmed factor stated in the narrative but absent from its coded field.Discordance is not treated as coding error because fields can lag narratives for procedural reasons.Source: §4.2 · pp. 8–9
  • The atlas reports domain totals and confirmed-discordance rates across year, road system, population group, setting, severity, road class, and county.Counties are pooled when county crash count multiplied by the statewide rate is below ten.Source: §4.2 · p. 9
  • The re-read list ranks unread narratives by Kumo Tabular’s discordance probability, with precision and lift estimated using recorded second-wave probabilities.The estimates are design-based and confirmed through the human-recalibrated reader.Source: §4.2 · p. 9
  • The evaluation compares foundation models and CatBoost on nested context sizes, measuring discrimination, calibration, throughput, and memory.Reading and human-coding purchases reduce different variance components, and their budget split follows a two-phase Neyman allocation.Source: §4.2 · p. 9

5 Results

The narrative adds factor information not fully captured by coded fields, and human checks support the resulting crash estimates. Confirmed discordance varies by setting, while the re-read list and budget analysis identify efficient uses of review effort.

5.1 What the narrative carries and how many crashes it documents

  • Kumo Tabular’s full coded record improves AUC over sixteen indicators most for phone use (0.72 to 0.89) and unbelted occupants (0.64 to 0.77).The full record also predicts hit-and-run at 0.93 and hospital transport at 0.91.Source: §5.1 · p. 10, Table 3 · p. 11
  • 15,074 phone-use and 28,345 medical-episode injury-or-fatal crashes had narrative statements, versus coded counts of 7,340 and 16,193, respectively.The design-based human check agrees with all fifteen model estimates within its 95 percent margin; the five factors above coded counts remain above them under the check.Source: §5.1 · pp. 10–11, Table 4 · p. 12
  • The recalibration tier accounts for 95 percent of alcohol variance and 99 percent of wrong-way-driving variance; the table term removes up to 64 percent of design half-width for animals.The second wave narrows phone-use Wald half-width from 16.8 to 16.0 percent, while total half-width narrows by 2 percent for the median factor.Source: §5.1 · p. 11, Figure 4 · p. 13

5.2 Discordance, reading priorities, and the table reader

  • Confirmed discordance ranges from 15 percent for animals to 69 percent for wrong-way driving, with animals’ rate 3.0 times higher in rural than urban crashes.Discordance is also higher on state highways than local roads for hydroplaning and fatigue; 2021-to-2019 intervals include one for every factor.Source: Table 5 · p. 13, §5.2 · pp. 13–14, Figure 6 · p. 15
  • At the top one percent of unread narratives, the re-read list’s lift over random reading ranges from 7 for unbelted occupants to 58 for animals.For alcohol, 31.6 percent of listed records are confirmed discordant, versus 0.76 percent under random reading.Source: §5.2 · p. 14, Figure 7 · p. 16
  • Kumo Tabular small scores 15 times as many rows per second as TabPFN 3.5, while TabPFN 3.5 has mean flag AUC 0.872 versus 0.865 on the same test rows.Across 18 targets, Kumo Tabular small’s mean flag AUC is 0.890 versus 0.881 for a tuned gradient-boosted tree.Source: §5.2 · p. 15, Table 6 · p. 16
  • 22.0 percent current relative half-width falls to 16.2 percent after 400 more human-coded narratives, compared with 21.2 percent after reading every remaining narrative.At the planning cost of $0.60 per human-coded narrative, the further human round buys more precision per dollar than full reading.Source: §5.2 · p. 14, Figure 8 · p. 17

6 Discussion

The coded record predicts narrative readings when evidence is routine, but narrative surplus varies substantially by factor. Human recalibration drives much of the uncertainty, while reader performance, re-read targeting, and budget allocation shape practical use.

6 Discussion

  • Across factors, coded fields predict narrative readings closely when evidence is routine, whereas narratives carry most information when evidence depends on what an officer observes or hears.The paper gives this factor-level contrast a quantitative form at state scale.Source: §6 · p. 17
  • Narrative surplus varies by factor: it is greater for phone use, fatigue, hydroplaning, medical episodes, and animals, while coded fields record alcohol and drugs at least as often.Phone use has about twice as many injury crashes documented in narratives as coded in fields.Source: §6 · p. 17
  • Unbelted-driver discordance requires human confirmation: coders confirmed about one in seven judged narrative flags, versus about nine in ten for phone use and fatigue.An unconfirmed discordance count can overstate narrative surplus for some factors several times over.Source: §6 · pp. 17–18
  • The recalibration tier carries most uncertainty for every factor, so the table reader contributes chiefly by targeting the second wave and ranking records for re-reading.A second human tier of a few hundred specialist-coded narratives agreed with every estimate within its sampling margin.Source: §6 · p. 18
  • Kumo Tabular small matched or exceeded a tuned booster on every target across five folds, while TabPFN 3.5 discriminated slightly better on this crash table.Because reader correlations with recalibrated narrative probabilities were nearly equal, throughput, hardware, and license guide reader choice.Source: §6 · p. 18
  • The atlas, ranked re-read list, and allocation rule respectively indicate where to investigate discordance, which unread records to review, and how to spend the next reading budget.The list ranks records by the probability that the narrative states a factor omitted by the field.Source: §6 · p. 18

7 Conclusion

The system estimates how coded crash records and narratives differ across Texas crashes, combining two readers with human review and sampling-based inference. Estimates cover crashes with usable narratives, and the authors identify reader, state, and human-tier boundaries for extending the work.

7 Conclusion

  • 5,601,890 Texas crashes were analyzed, with estimates over the 5,018,080 crashes carrying usable narratives; narratives documented more injury crashes for five factors, while fields recorded alcohol, drugs, and unbelted occupants at least as often.A second human tier agreed with every count within its sampling margin, and the atlas located confirmed discordance by setting, road system, and county.Source: §7 · pp. 18–19
  • 7 to 58 times as often as random reading, the re-read list found confirmed discordance at the top one percent; at the planning cost of human coding, another human-judgment round buys more precision than narrative reads for every factor.Kumo Tabular small matched or exceeded a tuned booster and came close to TabPFN 3.5 while scoring rows 15 times faster.Source: §7 · p. 19
  • The narrative reader is a closed commercial model with undisclosed training data, and estimates for crashes without usable narratives are model-based and reported separately.The study covers one state.Source: §7 · p. 19
  • Repeating the design in other states and across reader versions would test geographic and reader stability; a larger human tier could narrow intervals dominated by recalibration.Reading narratives of crashes without coordinates after a cleaning pass could bring that extrapolated group into the measured population.Source: §7 · p. 19
  • The crash-record extract cannot be redistributed under the data agreement, while analysis code, question schemas, coder guidance, design summaries, and aggregate result tables are available.Aggregate CRIS data are public, and bulk extracts can be requested.Source: Data and code availability · p. 19

Ethics

The IRB determined that this secondary analysis of administrative records was not human subjects research. Identifiers were replaced before transfer, and no narrative is reproduced in the paper.

Ethics

  • The IRB determined that the secondary analysis of administrative records was not human subjects research.Source: Ethics · p. 19
  • Names, telephone numbers, and other identifiers were replaced with placeholders before transfer; only cleaned narratives went to the narrative reader.Source: Ethics · p. 19

Funding

  • The work received no specific funding, and TabPFN was used under its evaluation license.Source: Funding · p. 19

Author contributions

  • Amir Rafe conceived the study, designed methods, analyzed data, and wrote the manuscript; Subasish Das contributed to design, provided data, and reviewed and edited it.Source: Author contributions · p. 19

Competing interests

The authors declare no competing interests, financial or advisory relationships with either company, or company involvement in credits, early access, or review.

Competing interests

A The coded feature set

The table reader uses 97 coded-record features spanning crash, unit, contributing-factor, person, and binary-indicator information, with narrative text and identifying details excluded.

A The coded feature set

  • 97 coded-record features comprise 32 crash fields, 18 unit fields, 12 contributing-factor families, 26 person fields, and nine binary factor indicators.The indicator set also contains seven of sixteen indicators that duplicate columns above.Source: A The coded feature set · pp. 22–23
  • Categorical labels Unknown, Not Reported, and No Data are treated as missing.Source: A The coded feature set · p. 23
  • Narrative text and identifiers—including coordinates, license numbers, birth dates, ZIP codes, and ethnicity—are never read.The released code lists each feature's source field and flattening rule.Source: A The coded feature set · p. 23

B Estimator, recalibration, and their uncertainty

The estimator combines table-reader and narrative-reader quantities across sampling waves, with recalibration and uncertainty procedures for population totals. The section also describes how recalibration maps and human-tier designs are evaluated.

B Estimator, recalibration, and their uncertainty

  • In the estimator notation, tildes mark recalibrated quantities and hats mark table-reader outputs.Source: B Estimator, recalibration, and their… · p. 23
  • The multiwave estimator is consistent for the recalibrated total when each wave term has conditional expectation one; cross-fitting separates each wave-1 prediction from its crash's measurement.The five fold models differ little, supporting the stated stability condition for cross-prediction guarantees.Source: B Estimator, recalibration, and their… · p. 23
  • Table B.2 reports recalibration maps and Jev-flag confirmation by coded-field status, with confirmation shares based on few flags and wide intervals.The table note defines confirmation as the calibration-weighted share of Jev flags at p≥0.5 confirmed by the human tier.Source: B Estimator, recalibration, and their… · p. 24, Table B.2 · p. 24
  • The variance-minimizing Ω is estimated from covariance terms and restricted to the unit interval; recalibration uncertainty is added to the design half-width.Two hundred narrative-cluster resamples refit the maps, and the interval bounds combine the design half-width and refit-total range.Source: B Estimator, recalibration, and their… · pp. 23–25
  • Recalibration maps are selected by held-out Brier score over fifty narrative half-splits, subject to a calibration-in-the-large eligibility check.Across factors, recalibrated estimates move by at most 8 percent except vehicle defects, whose isotonic map gives a total 21 percent lower.Source: B Estimator, recalibration, and their… · p. 24
  • The retrospective human-tier comparison evaluates Neyman variance for the mean held-out residual under two equal-size stratifications.The reported ratio compares stratification on reader disagreement with stratification on narrative-reader probability.Source: B Estimator, recalibration, and their… · p. 25

C Sampling designs of the second wave and the second human tier

The second wave samples narratives outside wave 1 using recorded inclusion probabilities, while a separately sampled human tier checks estimates. A simulation reports interval coverage and the effect of adding wave 2.

C Sampling designs of the second wave and the second human tier

  • The second-wave frame contained 4,868,080 narratives outside wave 1; 79,510 draws had inclusion probabilities from 0.0057 to 0.68, and 78,267 were newly read for this paper.The sampling constant was set by bisection to target a weighted total of 60,000.Source: C Sampling designs of the second wave… · p. 25
  • In simulation, 95 percent intervals covered the truth in 0.948 of wave-1 designs, 0.952 of sequential-estimator designs, and 0.972 of multiwave-estimator designs.The second wave narrowed the multiwave interval by 30 percent.Source: C Sampling designs of the second wave… · p. 25
  • The second human tier drew 757 narratives, withheld 112 for identifier screening, and retained 600 in recorded random order for coding.The tier used two of three coders per narrative, with specialist review whenever the third coder read a narrative.Source: C Sampling designs of the second wave… · pp. 25–26
  • Specialists agreed at Cohen’s κ of 0.93 on 148 shared judgments, compared with κ of 0.50–0.57 between the third coder and specialists.The specialist label is the reference for the second tier; phase-2 coders agreed on 99 percent of pairs, with 48 disputed pairs excluded.Source: C Sampling designs of the second wave… · p. 26

D Further conditions of the table-reader evaluation

The evaluation examines table-reader performance across context sizes and model conditions. Context size is associated with larger AUC changes than model size in the reported comparisons, while soft labels and temporal context yield mixed or limited gains.

D Further conditions of the table-reader evaluation

  • Mean calibration errors for Kumo Tabular small, TabPFN 3.5, and CatBoost range from 0.0020 to 0.0023, with mean slopes of 1.01, 1.02, and 1.01.TabICLv2 is overconfident with a slope of 0.90.Source: D Further conditions of the… · p. 26
  • Kumo Tabular small's mean flag AUC rises from 0.792 at the smallest context to 0.865 at 60,000 rows; at 10,000 rows, small, medium, and large models reach 0.849, 0.847, and 0.853.Table D.1 reports AUC for each reader and target; its note identifies paired bootstrap differences from Kumo Tabular small.Source: Table D.1 · p. 26, D Further conditions of the… · p. 26
  • Soft-label training yields mean flag AUC 0.856 versus 0.865 with hard labels, while squared correlation with recalibrated probability is 0.376 versus 0.375.Across temporal contexts, phone-use AUC on 2021 test rows is 0.889 with 2021 context and 0.831 with 2019 context.Source: D Further conditions of the… · p. 26
  • The production comparison uses a per-target gradient-boosted tree tuned by random search on training rows outside each held-out fold.The tuning procedure favors the booster on the other folds.Source: D Further conditions of the… · p. 26

Related papers

Loading 2610.10321v1…