PaperlayerView exact revision on arXiv

Source-linked AI summary

Auditing Contextual Bias in Human Ball-Strike Calls Using KBO's Automated Umpiring Transition

Kichang Lee, JeongGil Ko

arXiv:2609.03786v1cs.HC

TL;DR

The paper audits whether human ball-strike calls vary with familiar game and player contexts, using the KBO’s ABS transition as a diagnostic benchmark. Its strongest evidence is count pressure: human calls vary with count near the boundary, while the same pattern is nearly absent under ABS.

  • Problem

    The paper revisits familiar but unverified suspicions that umpires call the same borderline pitch differently depending on game context.

    Source: Introduction · p. 1

  • Method

    The study models called-strike probability for borderline pitches and compares human-era context patterns with ABS as a benchmark.

    Source: Modeling strategy · p. 3, Conclusion · p. 11

  • Results

    Count pressure is the clearest signal: human calls were more strike-prone in three-ball counts and less strike-prone in two-strike counts, while the pattern was nearly absent under ABS.

    Source: Introduction · p. 2, Conclusion · p. 11, Discussion · p. 11

  • Takeaways & Limitations

    ABS can serve as audit infrastructure for identifying where human rule enforcement was most context-sensitive and where common suspicions receive little support.

    Source: Discussion · p. 11

    The study is observational, and ABS is a diagnostic benchmark rather than a randomized counterfactual; the analysis is also conditioned on taken pitches.

    Source: Limitations · p. 11

Abstract

from arXiv · show

This paper uses the Korean Baseball Organization's adoption of the Automated Ball-Strike (ABS) system to audit long-standing claims about contextual bias in human ball-strike calls. Using pitch-level KBO data from 2021 through the available portion of the 2026 season, we model called-strike probability for taken pitches near the strike-zone boundary, with 2022-2023 as the primary human-umpire baseline and ABS seasons (2024 and onward) as a diagnostic benchmark. The strongest evidence concerns count pressure. Relative to 0--0 counts, human umpires called substantially fewer strikes in two-strike counts and more strikes in hitter-ahead three-ball counts. Specifically, in the main 0.25-ft boundary band, 0--2 was associated with a -17.17 percentage-point effect and 3--0 with a +6.61 percentage-point effect. Under ABS, the corresponding effects were close to zero and did not survive false-discovery-rate correction. Game progression shows a smaller but coherent pattern as human calls were less strike-prone in early innings and more strike-prone in innings 7--9+, especially in late-close situations, while complete ABS seasons were essentially flat. Other suspected biases are weaker or more localized. Salary-based reputation proxies provide suggestive but proxy-sensitive evidence, and catcher identity shows human-period residual heterogeneity that disappears under ABS. Home-context evidence is mostly null at the umpire level, with one FDR-significant human-period exception and an exploratory umpire-team gap best treated as an audit lead. Overall, the results do not show that human umpires were biased everywhere. Instead, they map where the human strike zone was most context-sensitive, where evidence was weaker, and where common suspicions received little support.

Introduction

The paper audits familiar claims that human umpires change their ball-strike calls with game context, a question difficult to test because contexts and pitch choices are not randomly assigned. It uses the KBO’s ABS transition as a diagnostic benchmark to map where contextual patterns are strongest and where they receive little support.

Introduction

  • The audit examines count pressure, salary-based player-status proxies, game progression, catcher- and pitcher-associated residual variation, and home context.The salary measure is an observable but imperfect proxy for reputation.Source: Introduction · pp. 1–2
  • The study compares called-strike probabilities for similar taken pitches near the rule-zone edge, controlling for pitch location, zone geometry, pitch characteristics, season, and relevant player or game context.It uses 2022–2023 as the human baseline and 2024–2026 as the automated benchmark.Source: Introduction · p. 2
  • Count pressure is the clearest signal: borderline pitches were more strike-prone in 3–0 counts and less strike-prone in two-strike counts under human umpires, while those patterns were near zero under ABS.The comparison is diagnostic rather than causal because the ABS transition was not randomized.Source: Introduction · pp. 1–2
  • The paper presents ABS as audit infrastructure for mapping where human rule enforcement was context-sensitive, not as proof that automation simply replaced an inaccurate system.Its results show a hierarchy of evidence rather than universal bias; home-context evidence is mostly null.Source: Introduction · p. 2

Methodology

The analysis uses KBO pitch-level records to model called-strike probability for taken pitches near the rule-zone boundary. It estimates a local strike-zone surface and tests whether context shifts calls among comparable pitches.

Data Collection

  • The dataset contains 1,216,246 KBO pitch rows from 2021 through the available portion of 2026, with 2022–2023 as the primary human-umpire baseline.The 2021 season is retained for historical sensitivity, and ABS began in 2024.Source: Data Collection · p. 2

Called pitches and strike-zone geometry

  • The analysis excludes swings and other pitches without a taken-pitch ball-strike adjudication, then uses validated batter-specific zone fields and controls for zone-bottom and zone-height differences.Its main boundary-band sample includes pitches within 0.25 ft of the nearest rule-zone boundary.Source: Called pitches and strike-zone geometry · pp. 2–3, Table 1 · p. 2

Modeling strategy

  • The model estimates a local called-strike surface before testing whether context shifts calls among comparable pitches, using boundary-aligned geometry and pitch, matchup, and season controls.Results are average marginal effects in percentage points, with confidence intervals, sample sizes, and false-discovery-rate-adjusted q-values.Source: Modeling strategy · p. 3

Results

The audits find the clearest contextual structure in count pressure, with smaller or more localized evidence for game progression, player status, catcher identity, and home context. Several human-period patterns are weak or absent under ABS, while salary and localized home-team findings require caution.

Audit 1: Do umpires balance the count?

  • Relative to 0–0, human-period 0–2 calls were 17.17 pp less likely to be strikes and 3–0 calls 6.61 pp more likely; ABS effects were near zero.The ABS estimates were +0.33 pp for 0–2 and +0.37 pp for 3–0, and neither survived FDR correction. Other two-strike contrasts were also negative, while 3–1 was close to zero and not significant.Source: Audit 1: Do umpires balance the count? · pp. 3–4, Figure 1 · p. 4

Audit 2: Do high-status players get a different zone?

  • Salary evidence was proxy-sensitive: excluding Choo shifted the human batter slope from -1.50 pp (unsupported) to -3.82 pp (supported), while pitcher contrasts favored higher-salary players.Human-period pitcher contrasts were FDR-supported at four cutoffs, with effects between +1.62 and +2.30 pp; corresponding ABS contrasts were close to zero. The batter result depends on treating salary as a noisy reputation proxy.Source: Figure 4 · p. 5, Figure 3 · p. 5, Audit 2: Do high-status players get a… · p. 5

Audit 3: Does the zone change late in games?

  • Human-period early innings were 0.72 pp less strike-prone and late innings 0.69 pp more strike-prone; late-close calls rose 1.38 pp, unlike late blowouts.Complete ABS seasons showed no FDR-supported phase effect; score-state diagnostics caution against treating individual cells as pure umpire preferences.Source: Figure 5 · p. 6, Audit 3: Does the zone change late in… · pp. 6–7

Audit 4: Do catchers and pitchers leave residual strike-zone signatures?

  • Human-period catcher residual dispersion was 1.13 pp with strong heterogeneity, versus effectively zero under ABS; pitcher heterogeneity was detectable but individual estimates were less stable.Several catcher residuals survived FDR correction and spanned roughly five percentage points, while individual pitcher estimates are exploratory given sparser, more fragmented cells.Source: Table 2 · p. 7, Audit 4: Do catchers and pitchers… · pp. 7–8, Figure 7 · p. 8

Audit 5: Do home teams get calls?

  • Among 32 eligible umpires observed in both periods, only U1 had a human-period home-favoring contrast surviving FDR correction: +4.80 pp (q = 0.014; 2,890 called pitches across 46 games); none was significant under ABS.The targeted U4 follow-up found an LT-over-HT gap of 23.13 pp (game-cluster bootstrap interval 7.61 to 37.57 pp; p = 0.002) across 591 pitches in nine games; it is an audit lead, not a confirmed league-wide team-bias claim.Source: Audit 5: Do home teams get calls? · pp. 8–9

Related work

Pitch-tracking research enabled pitch-level strike-zone analysis [9, 10, 11], while contextual-bias work asks whether non-location factors affect calls after pitch geometry is considered [16, 17, 18]. This paper uses ABS as a diagnostic benchmark for mapping contextual effects in human calls [22, 23, 24, 2].

Pitch tracking and strike-zone modeling

  • Strike-zone research progressed from simple rectangular comparisons to probabilistic, geometric, and hierarchical models, supporting location-controlled analysis of boundary pitches [5, 7, 6, 8].Pitch-tracking systems made location, movement, release, pitch quality, and zone geometry measurable at the pitch level [9, 10, 11, 12, 13, 14, 15].Source: Pitch tracking and strike-zone modeling · p. 10

Umpire accuracy and contextual bias

  • Contextual-bias studies test whether non-location features affect calls after pitch geometry is considered, without treating every deviation from a modeled zone as an error [16, 17, 18].The paper identifies count pressure as a clear case: marginal calls in 3–0 and 0–2 counts have different plate-appearance consequences.Source: Umpire accuracy and contextual bias · p. 10

Catcher framing, status, and home context

  • Catcher framing may shift calls on similar pitches [6, 19, 20]; salary is an imperfect status proxy, and home effects may be weaker for closely monitored ball-strike decisions [21, 18, 17].The study estimates catcher-associated residual effects while avoiding claims about glove mechanics or intent.Source: Catcher framing, status, and home… · p. 10

Automated officiating as audit infrastructure

  • Rather than treating ABS only as replacement or accuracy infrastructure [22, 23, 24, 2], the paper uses it as a diagnostic benchmark to map suspected contextual effects as strong, weak, or unsupported.Human-period patterns that attenuate under ABS are more consistent with context-sensitive human adjudication than pitch distribution alone, though ABS is not a randomized counterfactual.Source: Automated officiating as audit… · pp. 10–11

Discussion

Count pressure is the clearest contextual pattern, while game-phase and catcher-associated effects are smaller or more qualified. The broader evidence supports using automated officiating as audit infrastructure, including in hybrid systems.

Discussion

  • Human calls were more strike-prone in 3–0 counts and less strike-prone in two-strike counts, while these count contrasts were nearly absent under ABS.Source: Discussion · p. 11
  • Game-phase effects were modest: human calls became less strike-prone early and more strike-prone late, especially in late-close situations, without supporting a simple “go-home” rule.Salary-based status effects were weaker and proxy-sensitive because salary only imperfectly measures reputation.Source: Discussion · p. 11
  • Catcher-associated residual variation appeared under human umpires and collapsed under ABS, while home-context evidence was mostly null with localized human-period exceptions.Source: Discussion · p. 11
  • The evidence hierarchy suggests count pressure is the most actionable target for umpire training, feedback, or post-game review, while catcher residuals can be monitored without reducing the analysis to individual blame.Source: Discussion · p. 11
  • In hybrid ABS challenge systems, contextual bias may shape which pitches players challenge, when they hesitate, and how umpires behave when calls can be reviewed.The paper calls for future study of contextual bias as part of a human–automation interaction loop.Source: Discussion · p. 11
  • ABS serves not only as replacement infrastructure but also as a benchmark for locating context-sensitive human rule enforcement and identifying suspicions with little support.Source: Discussion · p. 11

Limitations

The study is observational, and its ABS comparison is diagnostic rather than a randomized counterfactual. Selection, measurement, and setting differences constrain interpretation and motivate replication.

Limitations

  • Because contextual factors are not randomly assigned, controls cannot eliminate all confounding from pitch selection, batter approach, pitcher command, or team strategy.Source: Limitations · p. 11
  • ABS is a diagnostic benchmark rather than a randomized counterfactual, and the human and automated periods differ in rosters, strategies, league conditions, and player adaptation.The analysis also conditions on taken pitches, whose selection may vary across counts, pitchers, and game states.Source: Limitations · p. 11
  • Several measures are imperfect, including salary as a reputation proxy and catcher identity reconstructed from lineup data rather than pitch-by-pitch defensive tracking.The paper also notes that batter-specific zone fields are not direct observations of perceived umpire zones or exact operational ABS boundaries; multiple tests motivate replication in other seasons or leagues.Source: Limitations · p. 11

Conclusion

The KBO’s ABS transition provides a benchmark for auditing contextual human umpiring. The findings identify structured count-pressure effects alongside smaller or weaker patterns, supporting automated officiating as audit infrastructure as well as replacement technology.

Conclusion

  • Count pressure was the strongest evidence: borderline pitches were more strike-prone in three-ball counts and much less strike-prone in two-strike counts before ABS, with the pattern nearly absent under ABS.Source: Conclusion · p. 11
  • Smaller human-period patterns appeared in late-game contexts and catcher-associated residual variation, while salary/status evidence was proxy-sensitive and average home-team advantage mostly unsupported.Source: Conclusion · p. 11
  • Human ball-strike calls were context-sensitive in structured, baseball-interpretable ways, but not everywhere.Source: Conclusion · p. 11
  • ABS can serve as audit infrastructure by benchmarking where human rule enforcement was most vulnerable to contextual drift and where common suspicions receive little support.Source: Conclusion · p. 11

Data availability

The study uses publicly accessible Naver Sports game records and plans to release derived materials subject to source terms.

Data availability

  • Pitch-level records came from publicly accessible Naver Sports game records, and cleaned derived tables and replication scripts will be released subject to data-source terms of use.Source: Data availability · p. 12
Loading 2609.03786v1…