PaperlayerView exact revision on arXiv

Source-linked AI summary

When Does "Post-Call" Begin? An Availability Contract for Earnings-Call NLP Benchmarks

Xiaoyu Ma, Kelvin J. L. Koa, Yuncong Liu, Ke-Wei Huang

arXiv:2610.21659v1q-fin.CPcs.CLcs.LG

TL;DR

Financial NLP benchmarks can start return targets before the complete textual or audio input exists, a within-event timing mismatch that chronological splits do not prevent. The paper introduces an Availability Contract and paired clock analyses, finding material changes in labels and losses even when prediction-stream rankings remain unchanged.

  • Problem

    Event-window targets may start before complete text or audio input is available, and chronological train–test splits do not prevent this within-event mismatch.

    Source: §1 · p. 1

  • Method

    The Availability Contract records input boundaries, timing evidence, outcome clocks, and supported claims, and the paper compares common-endpoint returns across three starting-price clocks.

    Source: §1 · pp. 1–2

  • Results

    At the stipulated 15-minute baseline, S→A flips return signs for 9.4% of calls and fixed-A cutpoints change quintiles for 28.3%; shifting before-market DEC targets raises three-day MSE by 18.1–21.4% while rankings remain unchanged.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The contract makes outcome-clock choices explicit and provides a basis for reporting target and loss sensitivity alongside model comparisons.

    Source: §7 · p. 7

    The documentary source timestamp, reconstructed audio-duration proxy, and stipulated allowance do not observe transcript readiness, model readiness, or the other workflow clocks relevant to deployment claims.

    Source: §3.1 · p. 3

Abstract

from arXiv · show

Corporate financial disclosures often unfold over time, with announcements, prepared remarks, and discussion spanning multiple high-frequency return intervals. Financial NLP benchmarks can therefore suffer from a within-event timing mismatch when the prediction target begins before the textual or audio input used by a model could be complete. Chronological train--test splits do not prevent this mismatch, because it occurs inside individual events rather than across samples. We introduce an Availability Contract, which requires a benchmark to state the input boundary, evidence for input availability, the outcome start, and the claims supported by that evidence. We apply this framework to 552 earnings calls from 24 firms. For each call, we hold the return endpoint fixed and compare the previous close, a documentary call-start proxy, and a completion proxy based on reconstructed audio end plus a stipulated processing allowance. At this baseline, moving from the call-start proxy to the completion proxy flips return signs for 9.4% of calls. Holding completion-clock cutpoints fixed, 28.3% change quintile. In an independent release-session analysis of a public earnings-call volatility benchmark, shifting before-market target windows forward by one trading session raises three-day mean squared error by 18.1-21.4% across four frozen prediction streams, while their point rankings remain unchanged. Together, the results show that labels and losses can depend materially on the declared outcome clock even when model rankings appear stable. The Availability Contract makes this assumption explicit and ties post-input claims to the available timing evidence.

1 Introduction

Event-window targets can begin before a call’s complete text or audio input is available, a within-event mismatch that chronological train–test splits do not remove. The paper formalizes benchmark timing assumptions and tests how changing outcome clocks affects labels and predictive losses.

1 Introduction

  • Identical models, data, and splits can yield different labels when disclosure times, input completion, or timestamp-to-price mappings differ.The paper therefore calls for reproducible event timing alongside model code.Source: §1 · p. 1
  • The Availability Contract records the input boundary, completion evidence, processing allowance, price-anchor rule, outcome endpoint, and claims supported by timing evidence.It does not prescribe one universally correct event window.Source: §1 · p. 1
  • A pre-completion return may measure total market response, whereas a post-completion return better matches prediction from the complete representation.The paper distinguishes these as different estimands rather than calling both post-input prediction.Source: §1 · p. 2
  • At the stipulated 15-minute baseline, S→A flips return signs for 9.4% of calls; fixed-A cutpoints change quintile assignments for 28.3%.The paired audit covers 552 calls from 24 firms while holding the return endpoint fixed.Source: §1 · p. 2
  • Three-day MSE rises 18.1–21.4% when before-market DEC targets shift forward one session, while four frozen prediction streams retain their rankings.The analysis uses the public Dense Earnings Call benchmark.Source: §1 · p. 2

2 Related Work and Evaluation Context

Prior work establishes the informational relevance of earnings-call language and audio and develops a wide range of financial-NLP representations and benchmarks. This section distinguishes that relevance from temporal validity and situates the paper’s representation-completion timing question alongside event studies, release-clock work, and multiplicity concerns.

2.1 Earnings calls as information events

  • Financial language contains information beyond accounting numbers; earnings-call Q&A adds to presentations, non-answers and linguistic patterns reveal managerial behavior, and question–answer similarity has been related to returns.Source: §2.1 · p. 2

2.2 Financial-NLP representations and benchmarks

  • Financial-NLP work spans finance dictionaries and FinBERT [2], retrieval, symbolic discovery [18], and earnings-call long-document, multimodal, fusion, and Q&A-quality benchmarks.Controlled synthetic benchmarks also diagnose forecasting under explicit financial mechanisms.Source: §2.2 · p. 2
  • Release-timing work separates announcement-day explanatory power from post-open trading, while the Availability Contract extends that principle to representation-specific completion.Related work also notes that day indexing can select the wrong first post-earnings day when release timing is omitted.Source: §2.2 · p. 2

2.3 Event windows, temporal validity, and multiplicity

  • Event studies and announcement-clock research frame event windows, while this paper asks what evidence permits a label to begin after its representation is available.Figure 1 maps source and completion proxies, common-endpoint returns, representation–target fits, and adjacent-clock coefficient contrasts; repeated predictor search raises a separate multiplicity issue.Source: §2.3 · pp. 2–3, Figure 1 · p. 3

3 Availability-Aware Paired Audit

The audit ties benchmark claims to an explicit input boundary, timing evidence, and outcome clock, using documentary and reconstructed completion proxies. It then compares return objects under paired designs to isolate differences associated with removed return intervals.

3.1 Availability Contract and instantiated proxy clocks

  • The Availability Contract records the input boundary, timing evidence, price-anchor rule, endpoint, sample rule, and the claim those choices support.Its label start applies the anchor rule after the source-to-completion-proxy rule and allowance, with interpretation tied to the recorded evidence.Source: §3.1 · p. 3
  • The audit’s clocks are retrospective proxies; complete-input, checkpoint-available, model-ready, order, and execution times are unobserved.A deployment claim requires the label start to follow every workflow-relevant clock and a checkpoint whose knowledge boundary precedes the forecast.Source: §3.1 · p. 3
  • The completion proxy reconstructs audio duration from archived records and adds a stipulated allowance to define the study’s ready time.The audio-duration proxy takes the maximum of two elapsed-time estimates measured relative to the documentary source timestamp.Source: §3.1 · p. 3
  • The source-start and completion-proxy anchors use the first regular-session five-minute bar starting strictly after their respective proxies, with the selected bar closing five trading minutes later.The completion anchor incorporates reconstructed duration and the stipulated allowance; both anchors are bar-close prices used to construct return labels.Source: §3.1 · p. 3

3.2 Return objects and exact coefficient identity

  • Log returns make adjacent intervals additive, enabling paired OLS contrasts to attribute outcome differences to removed intervals while holding the design and other analysis choices fixed.A separate return ending after exactly 390 XNYS trading minutes tests duration and is not part of the coefficient identity.Source: §3.2 · p. 4

4 Earnings-Call Case Study

The case study uses a selective 552-call, 24-firm panel, applies four retrospective Q&A representations, and compares benchmark sensitivity under common regression and resampling designs.

4.1 Sample and clock construction

  • The common complete-case panel contains 552 calls from 24 firms over 2018–2024, retaining 26.4% of the 2,094-call parent panel.Attrition is selective: the source-clock join shifts the panel toward later years and larger firms, and the price join removes 117 calls from five firms.Source: §4.1 · p. 4
  • The audit parses 842 archived source pages and constructs audio-duration records, while most call-start times remain documentary headers rather than verified starts.In a worked after-close MSFT example, both proxy clocks map to the next-session close.Source: §4.1 · p. 4

4.2 Four retrospective representations

  • Four call-level Q&A representations cover lexical tone, finance sentiment, answer valence, and within-firm narrative change.Qwen scores are retrospective because the 2025-vintage checkpoint had no established historical knowledge cutoff.Source: §4.2 · p. 4

4.3 Regression specification and inference

  • Each regression uses the same 552 rows and sparse controls; coefficients measure percentage points per signal standard deviation with firm, quarter, and timing-class fixed effects.Target metrics use separate firm-pair and calendar-week-pair resampling intervals; exploratory coefficient inference uses two one-way resampling schemes and a Holm adjustment across four max-p values.Source: §4.3 · pp. 4–5
  • 9.4% of raw-return signs, 28.3% of fixed-A quintiles, and 30.1% of re-estimated relative quintiles change under S→A on 552 paired calls.These are the Figure 2 target-clock sensitivity measures.Source: Figure 2 · p. 5

5 Results

Across the three-clock audit and the DEC rescoring, changing the outcome clock alters return targets and reported losses, even when point-model rankings remain unchanged. The results distinguish completion-proxy effects from earlier event-window realignment and from fixed-horizon sensitivity.

5.1 Completion Alignment Reassigns Targets

  • The L→S realignment has Spearman correlation 0.506, mean absolute return change 3.427 pp, 32.1% sign changes, and 64.3% relative-quintile reassignment.These quantities describe conventional event-window realignment.Source: §5.1 · p. 5
  • At the 15-minute baseline, S→A changes raw-return signs for 9.4% of calls; 28.3% change quintile under common A-clock cutpoints, versus 30.1% with clock-specific reference distributions.Fixed-cutpoint reassignment measures crossing the same thresholds; relative reassignment measures movement within each clock’s distribution.Source: §5.1 · p. 5
  • For 311/552 calls, S and A map to different price anchors; among the 241 equal-anchor calls, returns and fixed-A bins never change, though 6.6% change relative quintile.All 206 after-close calls share the same next-session bar under both proxies, so their returns coincide.Source: §5.1 · p. 5

5.2 Frozen Rescoring Changes Losses, Not Rankings

  • At three days, shifting DEC before-market targets raises MSE by 18.1–21.4% across four frozen prediction streams, while point rankings remain unchanged across metrics and horizons.By 15 days, MSE is 3.8–6.2% higher; target sensitivity attenuates with horizon.Source: Table 3 · p. 6, §5.2 · p. 6
  • At three days, at least one model pair reverses in 43.1% of issuer resamples and 45.0% of calendar-quarter resamples, despite unchanged point rankings.The same-best-MSE model is retained in 67.1% of issuer resamples and 72.1% of calendar-quarter resamples at three days.Source: §5.2 · p. 6

5.3 Associations Localize before Source Start

  • The exploratory Qwen association shift from L to A is +1.355 pp per signal SD, with +1.256 over L→S and +0.098 over S→A.The reported estimates are retrospective point estimates.Source: §5.3 · p. 6

5.4 Holding-Period and Design Sensitivities

  • Across the 0–120-minute allowance grid, sign, relative, and fixed-A15 move rates span 6.5%–15.0%, 23.9%–39.9%, and 21.0%–35.9%, respectively.The 15-minute baseline sign-flip rate of 9.4% lies within the allowance-sensitivity envelope.Source: §5.4 · p. 6, Table 4 · p. 6
  • Conditional on the same A start, following-close and exact-390-minute Qwen point estimates are −0.233 and −0.226, while exact-horizon coefficients remain nonsignificant after multiplicity correction.The fixed-horizon check retains the A start and fixes elapsed trading time.Source: §5.4 · p. 6

6 Discussion and Availability Card

The Availability Card records representation-specific completion evidence and the clock choices that define benchmark outcomes. The discussion distinguishes event-response windows, completed-input prediction, and comparisons that hold either forecast origin or elapsed horizon fixed.

6.1 A practical benchmark specification

  • The Availability Card specializes BenchmarkCards to representation completion, clock evidence, price anchors, endpoints, and permitted claims.Its fields link the input boundary and completion rule to the model checkpoint, return definition, sample, and inference records.Source: §6.1 · p. 6

6.2 Interpreting the comparison

  • The S→A comparison isolates completion-proxy alignment, whereas most of the exploratory Qwen association shift occurs over L→S.DEC separately shows that stable rankings can coexist with changed losses under release-session realignment.Source: §6.2 · p. 7
  • An event-response study can use L, while evaluation using a completed representation specifies a later input boundary; delaying label start also shortens the interval to a common endpoint.A separate 390-trading-minute check retains the A start and fixes elapsed trading time.Source: §6.2 · p. 7
  • Paired rescoring with predictions fixed isolates target-definition effects; retraining under each clock additionally measures model adaptation.A common-anchor comparison holds forecast origin fixed, while separate clocks preserve each representation’s declared availability boundary.Source: §6.2 · p. 7
  • Reporting should predeclare the contract, freeze a joined sample, report attrition, and record input boundary, timestamp provenance, market calendar, checkpoint, allowance, and endpoint.Common-endpoint and fixed-duration tests distinguish label-start from holding-period sensitivity.Source: §6.2 · p. 7
  • Representations with different readiness times require a shared supported anchor or separately declared tasks.Each representation’s boundary defines a different information set and return interval when evaluated from its own availability boundary.Source: §6.2 · p. 7

7 Conclusion

The paired three-clock audit and independent DEC release-session analysis show that outcome-clock choices can materially alter labels and losses while model point rankings remain unchanged.

7 Conclusion

  • At the stipulated 15-minute baseline, S→A changes raw-return signs for 9.4% of calls and fixed-A/relative assignments for 28.3%/30.1%.Source: §7 · p. 7
  • In the DEC release-session analysis, three-day MSE rises by 18.1–21.4% under frozen predictions while point rankings remain unchanged.Source: §7 · p. 7
  • The Availability Contract and Availability Card make outcome-clock choices explicit and support reporting target and loss sensitivity alongside model comparisons.Source: §7 · p. 7

Related papers

Loading 2610.21659v1…