Source-linked AI summary
Epistemic orientation predicts legislative effectiveness among members of the US Congress
Segun Aroyehun, Stephan Lewandowsky, David Garcia
TL;DR
Prior research documented aggregate changes in congressional evidence-oriented language but left individual differences, cross-platform consistency, and legislative consequences unresolved. Using EMI across congressional speeches and Twitter alongside legislator-level data, the study finds that ideological extremity is associated with lower EMI, EMI is consistent across platforms but lower on Twitter, and higher floor-speech EMI is associated with greater legislative effectiveness. The authors caution that these are associations rather than causal effects.
Problem
Prior work focused mainly on aggregate congressional sessions, leaving individual differences in epistemic orientation, cross-platform generalization, and links to legislative effectiveness unresolved.
Method
The study measures evidence- versus intuition-oriented language with EMI in congressional floor speeches and Twitter posts, linking it to ideology, communication context, and Legislative Effectiveness Scores.
Results
More ideologically extreme members use less evidence-oriented floor language; EMI correlates across Twitter and the floor (r = 0.442), is lower on Twitter, and positively predicts legislative effectiveness after extensive controls.
Takeaways & Limitations
Epistemic orientation is both an individual-level and context-sensitive feature of congressional communication associated with legislative effectiveness.
Takeaways & Limitations
The observed relationships do not permit causal inferences.
Abstract
from arXiv · showhide
Truth and evidence-based communication provide important foundations for democratic governance, accountability, and collective decision-making. Prior work shows that evidence-oriented language in US congressional floor speeches has declined since the mid-1970s, alongside broader changes in legislative productivity and polarization. This study shifts the analysis from congressional sessions to individual members of Congress to examine whether epistemic orientation varies systematically across legislators and whether it relates to political behavior and legislative effectiveness. Using the Evidence-Minus-Intuition (EMI) score, we measure the relative prevalence of evidence-oriented versus intuition-oriented language in congressional floor speeches and Twitter posts. We link these measures to legislator-level data on ideology, institutional position, communication context, and Legislative Effectiveness Score (LES). The results show that more ideologically extreme members use less evidence-oriented language on the congressional floor. EMI also exhibits cross-platform consistency with members who use more evidence-oriented language in floor speeches also being more evidence-oriented on Twitter, although EMI is lower on Twitter overall. Finally, EMI in congressional speeches is positively associated with individual legislative effectiveness, even after accounting for ideology and extensive political, institutional, demographic, topical, and communication volume controls. These findings suggest that evidence-oriented language is not only an aggregate feature of congressional discourse but also a meaningful attribute of individual-level legislative communication and effectiveness.
Main text
This study examines epistemic orientation as an individual-level feature of congressional communication and its relationship to ideology, communication context, and legislative effectiveness. More extreme legislators use less evidence-oriented language, EMI is consistent across platforms but lower on Twitter, and floor-speech EMI is positively associated with legislative effectiveness.
- Relationship between EMI and ideological extremity: More ideologically extreme legislators use less evidence-oriented language in congressional floor speeches.Ideological extremity significantly predicts lower EMI (b = -0.116, 95% CI [-0.133, -0.098], p < 0.05) across alternative specifications.
- Consistency of epistemic orientation across communication platforms: EMI is positively correlated across congressional floor speeches and Twitter, indicating broadly consistent epistemic orientation across communication arenas.The cross-platform correlation is r = 0.442, 95% CI [0.418, 0.466], p < 0.05; Twitter EMI also significantly predicts floor EMI.
- Association between communication channel and EMI: Legislators have lower EMI on Twitter than in congressional floor speeches despite maintaining broadly consistent cross-platform orientations.The communication-channel coefficient for Twitter is b = -0.150, 95% CI [-0.242, -0.057], p < 0.05, robust to party, chamber, and opposition controls.
- EMI is a predictor of legislative effectiveness: Floor-speech EMI is positively associated with legislative effectiveness after controlling for established political, institutional, topical, and communication-volume predictors.EMI remains a significant predictor of LES across two specifications (b = 0.163, 95% CI [0.142, 0.185], p < 0.05).
- Discussion and conclusion: The findings extend aggregate congressional evidence by showing that epistemic orientation varies across individual legislators and relates to legislative effectiveness.The authors interpret evidence-oriented communication as associated with both congressional productivity and individual legislative effectiveness, without claiming causality.
Methods
The study combines legislator-level congressional and Twitter data with a hybrid EMI measure to analyze epistemic orientation across members and communication settings. It validates EMI against human annotations and uses aggregated scores and mixed-effects models for inference.
- Data: The analysis covers congressional speeches from 1873 to 2024 and Twitter posts from members active on both platforms within the same quarter from 2013 to 2022.Speech data are linked to legislators using unique identifiers, while tweets are deduplicated and filtered for insufficient textual content.
- EMI computation: EMI combines ratings from three instruction-tuned language models with embedding-based semantic similarity scores for evidence- and intuition-oriented language.The component scores are standardized and averaged into a final EMI score.
- Validation: 592 human-annotated text segments yield an EMI AUC of 0.825, outperforming word embeddings at AUC = 0.791 and a random baseline at AUC = 0.5.The hybrid measure is evaluated against separate human ratings of evidence-based and intuition-based language.
- Aggregation: Scores are aggregated within speeches and then across legislators by two-year Congresses or calendar quarters, aligning floor speeches and Twitter posts for comparison.Quarterly aggregation accommodates Twitter’s continuous activity and congressional recesses.
- Statistical modeling: Linear mixed-effects models account for repeated observations within legislators, period-specific fixed effects, and clustered standard errors.The supplied model specifications include political, institutional, demographic, topical, leadership, electoral, and other covariates across nested models.
Author Contributions Statement
The authors divide responsibility across conceptualization, data collection, text-analysis development, statistical analysis, drafting, and final editing.
- Author contributions: SA conceptualised the research, collected the data, developed the text-analysis pipeline, performed the statistical analyses, and prepared the initial manuscript draft.All authors contributed to preparing and editing the final version.