Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening

• LLM summaries of raw employee comments reach time-constrained leaders, while these pipelines enter production faster than they are scrutinized.Source: §1 · p. 1
• Concerns raised in only one or two comments are dropped even when critical; the summary filters by frequency rather than sentiment.Source: §1 · p. 1
• Employees answered the IMPROVE prompt 99.8% of the time versus 80.9% for WENT_WELL; withholding praise was roughly 82 times more common than withholding criticism.Source: §1 · p. 2
• The study asks which voices are retained or dropped across voice type, language, response length, and theme, and whether prompts can reduce under-representation.Source: §1 · p. 2
• The paper contributes a Voice Retention / Representation Ratio, field evidence, prompt-level mitigations, and a disaggregated voice-retention card.Source: §1 · p. 2

2.1. Employee voice, silence, and psychological safety

• Voice research distinguishes promotive suggestions from riskier prohibitive concerns, while treating voice and silence as distinct and silence as potentially self-reinforcing.Source: §2.1 · p. 2

2.2. How voice is transformed as it travels upward

• Prior work examines how voice moves through human intermediaries, but had not examined an LLM compressing comments for leaders.Source: §2.2 · p. 2

2.3. Faithfulness and fairness in LLM summarization

• Summarization research documents omission and distortion as well as unequal representation of social groups and dialects, but not employee voice or its promotive/prohibitive axis.Source: §2.3 · p. 2

2.4. LLM-as-coder and automation bias

• LLM coding is debated, so the study validates automated coding against human adjudication; automation-bias research also warns that leaders may overlook omissions.Source: §2.4 · p. 3

2.5. People analytics and governance

• People-analytics critiques raise transparency, marginalization, and surveillance concerns, while the paper models its voice-retention card on model cards and datasheets.Source: §2.5 · p. 3

2.6. Research gap

• The identified gap combines unstudied algorithmic aggregation of employee voice, absent fairness evidence on its voice types, and a lack of measured, auditable mitigation.Source: §2.6 · p. 3

3. Data and Context

• The corpus contains 2,586 free-text responses from two optional fields in a weekly survey, linked to 538 project codes and 260 distinct teams.Source: §3 · p. 3

3.1. The employee-listening instrument

• WENT_WELL captures appreciation and IMPROVE captures change-oriented voice; only about 16% of participants fill at least one field, and both are answered in one sitting.Source: §3.1 · p. 3

3.2. Ethics, anonymization, and provenance

• The firm authorized use of the anonymized data, which are controlled-access and unreleased as raw text; the company, people, and authors are undisclosed.Source: §3.2 · p. 3

4.1. The construct framework

• Each filled WENT_WELL or IMPROVE field is one analysis unit, coded at its dominant register against a 17-key framework of voice and related constructs.Source: §4.1 · pp. 3–4

4.2. Construct coding pipeline

• An LLM codes the corpus, with a 320-unit gold sample independently coded and adjudicated; core constructs show high agreement, while directness is treated as directional.Source: §4.2 · p. 4

4.3. Summary pipelines

• The same model produces terse, balanced, and mitigation-prompt summaries for 15 teams, yielding 45 summaries from 426 source units.Source: §4.3 · p. 4

4.4. Audit metrics

• Theme retention and Representation Ratio compare summary presence with source presence; cluster-robust models and permutation nulls test survival, with p-values treated as approximate because there are 15 clusters.Source: §4.4 · p. 4

• 493 responses contained criticism without praise, versus 6 with praise without criticism, an asymmetry of roughly 82 to 1.IMPROVE was filled in 99.8% of free-text responses, compared with 80.9% for WENT_WELL.Source: §5 · p. 4
• The 82-fold asymmetry describes how engaged commenters use this channel, not a universal pattern of withholding praise.The authors contrast this with literature focused on silence about problems.Source: §5 · p. 5
• WENT_WELL completion averaged 82% across 169 teams with at least three responses, ranged from 25% to 100%, and was below 60% in 19 teams.An estimated 6–14% of variance sat at the team level (ICC 0.06–0.07; logistic latent-scale 0.14).Source: §5 · p. 5
• Promotive voice outnumbered prohibitive voice 3.23 to 1, while open problem-statements were the rarest substantive register at about 12% of units.Employees hedged problems and suggestions about equally; improvement asks most often targeted processes and systems (38%) and peers (27%), while 8% reached senior managers. German improvement voice was coded as less direct than English, directionally.Source: §5 · p. 5
• Across 966 responses, wave rank showed no measurable trend in appreciation completion (odds ratio 1.05, p = 0.49), futility markers, suggestion rate, or response length (all p > 0.1).The asymmetry was present from the first wave and held throughout.Source: §5 · p. 5
• Workload comments clustered most strongly with work-life predictability (lift 1.61, n = 99, p = 2.6×10−8), and 29% of overload comments also raised predictability.Workload also clustered with wellbeing (lift 1.66, p = 0.004); the expected chain from client pressure through workload to lost work-life was unsupported (mean edge-lift 0.95).Source: §5 · p. 5

6. Results II: Auditing What Survives the Summary (RQ2)

• All three voice types appear in every summary under every configuration, but presence coding cannot detect whether criticism’s intensity or tone was softened.The result establishes that problems survived, not that their full force did.Source: §6 · p. 5
• 61% theme retention for exec_terse versus about 71% for paragraph configurations, with salience-weighted retention exceeding plain retention in every configuration.Balanced is most faithful to the source theme set (Jaccard 0.30), while exec_terse is least faithful (0.40).Source: §6 · pp. 5–6, Table 2 · p. 6
• 0.73 vs 0.49 retention under exec_terse for critical themes versus positive-only themes disappears as a valence effect after controlling for frequency.Source-unit count predicts survival (+1.61, p < 0.001), while the critical indicator is not distinguishable from zero (+0.22, p = 0.55); the permutation test is the primary basis for the null.Source: §6 · p. 6, Table 3 · p. 6
• 0.14 vs 0.74 retention under exec_terse for singleton themes versus themes raised in at least three comments means singletons are dropped about 86% of the time.The intervals do not overlap; the high-frequency workload hub passes through nearly intact (representation ratio 1.07).Source: §6 · pp. 6–7
• 0.27 vs 0.65 retention for themes raised only in short comments versus those with an elaborated comment, while German-only themes retain at 0.20 versus 0.61 overall.The German-only result is directional because all five German-only cells are singletons.Source: §6 · p. 6

7. Results III: Mitigation and Design Principles (RQ2)

• 0.900 protected-theme retention under preserve exceeds 0.725 under exec_terse and 0.850 under balanced, while German-only retention doubles from 0.20 to 0.40.The prompt names workload, work-life, and wellbeing; it does not outperform balanced across all critical themes or repair frequency and length disparities it does not mention.Source: §7 · p. 7
• A concern raised by many is not necessarily more important, so summarizers should protect singleton concerns and up-weight terse voice rather than treating repetition as importance.The authors also recommend explicit parity checks for non-majority languages and a disaggregated voice-retention card.Source: §7 · p. 7
• Frequency, length, and language parity should be audited directly: single-comment concerns are dropped about 86% of the time, and short comments drop about 2.4 times more often.The proposed voice-retention card reports retention by construct, language, and length, and where data permit, by social group and role.Source: §7 · p. 7

8.1. Theoretical implications

• The discussion extends voice/silence theory to algorithmic re-suppression at summarization, where prevalence rather than valence predicts which raised concerns reach decision-makers.This adds a prevalence dimension to literature framed around motive and climate.Source: §8.1 · pp. 7–8, Figure 3 · p. 8

8.2. Responsible-AI implications

• The audit reframes summarization failure as prevalence-based representational harm: criticism largely survives, while German-only voice is attenuated; demographic parity is a possible extension, not tested here.The German-only result is directional, and the corpus lacks employee demographics for a demographic-parity audit.Source: §8.2 · p. 8

8.3. Governance and surveillance

• Disclosing the voice-retention card to employees as well as leaders shifts the pipeline toward mutual accountability by showing employees whether their concern reached the top.The authors connect this transparency to Bernstein’s (2012) transparency paradox: disclosing what the summary hides.Source: §8.3 · p. 8

8.4. Limitations

• Evidence from one firm's self-selected comments is descriptive; without outcomes or demographic data, the study makes no causal claims and cannot audit demographic parity.German-only and length findings are directional; theme presence measures a floor and cannot detect tone-flattening in retained criticism.Source: §8.4 · p. 9

9. Conclusion

• About 82 to 1: employees in this corpus withhold praise more than criticism, while LLM summaries retain criticism but drop concerns raised by only one or two people.Terse and German-only voice are also lost along the popularity gradient, with the German finding directional.Source: §9 · p. 9
• A targeted prompt recovers themes it is told to protect but leaves frequency and length bias in place, making prompt-level fixes partial by construction.Source: §9 · p. 9
• The Voice Retention and Representation Ratio and disaggregated voice-retention card offer a way to measure retention equity and guide listening-system design.The conclusion warns that equating frequency with importance keeps leaders hearing what is already said most often.Source: §9 · p. 9