PaperlayerView exact revision on arXiv

Source-linked AI summary

Understanding Everyday Communication Patterns, Stress, and Productivity in Hybrid Work through Egocentric Wearable Audio Sensing

Tiantian Feng, Aditya Kommineni, Emily Zhou, Burcin Becerik-Gerber, Gale Lucas, Shawn C. Roll, Shrikanth Narayanan

arXiv:2610.37270v1cs.SD

TL;DR

Home and office work may shape everyday communication differently, but their relationships with productivity and well-being require empirical measurement. The paper combines wearable acoustic sensing, offline speaker diarization, and knowledge distillation in a four-month field deployment. It reports location-dependent communication patterns and associations between communication measures and self-reported stress and productivity.

Abstract

from arXiv · show

The hybrid work trend in the post-COVID period has transformed how people communicate while at work. However, there is a lack of quantitative understanding of how home and office environments shape communicative behavior and affective well-being. To study this, as a part of a larger study of the role of work environment on stress, we designed an egocentric wearable audio sensing system that captured interpersonal communication across both working locations. The system extracts real-time acoustic features on a wearable audio sensor and performs offline foreground speaker diarization to derive speaking load and activeness features. To improve diarization performance under low-dimensional feature constraints, we apply a teacher-student knowledge distillation framework that transfers knowledge from large-scale speech foundation models into a compact ResNet-based model, achieving consistent improvements in diarization performance over the baseline across three public egocentric speech datasets. In a four-month longitudinal deployment with 16 participants, our system revealed location-dependent differences in work communication patterns. Moreover, the mixed linear model analyses showed that office-based speaking load was significantly associated with higher self-reported stress and lower self-reported productivity, while speaking activeness during home-based work amplified both productivity and affective load in this cohort.

I. INTRODUCTION

This study examines how everyday communication differs between home and office work and relates to stress and productivity. It combines wearable sensing and foreground-speaker detection with a four-month deployment to measure those patterns.

I. INTRODUCTION

  • Hybrid work has shifted work away from traditional offices, with everyday behaviors—including communication—varying across locations.The introduction also describes home work as involving prolonged sedentary periods, household interruptions, and blurred work-personal boundaries.Source: §I · p. 1
  • The study compares communication across home and office settings, where work commonly involves different forms of social interaction and engagement.The introduction contrasts spontaneous or scheduled in-person exchanges at the office with more scheduled virtual meetings and less informal exchange at home.Source: §I · p. 1
  • The authors motivate continuous in-the-wild sensing by noting that work-location differences in communication may relate to productivity, stress, and well-being.They describe frequency and duration as fundamental features of social communication and cite prior work linking frequent communication with cohesion, coordination, and trust.Source: §I · p. 1
  • A compact speaker-detection model uses a limited MFCC set and knowledge distilled from speech foundation models to identify the wearer’s speech while reducing privacy risks.The system processes acoustic features on-device rather than retaining raw audio, and training data are augmented with indoor environmental noise.Source: §I · p. 2
  • A four-month deployment with 16 participants found location-dependent communication patterns, with mixed linear models showing significant associations between office-based communication and stress and productivity.The introduction reports associations but does not specify their direction.Source: §I · p. 2

II. STUDY METHODS

The study recruited full-time workers with dedicated workspaces and collected wearable-sensor and survey data during a four-month study.

II. STUDY METHODS

  • Participants were recruited from a higher education institution and had to work at least 30 hours weekly at a computer workstation with a dedicated office workspace.The study procedures received institutional review board approval, and participants provided written informed consent.Source: A. Overview · p. 2
  • Table I summarizes participant demographics and work-schedule characteristics, including self-reported office time as a percentage of total working time.Source: Table I · p. 2
  • Participants were instructed to complete four surveys per workday and wear the audio sensor throughout working hours.Data collection lasted four months for each participant.Source: A. Overview · p. 2

B. Study Participants

The analysis included 16 sensor-wearing participants from a higher education institution and used repeated workday surveys to capture location, productivity, and stress.

B. Study Participants

  • Of 19 recruits, 16 wore the sensor for more than 10 workdays and entered the analysis; the cohort included 10 women and 6 men, averaging 43.5 years old.Participants spent an average of 70.0% of working time in the office and worked an average of 8.6 hours.Source: B. Study Participants · p. 2

C. Ecological Momentary Assessment

  • Four EMA surveys were sent by text message each workday; the first and last aligned with reported work start and end times, and midday prompts arrived at random times.Participants reported location and rated perceived productivity and stress on 0–100 scales.Source: C. Ecological Momentary Assessment · p. 2
  • The protocol paired the surveys with wearable audio sensing during working hours, followed by offline foreground-speaker extraction and communication-feature computation.Source: Figure 1 · p. 3

III. WEARABLE AUDIO SENSING AND COMPUTING

The system combines real-time wearable acoustic-feature extraction with offline speaker detection to derive speaking-load and speaking-activeness measures.

III. WEARABLE AUDIO SENSING AND COMPUTING

  • The system has two components: a wearable sensor for real-time acoustic-feature extraction and an offline speaker-detection module that identifies the speaker of interest.Their outputs are used to compute speaking load and speaking activeness.Source: §III · p. 3

A. Wearable Audio Sensor

  • A customized Android app on the Atom smartphone uses openSMILE to extract F0, loudness, and MFCC 1–12 from audio on-device.It processes a 20-second window at one-minute intervals; the per-unit system cost is no more than $200 USD.Source: A. Wearable Audio Sensor · p. 3
  • The Atom device is described as a lightweight, cost-effective Android smartphone used for the wearable audio sensor.Source: A. Wearable Audio Sensor · p. 3

B. Foreground Speaker Diarization

The diarization model distinguishes the device wearer’s foreground speech from other speakers and silence or environmental noise. Because privacy constraints limit wearable inputs to a small set of acoustic features, the paper distills speech-foundation-model knowledge into a compact ResNet-based model.

B. Foreground Speaker Diarization

  • The model labels each frame as foreground speech from the device wearer, background speech from other speakers, or silence/environmental noise.This targets features originating primarily from the wearer.Source: B. Foreground Speaker Diarization · p. 3
  • Privacy constraints make collecting annotated raw audio infeasible, so the ResNet-based diarization model uses MFCC 1–12 and public egocentric speech corpora.Training audio is augmented with indoor noise samples from AudioSet.Source: B. Foreground Speaker Diarization · p. 3
  • The teacher–student framework transfers knowledge from speech-foundation-model teachers to improve FG/BG predictions from low-dimensional acoustic features.The teacher is selected empirically, frozen during distillation, and its outputs guide continued student fine-tuning.Source: B. Foreground Speaker Diarization · pp. 3–4, Figure 2 · p. 4

C. Communication Behavior Extraction

Communication behavior is represented by speaking load and speaking activeness, using measures designed to capture hourly speech and peak activation during the workday.

B. Communication Behavior Extraction

  • Speaking load is speaking time per hour, excluding total speaking time to avoid confounding communication behavior with differences in work hours.A minute counts as valid speech activity when at least 250 FG speech frames occur in its 20-second feature recording window.Source: C. Communication Behavior Extraction · p. 4
  • Speaking activeness uses log-pitch and intensity scores relative to each participant’s personalized empirical distributions, then combines them using Spearman-correlation weights.The daily activeness measure is the 90th percentile of scores, capturing peak communicative activation.Source: C. Communication Behavior Extraction · p. 4

IV. FG/BG DIARIZATION EXPERIMENTS

The experiments evaluate FG/BG diarization on three egocentric audio datasets, with student-model performance compared against teacher models and robustness tested under environmental noise.

A. Datasets

  • The evaluation combined MMCSG, EasyCom, and ICSI, with manual speaker-boundary annotations used to derive foreground, background, and silence ground truth.MMCSG contains 530 three-minute dyadic recordings from 138 participants; EasyCom has 12 sessions totaling about five hours, and ICSI provides close-talk and far-field recordings.Source: A. Datasets · p. 4

B. Experimental Details

  • Models were trained on the combined datasets for 10 epochs, using 60/20/20 session splits for datasets other than MMCSG and noise augmentation from 0–30 dB SNR.The MMCSG default split was retained; inference generated frame-wise labels every 10 seconds, and evaluation reported DER, speaker confusion, and VAD error rates.Source: B. Experimental Details · p. 4

C. FG/BG Diarization Results

  • Distilled-ResNet consistently outperformed baseline ResNet across all metrics and datasets, with a reported 2.8-point DER reduction on EasyCom; it also achieved performance comparable to Whisper-Tiny.Indoor-noise augmentation reduced DER by 5.8% and 2.7% at SNR 5 and 15, respectively, while the models performed comparably at SNR 25.Source: C. FG/BG Diarization Results · pp. 4–5

V. COMMUNICATION PATTERNS

The communication-pattern analysis compared office and home work using foreground speech inferred by the FG/BG diarization model. Participants wore the sensor for an average of 39.6±16.1 workdays.

V. COMMUNICATION PATTERNS

  • The office-versus-home communication analysis used foreground speech regions inferred by the Distilled-ResNet FG/BG diarization model.Frames with fundamental frequency greater than zero were retained to exclude likely unvoiced segments, addressing larger VAD errors.Source: §V · p. 5
  • Participants wore the sensor for 39.6±16.1 days at work, with individual durations from 13 to 74 days.Most participants wore it for 30–50 days; Figure 4 summarizes wearing statistics.Source: §V · p. 5, Figure 4 · p. 5

A. Communication Patterns between Home and Office

Office work was associated with greater speaking load and activeness than home work. Speaking time was higher in the office across all three workday periods.

A. Communication Patterns between Home and Office

B. Communication Patterns and Self-report EMAs

Office speaking load was associated with higher stress and lower productivity, while home-based speaking activeness was associated with increases in both outcomes.

B. Communication Patterns and Self-report EMAs

  • Office speaking load was associated with higher self-reported stress (β = 7.49, p=0.004) and lower self-reported productivity (β = -5.87, p = 0.016).Office location itself was a positive predictor of productivity (β = 18.24, p<0.001).Source: B. Communication Patterns and… · p. 6, Table IV · p. 6
  • Home-specific speaking activeness was positively associated with both stress (β = 9.23, p=0.030) and productivity (β = 14.07, p<0.001).The office-specific activeness slope and home-specific speaking-load slope were not statistically significant for either outcome.Source: B. Communication Patterns and… · p. 6

VI. CONCLUSION

The paper presents a wearable audio sensing system that derives speaking load and activeness, and reports location-dependent communication patterns and associations with self-reported stress and productivity. Future work will expand the cohort and integrate multimodal signals.

VI. CONCLUSION

  • The system combines on-device acoustic feature extraction with offline foreground/background speaker diarization to derive speaking load and activeness.Knowledge distillation transfers speech foundation model knowledge to a compact ResNet, improving over baseline ResNet and reaching performance comparable to Whisper-Tiny.Source: §VI · p. 6
  • A four-month deployment among office workers found systematic communication-pattern differences between home and office, with office speaking load associated with self-reported stress and productivity.The abstract reports 16 participants; it specifies that office speaking load was associated with higher stress and lower productivity.Source: §VI · p. 6
  • The authors identify cohort expansion and multimodal signals as future directions for characterizing hybrid-work experiences.Source: §VI · p. 6

AI-GENERATED CONTENT DISCLOSURE

Loading 2610.37270v1…