Source-linked AI summary
How Work From Home Affects Collaboration: A Large-Scale Study of Information Workers in a Natural Experiment During COVID-19
Longqi Yang, Sonia Jaffe, David Holtz, Siddharth Suri, Shilpi Sinha, Jeffrey Weston, Connor Joyce, Neha Shah, Kevin Sherman, CJ Lee, Brent Hecht, Jaime Teevan
TL;DR
The paper asks whether WFH changes information workers’ collaboration independently of other pandemic-related factors. Using a difference-in-differences analysis of Microsoft software-usage logs, it finds that WFH decreased collaboration and increased focus time, while shifting activity from meetings toward messaging.
Problem
The study asks whether WFH’s effects on information workers’ collaboration can be isolated from other pandemic and seasonal factors, a question relevant to future remote work.
Method
The authors use difference-in-differences on large-scale Microsoft software-usage logs, comparing changes for workers previously WFH with those previously working in offices.
Results
WFH decreased collaboration hours by 1.95%, increased focus hours by 3.57%, decreased meeting hours by 6.43%, and increased message hours by 27.71%.
Takeaways & Limitations
Observed pandemic-era collaboration changes were mainly attributable to factors other than WFH, while WFH under normal circumstances likely decreases collaboration and increases focus time.
Takeaways & Limitations
There were not enough previously office-based employees with 100% remote collaborators to separate physical homeworking from differences between 70% and 100% remote collaboration.
Abstract
from arXiv · showhide
The COVID-19 pandemic has had a wide-ranging impact on information workers such as higher stress levels, increased workloads, new workstreams, and more caregiving responsibilities during lockdown. COVID-19 also caused the overwhelming majority of information workers to rapidly shift to working from home (WFH). The central question this work addresses is: can we isolate the effects of WFH on information workers' collaboration activities from all other factors, especially the other effects of COVID-19? This is important because in the future, WFH will likely to be more common than it was prior to the pandemic. We use difference-in-differences (DiD), a causal identification strategy commonly used in the social sciences, to control for unobserved confounding factors and estimate the causal effect of WFH. Our analysis relies on measuring the difference in changes between those who WFH prior to COVID-19 and those who did not. Our preliminary results suggest that on average, people spent more time on collaboration in April (Post WFH mandate) than in February (Pre WFH mandate), but this is primarily due to factors other than WFH, such as lockdowns during the pandemic. The change attributable to WFH specifically is in the opposite direction: less time on collaboration and more focus time. This reversal shows the importance of using causal inference: a simple analysis would have resulted in the wrong conclusion. We further find that the effect of WFH is moderated by individual remote collaboration experience prior to WFH. Meanwhile, the medium for collaboration has also shifted due to WFH: instant messages were used more, whereas scheduled meetings were used less. We discuss design implications -- how future WFH may affect focused work, collaborative work, and creative work.
1 Introduction
The study asks whether WFH’s effects on information workers’ collaboration can be separated from other pandemic and seasonal changes. Using the COVID-19 shift as a natural experiment, it applies causal inference to characterize collaboration and focus behavior.
- Motivation: WFH effects cannot be inferred from simple before-and-after pandemic changes because COVID-19 introduced caregiving, stress, workload, and other confounding factors.Seasonality and other simultaneous changes could also affect work behavior.
- Motivation: The study uses the rapid COVID-19 shift to WFH as a natural experiment relevant to both pandemic work and future remote-work arrangements.The authors motivate the question by noting that WFH may become more common after COVID-19.
- Approach: Difference-in-differences compares changes among workers already working from home with changes among those previously working in offices to isolate WFH effects.The analysis also examines moderation by prior remote collaboration experience.
- Findings: WFH decreased collaboration hours by 2.0% and increased focus hours by 3.6%, despite higher average collaboration and lower focus time in April than February.The observed pandemic-period changes therefore pointed in the opposite direction from the WFH-specific estimates.
- Findings: WFH shifted collaboration from scheduled meetings to messaging while leaving email hours stable.Meeting hours decreased by 6.4%, whereas messaging hours increased by 27.7%.
- Implications: The findings suggest WFH may benefit focused work but hinder highly collaborative work, while demonstrating that causal inference is essential for interpreting pandemic-era changes.Simple pandemic-period comparisons can imply the opposite sign of the actual WFH effect.
2 Related Work
Prior telecommuting evidence is often noncausal because it relies on cross-sectional data. This study positions itself as broader and more behaviorally detailed than earlier causal studies.
- Prior evidence: Most prior telecommuting and remote-work studies use cross-sectional data, which cannot directly establish causal WFH effects.The paper contrasts this evidence base with two large studies using experimental variation.
- Prior evidence: Earlier experimental studies examined specific settings, including a travel company’s call center and employees of the US Patent Office.The cited call-center study found a 13% increase in calls answered among workers assigned to WFH.
- Contributions: This study examines finer-grained work practices such as collaboration and meeting time rather than only broad productivity outcomes.Its contribution is explicitly framed relative to prior causal work.
- Contributions: The study measures WFH effects for mandatory pandemic-era remote work among a broad population of information workers.This differs from evidence focused on volunteers or a single specific profession.
3 Methodology
The methodology uses difference-in-differences to separate WFH effects from group differences and shared time shocks. It strengthens identification through matching and explicitly models assumptions and prior remote-collaboration experience.
- Identification: Simple comparisons between WFH and office workers are biased by job and working-style differences, while before-and-after comparisons confound WFH with pandemic and seasonal changes.The methodology motivates a design that addresses both cross-group and over-time confounding.
- Identification: Difference-in-differences separates WFH’s causal impact by comparing outcome changes across groups and time.The model includes group effects, time effects, a treatment indicator, and a residual.
- Assumptions: The treatment effect requires parallel counterfactual trends and equal average non-WFH COVID-19 effects across previously remote and office-based workers.These are the study’s two identifying assumptions.
- Design: Previously WFH workers serve as the control group, while the additional change among previously office-based workers is attributed to switching to WFH.The control-group change captures seasonality, product cycles, social isolation, and related factors.
- Estimation: Coarsened exact matching balances job role, managerial status, and seniority between groups and removes approximately 1% of employees.The procedure is intended to improve robustness to observable deviations from parallel trends and increase precision.
- Heterogeneity: The analysis estimates heterogeneous WFH effects by separating workers with less and more prior remote collaboration experience.The authors hypothesize larger effects for employees with less remote-collaboration experience.
4 Dataset
The dataset uses Workplace Analytics software-usage logs to measure collaboration-related outcomes for Microsoft US employees. February and April averages represent pre-treatment and post-treatment observations around the March 2020 WFH mandate.
- Data source: Workplace Analytics aggregates Microsoft US employees’ software-usage patterns for measuring collaboration-related outcomes.The study uses these logs to estimate WFH effects on collaboration metrics.
- Measures: The measured metrics are focus hours, email hours, message hours, meeting hours, and collaboration hours.These metrics define the study’s collaboration-related outcome measures.
- Time periods: February and April monthly averages serve as the pre-treatment and post-treatment observations because the US WFH mandate took effect in March 2020.The analysis aggregates each worker’s outcomes into these two periods.
5 Findings and results
WFH lowered collaboration and increased focus time, while shifting collaboration from scheduled meetings toward messaging. These effects varied with employees’ prior remote collaboration experience, but the study could not fully separate remote-collaboration needs from physically working at home.
- 1.95% lower collaboration hours and 3.57% higher focus hours were caused by WFH.These causal estimates contrast with the higher collaboration and lower focus hours observed in April relative to February.
- 6.43% fewer meeting hours and 27.71% more message hours indicate that WFH shifted collaboration tools.Email hours showed no statistically significant WFH effect.
- WFH effects were larger for employees with less prior remote collaboration experience than for those with more experience.For less experienced employees, total collaboration and meeting hours fell 3.1% and 9.23%, while message hours rose 38.5%; the corresponding message increase for more experienced employees was 5.24%.
- The difference between experience groups was statistically significant for every outcome except email hours.Neither experience group showed a significant email-hours effect.
- The stronger effects among less experienced remote collaborators suggest that some WFH effects reflect the need to collaborate remotely rather than only working physically at home.There were not enough office-based employees with 100% remote collaborators before COVID-19 to separate these effects.
6 Implications for the future of remote work
The study cautions against generalizing pandemic-era workplace disruptions to WFH and identifies mixed implications for focused, collaborative, and creative work. It also points to productivity-tool designs that accommodate remote communication and individual differences.
- Pandemic-era disruptions are mostly attributable to factors other than WFH, so raw changes should not directly predict future remote-work outcomes.
- Normal WFH may increase free-time blocks for focused work while reducing meeting time and isolating workers collaboratively.
- Because creative work requires both deep thought and frequent idea exchange, WFH has mixed implications for innovation and creation.
- Remote productivity tools could support rapid messaging, replace some informal in-person interaction, and personalize assistance based on prior remote collaboration experience.
7 Conclusions
The study uses a large-scale COVID-19 natural experiment and difference-in-differences analysis to isolate WFH effects from concurrent pandemic factors. It concludes that pandemic-era changes were mainly driven by other factors, while normal WFH likely decreases collaboration and increases focus time.
- The study applies difference-in-differences causal inference to control for confounding factors occurring concurrently with WFH.
- Observed pandemic changes were mainly due to factors other than WFH, while normal WFH likely decreases collaboration and increases focus time.
- Future work will examine broader measures of work and productivity.