Do feelings matter? On the correlation of affects and the self-assessed productivity in software engineering

1. INTRODUCTION

• Software engineering research has limited knowledge of how affects relate to individual developers’ performance, especially in real time.The paper frames affects as emotions, moods, and feelings linked to cognitive activities.Source: §1 · pp. 2–3
• The study examines real-time variation in affects and self-assessed productivity while developers program, focusing on valence, arousal, and dominance.Its stated context is students and professional developers working on their own projects in natural settings.Source: §1 · p. 3
• Valence and dominance are positively correlated with software developers’ self-assessed productivity.The study also presents psychological measurement and linear mixed-effects models as useful for empirical software engineering with within-participant repeated measures.Source: §1 · p. 3
• The article reports guidance on affect theories, classification, measurement, and psychological measurement practices in empirical software engineering.It is an extension of a conference article and describes its sections on background, methodology, analysis, discussion, and conclusions.Source: §1 · p. 3

2.1.1 Human aspects in productivity studies

• Prior software productivity research identified technical, knowledge, and developer characteristics, while later work emphasized organizational, social, and people-related factors.Reviews also noted that productivity variation among individual programmers remained poorly described.Source: §2.1.1 · p. 4

2.1.2 Affects, emotions, moods, and feelings

• The paper uses affective states as an umbrella for emotions, moods, and feelings, and contrasts discrete categories with dimensional accounts.The dimensional approach characterizes affect through valence, arousal, and dominance and is selected for quick, repeated assessment.Source: §2.1.2 · pp. 5–6

2.1.3 Measuring affects

• The Self-Assessment Manikin measures valence, arousal, and dominance with pictorial ratings suited to brief assessments.It was chosen for its short questionnaire and prior use, including in computer science research.Source: §2.1.3 · p. 6
• Participant-specific z-scores make repeated affect ratings comparable across people, while mixed-effects models address participant- and time-level dependencies.The cited methodological discussion discourages ANOVA procedures in favor of mixed-effects models for repeated within-participant longitudinal data.Source: §2.1.3 · pp. 6–7

2.2 Related studies

• Studies outside software engineering associated positive affect or mood with better task performance, including shorter call times and productivity gains in a controlled experiment.Other research sampled workers across varied occupations or call-center employees.Source: §2.2 · pp. 7–8
• Other software engineering work documented affect changes during student project work and called for further research on developers’ affective states.A study of IT professionals treated each student as a case study and did not use statistical analysis.Source: §2.2 · p. 8
• Software engineering studies examined affect in debugging, requirements work, developer surveys, problem-solving tasks, and music-listening field work.Findings varied by task and measure; the music study reported a nonsignificant correlation between positive affects and self-assessed performance.Source: §2.2 · pp. 8–9

3.1 Participants

• Participants could be students or professionals working individually on a software project, with no restrictions on gender, age, or nationality.Participation was voluntary and unrewarded, with confidentiality and anonymity assured.Source: §3.1 · p. 10

3.2 Experimental material

• The measurement setup uses a researcher-operated tablet questionnaire, with a notepad and audio recorder for interviews and observations.The pictorial questionnaire requires four screen touches per measurement session.Source: §3.2 · p. 10

3.3 Procedure

• The procedure combines pre- and post-task interviews with 90 minutes of programming and questionnaires after each 10-minute interval.Valence, arousal, dominance, and self-assessed productivity are each measured nine times per participant while the researcher observes without interfering.Source: §3.3 · pp. 10–11
• The pre-task interview records demographics, project and task information, participant role, and perceived language and domain experience.The experiment is conducted individually, and its steps are automated through electronic systems.Source: §3.3 · pp. 10–11

3.4 Goals, hypotheses, parameters, and variables

• The hypotheses predict positive correlations between each of valence, arousal, and dominance and self-assessed productivity.Affects use five-point constructs, while productivity is a five-point self-rating from very low to very high.Source: §3.4 · pp. 11–12
• The analysis converts each participant’s construct measurements to individual z-scores before modeling the repeated observations.The paper describes the practical range of z-scores as [-3, +3] under the three-sigma rule.Source: §3.4 · p. 12

3.5 Analysis procedure

• Repeated measurements depend on participant and time, motivating a linear mixed-effects model with fixed and random effects.The paper follows a full-model approach to significance estimation rather than adding factors one at a time, which it describes as time-consuming and error-prone.Source: §3.5 · pp. 12–13
• The model treats valence, arousal, dominance, and their interaction with time as fixed effects, with participant intercepts and time slopes as random effects.This structure accounts for measurement dependencies within participants across time.Source: §3.5 · p. 13

4.1 Preparation and deviations

• The eight participants were recruited by convenience sampling, trained on the measurement instrument, and completed the study without reported deviations.The sample comprised four computer science students and four professional developers.Source: §4.1 · p. 13

4.2 Descriptive statistics

• Eight participants contributed 72 measurements; four were students and four were professional developers.Their mean age was 23.75 years (standard deviation 3.29), and seven were male.Source: §4.2 · p. 14

4.2.1 Pre-task interviews

• Participants worked on varied, non-trivial projects, and professional or student role did not always correspond to reported experience.Five used C++, two Java, and one Python.Source: §4.2.1 · p. 14

4.2.2 Repeated measurements

• Affect and productivity varied over the 90-minute sessions; valence often tracked productivity, arousal less often, and dominance more closely than arousal.Valence did not strongly predict productivity for P1, and a clear dominance-productivity correlation was not apparent for P6.Source: §4.2.2 · pp. 14–15

4.2.3 Post-task interviews

• Interviews found that participants could rate productivity but generally could not define it, while most denied that affects influenced task performance.Six linked ratings to progress against expectations; seven reported technical difficulties. None found the measurement procedure disturbing, but all expected 10-minute intervals to reduce productivity. Three participants were dissatisfied with their task performance.Source: §4.2.3 · pp. 15–16

4.3 Hypotheses testing

• Valence and dominance were positively correlated with self-assessed productivity, and the model explained 38.17% of deviance.The full model also significantly outperformed its corresponding null model; there was no evidence supporting the arousal hypothesis.Source: §4.3 · p. 16

5.1 Evaluation of results and implications

• Valence and dominance were positively correlated with self-assessed productivity; high task happiness and perceived adequate skills showed the same relationship.Source: §5.1 · pp. 16–17
• The findings align with Fisher and Noble but differ from aspects of Khan et al.; the authors describe this as the first study in natural, real-software settings.Source: §5.1 · p. 17
• Post-task interviews triangulated the quantitative results, and participants’ lack of awareness that affects were being assessed was reported as supporting reliability.Source: §5.1 · p. 17

5.2 Threats to validity

• The study’s eight-person sample limits inference, although 72 repeated measurements were considered valuable and the statistical results remained significant after Bonferroni correction.Source: §5.2 · p. 18
• Convenience sampling limits generalizability, and participants worked individually to better control measurements despite the affective complexity of group work.Source: §5.2 · p. 19

5.3 Lessons learned

• Ten-minute affect and productivity self-assessments were judged suitable for one session, not continuous industry use; longer sessions need suitable intervals.Source: §5.3 · p. 19
• For complex mixed-effects models, the authors recommend LMERConvenienceFunctions for significance testing and keeping random effects consistent in likelihood-ratio comparisons.Source: §5.3 · p. 20
• The full model was retained as an example; a reduced significant model did not fit significantly differently, so reducing it offered no advantage.Source: §5.3 · pp. 20–21

6.1 Summary

• The model explained about 38% of self-assessed productivity deviance, with valence and dominance providing nearly all of its explanatory power.Source: §6.1 · p. 21

6.2 Impact

• To the authors’ knowledge, this was the first study correlating developers’ affects and performance in natural settings on real-world software projects.Source: §6.2 · p. 21
• The study validates psychological measurement instruments in software engineering and recommends linear mixed-effects models for repeated measurements instead of rANOVA.Source: §6.2 · p. 21
• It presents affect theories, classifications, measurement approaches, and best practices for psychological measurement in empirical software engineering.Source: §6.2 · p. 22

6.3 Future work

• Larger studies using the same programming task could enable traditional productivity metrics and improve generalizability; mood induction could support causal research.Source: §6.3 · p. 22
• Future work includes qualitative studies and process-based studies of software teams, while the authors propose this approach as a candidate for Essence’s human aspect.Source: §6.3 · p. 22
• The study offers a new perspective for investigating human factors by examining developers’ task perceptions and affect–performance correlations.Source: §6.3 · p. 22

• Table 1 is titled “Participants and Projects Details.”Source: Table 1 · p. 30
• Table 2 is titled “Fixed Effects Estimation.”Source: Table 2 · p. 30
• Table 3 is titled “Random Effects Estimation.”Source: Table 3 · p. 30
• Figure 1 presents a timeline of the research design.Source: Figure 1 · p. 31
• Figure 2 represents the study hypotheses and fixed and random effects in the research design.Source: Figure 2 · p. 31
• Figures 3–5 show valence, arousal, and dominance, respectively, against productivity over time by participant.Source: Figure 3 · p. 32, Figure 4 · p. 32, Figure 5 · p. 33