PaperlayerView exact revision on arXiv

Source-linked AI summary

Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysis

Saif M. Mohammad

arXiv:2109.08256v3cs.CLcs.AI

TL;DR

AER and sentiment-analysis systems may support progress but also enable serious harms, so their ethical implications warrant attention. This paper synthesizes AI ethics and emotion-recognition literature into an ethics sheet that organizes considerations and recommendations. It surfaces assumptions in task framing and development choices, with particular attention to privacy and social groups.

  • Problem

    AER and sentiment-analysis systems can facilitate progress while enabling harm, raising ethical questions for the affective-computing community.

    Source: §1 · p. 1

  • Method

    The paper synthesizes and organizes AI Ethics and emotion-recognition literature into an ethics sheet that examines assumptions in AER framing and choices about data, methods, and evaluation.

    Source: §1 · p. 2

  • Results

    The sheet presents task-relevant ethical considerations, including assumptions in common AER framing and choices concerning data, methods, and evaluation.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The sheet encourages reflection on why and how to automate and how to judge success before building AER systems, while helping readers navigate choices and work toward consensus.

    Source: §1 · p. 2

    The author presents the sheet as one perspective rather than an objective or definitive ethics sheet and welcomes dissenting views.

    Source: §1 · p. 3

Abstract

from arXiv · show

The importance and pervasiveness of emotions in our lives makes affective computing a tremendously important and vibrant line of work. Systems for automatic emotion recognition (AER) and sentiment analysis can be facilitators of enormous progress (e.g., in improving public health and commerce) but also enablers of great harm (e.g., for suppressing dissidents and manipulating voters). Thus, it is imperative that the affective computing community actively engage with the ethical ramifications of their creations. In this paper, I have synthesized and organized information from AI Ethics and Emotion Recognition literature to present fifty ethical considerations relevant to AER. Notably, the sheet fleshes out assumptions hidden in how AER is commonly framed, and in the choices often made regarding the data, method, and evaluation. Special attention is paid to the implications of AER on privacy and social groups. Along the way, key recommendations are made for responsible AER. The objective of the sheet is to facilitate and encourage more thoughtfulness on why to automate, how to automate, and how to judge success well before the building of AER systems. Additionally, the sheet acts as a useful introductory document on emotion recognition (complementing survey articles).

1. Introduction

AER can facilitate progress while enabling serious harm, making ethical engagement essential. The paper organizes task-specific ethical considerations to expose assumptions and clarify choices before systems are built.

1. Introduction

  • The Ethics Sheet organizes ethical considerations around task framing, data, methods, evaluation, tradeoffs, and harm mitigation.It also communicates societal implications and outlines considerations specific to AER as well as common ethical issues in the task.Source: §1 · p. 2
  • The sheet encourages earlier reflection on why and how to automate and how to judge success, while helping readers navigate choices and work toward consensus.It can also inform post-production documents, provide pointers for further reading, engage stakeholders, and serve as an introduction to emotion recognition.Source: §1 · p. 2
  • Many considerations extend beyond AER to natural-language tasks, making the sheet a possible template for ethics sheets in other areas.The paper also situates the sheet among other resources by describing its organization and the recommendations collected in the appendix.Source: §1 · pp. 2–3
  • The author presents the sheet as one perspective rather than an objective or definitive account, and welcomes dissent and future revisions.The author notes that discussions with scholars and community feedback shaped the document.Source: §1 · p. 3

2. Preface for the Ethics Sheet on AER

  • The sheet does not give a single yes-or-no answer about building AER; it helps clarify context-specific questions about whether and how to build or use systems and assess them.The question is under-specified because emotion recognition has different forms and can be deployed in different contexts.Source: §2 · p. 3

Q2. What does automatic emotion recognition mean?

  • Emotion recognition can refer to different capabilities and contexts: automated insurance decisions may be inappropriate, while studying emotional expression in language is considered acceptable in many contexts.Human–computer interaction systems may use recognition of utterances conveying emotions to support more appropriate interactions.Source: Q2. What does automatic emotion… · p. 3

Q3. Can machines infer one’s true emotional state ever?

The sheet rejects inferring a person's true emotional state as distinct from making limited, context-specific inferences, and organizes ethical considerations across AER tasks, modalities, applications, and development choices. It focuses on written-text AER while noting that many considerations apply more broadly.

Q3. Can machines infer one’s true emotional state ever?

  • The sheet answers no to inferring a person's true emotional state, while allowing that limited, context-specific inferences about emotions may be useful.It recommends recognizing AER's limitations, protecting and empowering people without power, using systems in designed contexts, and mitigating harms.Source: Q3. Can machines infer one’s true… · p. 4

3. Main Sheet (version 1.0)

  • The main sheet organizes AER into four sections, with its fourth presenting fifty ethical considerations in thematic groups.Privacy and social-group considerations are especially pertinent and cut across task, design, data, and evaluation.Source: §3 · p. 4, §3.4 · p. 7

3.1 Modalities and Scope

  • AER draws on modalities including facial and bodily signals, physiological measurements, touch, speech, and written language; this sheet focuses on written text and NLP.The sheet notes that many considerations apply across modalities and fields.Source: §3.1 · p. 4

3.2 Task

  • AER task framings include inferring a speaker's felt, perceived, or conveyed emotions, their intensity or longer-term patterns, and their sentiment toward a target.These framings can differ for a particular instance, and the sheet notes that inferring felt emotions is often misused or misinterpreted.Source: §3.2 · p. 5
  • Other tasks infer emotions evoked in readers or attributed to people mentioned, emotionality or emotional expression in language, and emotional impact from figurative or harmful language.The same text may evoke different feelings among readers with different experiences and worldviews.Source: §3.2 · pp. 5–6
  • The sheet distinguishes product-review sentiment analysis from stance detection about policies or issues, and notes that systems may target emotionality instead of their stated task.Emotionality-focused systems may be appropriate in restricted contexts such as reviews or personal diary blogs, but not always.Source: §3.2 · p. 5

3.3 Applications

  • Potential applications span public health, commerce, policy, art and literature, social sciences, and military, policing, or intelligence uses.The sheet does not endorse these applications and identifies military, intelligence, and education uses as especially controversial.Source: §3.3 · p. 6

3.4 ETHICAL CONSIDERATIONS

  • The sheet groups ethical considerations around task design, data, method, impact and evaluation, and privacy and social groups.It describes a development process involving task design, data, model training, and held-out evaluation, with privacy and social-group concerns cutting across stages.Source: §3.4 · pp. 6–7

I. TASK DESIGN

The four thematic summaries frame ethical questions across task design, data, method, and impact and evaluation. They emphasize hidden assumptions, human variability, methodological tradeoffs, and planning for harms even when systems work as designed.

I. TASK DESIGN

  • Task design requires considering whether internal states can be determined, whether doing so is ethical, who is excluded, and whether appearances should map to inner states.The section highlights choosing an appropriate emotion formulation for the particular project.Source: §I · p. 7

II. DATA

  • Data choices raise issues of dataset type, human variability versus machine normativeness, and the people who produced the data.The section notes that systems can implicitly convey which forms of emotion expression are normal and invalidate others.Source: §II · p. 7

III. METHOD

  • Method choices involve tradeoffs and questions about exclusion, spurious correlations, context, emotion dynamics, manipulation, and green AI.The section also considers whether historical behavior indicates future behavior.Source: §III · p. 8

IV. IMPACT AND EVALUATION

  • Impact and evaluation extend beyond traditional metrics to interpretability, visualization, safeguards, and contestability, while planning for harms even when systems work as designed.The section also raises reliability, demographic bias, sensitive applications, and testing across diverse datasets and metrics.Source: §IV · p. 8

V. IMPLICATIONS FOR PRIVACY, SOCIAL GROUPS

The sheet treats privacy and social-group effects as cross-cutting AER concerns, including group privacy from soft biometrics and risks of ignoring subgroup differences. It also urges careful task framing: AER should not be presented as determining internal emotional states, and individual-level inferences require particular caution.

V. IMPLICATIONS FOR PRIVACY, SOCIAL GROUPS

  • Privacy and social-group implications cut across task, data, method, and impact; group privacy matters for soft biometrics, and analysis should consider subgroup differences without reifying social constructs.The section calls for disaggregation and intersectionality while minimizing essentialization.Source: §V · p. 8

3.4.1 TASK DESIGN. (Ten considerations.)

  • AER should not claim to determine a person’s emotional state; at best it captures conveyed or perceived emotion, often inaccurately, while aggregate trends also warrant caution.For public-health inferences, the sheet recommends treating AER as one source of evidence alongside expertise from public health and psychology.Source: §3.4.1 · p. 10
  • Emotion formulations should fit the task; categorical emotion work does not itself endorse Basic Emotions Theory, whose universality and fixed facial mappings are disputed.Careless endorsement can perpetuate the idea that internal states can be determined from outward appearance.Source: §3.4.1 · p. 11
  • Emotion varies with context and extra-linguistic information, so systems that ignore it risk being systematically biased, insensitive, and discriminatory.Source: §3.4.1 · p. 12
  • Current AER systems cannot handle health-related inter- and within-person variability and should not be used where their decisions could negatively affect people.The sheet describes patient tracking as assistive to clinicians and psychologists, who make decisions using multiple sources.Source: §3.4.1 · p. 12
  • Individual predictions often substitute group data for personal evidence, risking errors and majority-group assumptions; these errors are especially consequential in direct interactions.Privacy, manipulation, bias, and free-speech concerns are further exacerbated when systems act on individuals; aggregate trends draw on more relevant information but still raise group-privacy concerns.Source: §3.4.1 · p. 12

B. Implications of Automation

The section asks researchers to examine who benefits from automation, how it may shift power, and whether the application is appropriate. It highlights risks to marginalized groups and sensitive domains, while calling for participatory design and disclosure of automated decisions.

What are the ethical implications of automating the chosen task?

  • Before automating, ask who benefits, who does not, and whether the work shifts power; also consider reframing the task to help those most in need.Source: What are the ethical implications of… · p. 13
  • Existing AER work often assumes neurotypical users and overlooks neurodiversity, alexithymia, and autism; report disaggregated performance for relevant groups.The sheet says existing systems implicitly cater to the more populous neurotypical group and should acknowledge this limitation.Source: What are the ethical implications of… · p. 13
  • Participatory design gives marginalized communities agency to shape research and system development rather than treating them as passive subjects.The approach is also called emancipatory research and is summarized by “nothing about us without us.”Source: What are the ethical implications of… · pp. 13–14
  • AER can be misused for applications such as inferring danger from facial expressions, persecuting dissidents, or targeting emotionally vulnerable people with advertising.Source: What are the ethical implications of… · p. 14
  • Personality-based hiring and team decisions are ethically fraught: personality measures have limitations, and evidence that such decisions benefit organizations is scant.The use of these tests has also been criticized on discrimination grounds.Source: What are the ethical implications of… · p. 14
  • Health and well-being applications require safeguards for error, privacy, and severe-risk disclosures because their sensitive decisions can carry costs that are hard to value.Source: What are the ethical implications of… · p. 14
  • AER in art and culture raises questions about automated creation, collaboration, artists’ impacts, and credit.Source: What are the ethical implications of… · p. 15
  • Disclose automated decisions to stakeholders and provide ways to understand and contest predictions; human-like agents should identify themselves as artificial.The sheet notes that studies have found some negative outcomes from such disclosure.Source: What are the ethical implications of… · p. 15

C. Why This Data

The section frames data choice as an ethical decision shaped by the data’s source, scale, sensitivity, language coverage, and documentation. It also flags risks in both large language-model corpora and emotion lexicons, as well as the role of manually labeled examples.

What are the ethical implications of using the chosen data?

D. Human Variability vs. Machine Normativeness

The section emphasizes that people express and represent emotions variably, while machine systems can privilege a narrow set of expressions as normal. It discusses dataset and annotation choices that may reinforce this normativity, and argues for useful recognition without systematic gaps.

D. Human Variability vs. Machine Normativeness

  • People express the same emotion differently, and the same text may convey different emotions to different readers; shared commonalities coexist with substantial variation.The variation should inform dataset construction, system design, and deployment choices.Source: D. Human Variability vs. Machine… · pp. 16–17
  • Systems trained on limited expressions can signal which forms of emotional expression are normal, implicitly invalidating others.Source: D. Human Variability vs. Machine… · p. 17
  • The goal is useful emotion recognition across diverse instantiations, without systematic gaps that convey normativeness; perfect recognition is unlikely given expression variability.Source: D. Human Variability vs. Machine… · p. 17
  • Influential films can shape global expectations about normal emotional expression, either reducing diversity or validating individual self-expression.Source: D. Human Variability vs. Machine… · p. 17
  • Majority-vote labels can privilege majority attitudes and reproduce stereotypes, while retaining all annotations also raises challenges such as handling racist or sexist opinions.The section notes that emotion data may have multiple appropriate answers rather than one correct label.Source: D. Human Variability vs. Machine… · p. 18
  • Historical datasets over-represent powerful and Western, English-speaking, white, and able-bodied people, so dataset review should examine who is included and whose voices are missing.The section also identifies training–deployment differences in domain, time, or class distribution as a likely source of poor predictions.Source: D. Human Variability vs. Machine… · p. 19

E. The People Behind the Data

The section considers the people represented in AER data and the ethical trade-offs of methods used to analyze it. It highlights risks to data contributors and to groups poorly represented in training data.

E. The People Behind the Data

  • Data used to build AER systems comes from people and can contain extensive personal information.The sheet calls for anonymization and mechanisms for people to remove their data, and warns that annotating highly emotional or harmful content can affect annotators’ well-being.Source: E. The People Behind the Data · pp. 19–20
  • Crowdsourced labeling raises concerns beyond compensation, including worker invisibility, well-being, and rights.The sheet also flags limited learning trajectories and the humans-as-a-service paradigm.Source: E. The People Behind the Data · p. 20

What are the ethical implications of using a given method?

  • Method choices trade off accuracy against interpretability, energy use, data needs, privacy, and bias, so priorities should reflect deployment context.The sheet notes that widely deployed systems often need a balance across dimensions, whereas research systems may focus on fewer.Source: What are the ethical implications of… · p. 20
  • Large pretrained models tend to work better for people well represented in raw and annotated data than for others.The sheet recommends documenting who is left out and exploring more inclusive methods.Source: What are the ethical implications of… · pp. 20–21
  • AER systems can learn spurious correlations that associate identity terms such as gay, Muslim, and Jew with offensive language.The cited example links this behavior to data-collection methods combined with the machine-learning algorithm.Source: What are the ethical implications of… · p. 21
  • More context can improve emotion judgments, but collecting more information about a person raises privacy concerns.The sheet also presents utterance emotion dynamics as useful contextual information while emphasizing care, concern, and meaningful consent.Source: What are the ethical implications of… · pp. 21–22

#29. Historical behavior is not always indicative of future behavior (for groups

The section warns that systems trained on past data may make inappropriate predictions as people’s perceptions, emotions, and behavior change. It also considers emotional management and evaluation beyond accuracy alone.

#29. Historical behavior is not always indicative of future behavior (for groups

  • People’s perceptions, emotions, and behavior change over time, so systems trained on static historical data may make inappropriate predictions on current data.The stated concern applies to groups as well as individuals.Source: #29. Historical behavior is not always… · p. 22
  • Emotion-sensitive systems can help chatbots respond appropriately, but can also enable companies or governments to manipulate people’s behavior.The sheet contrasts potentially harmful targeting when people are susceptible with uses such as supporting public-health compliance or smoking cessation, and says context and meaningful consent matter.Source: #29. Historical behavior is not always… · p. 22

G. Metrics

  • AER systems can be systematically unreliable for particular groups, including by race, gender, health condition, or country.The sheet recommends reporting disaggregated accuracies and addressing performance disparities.Source: G. Metrics · p. 23
  • More sensitive applications require higher-quality emotion recognition, if such systems are used at all.The sheet gives assisting patients and health experts in tracking a patient’s emotional state as an example of a possible human-support role.Source: G. Metrics · p. 23
  • Test-set results may not reflect real-world performance or systematic errors, so evaluation should cover diverse datasets and metrics.The sheet recommends datasets that explore varied input characteristics.Source: G. Metrics · p. 23

H. Beyond Metrics

The section argues that responsible AER requires examining what evaluation captures, making systems and their impacts open to scrutiny, and addressing privacy risks. It also highlights harms that can arise even when systems work as designed.

Are we even measuring the right things?

  • Understanding why a system makes a prediction can help people judge its validity and accuracy across input types and over time.The sheet encourages explanations that are faithful, comprehensible, and suited to their intended audience.Source: Are we even measuring the right things? · pp. 23–24
  • Visualizations should expose the data behind trends, while safeguards should anticipate misuse and harms from system limitations and biases.The sheet recommends letting users drill down to source data and providing resources for unanticipated harms, including when a system works correctly.Source: Are we even measuring the right things? · p. 24
  • Contestability tools can let people inspect counterfactuals, test inputs, compare demographic accuracies, and examine prediction confidence.The sheet describes these mechanisms as ways to challenge system predictions and participate in understanding system behavior and limitations.Source: Are we even measuring the right things? · pp. 24–25
  • Ethics work should avoid cursory attention undertaken merely for appearances.The sheet encourages constructive dissent, sharing shortcomings, and acknowledging gray-area decisions and the scarcity of clear ethics metrics.Source: Are we even measuring the right things? · p. 25

I. Implications for Privacy

  • Emotion detection applied at scale without meaningful consent can invade privacy and harm individuals and society.The sheet frames privacy as individual choice and recommends privacy-by-design principles.Source: §I · p. 25
  • AER-based soft biometrics can identify groups and preferences, making group privacy relevant even when systems do not aim to identify individuals.The sheet also warns that emotion recognition and related methods can be used for mass surveillance, with extreme cases curtailing expression and protest rights.Source: §I · pp. 25–26

J. Implications for Social Groups

The section addresses how AER research can overlook or mischaracterize social groups, and calls for careful disaggregation and contextualization. The paper’s broader ethics sheet organizes fifty considerations and aims to prompt reflection before systems are built.

J. Implications for Social Groups

  • Disaggregation can reveal overlooked needs, but demographic reporting and multiple annotation perspectives should be considered across dataset creation, hypothesis testing, and system evaluation.The sheet connects overlooked group-specific needs with unequal social and power structures.Source: J. Implications for Social Groups · p. 26
  • Research organized around neat, populous categories can omit people whose identities overlap, leaving their experiences underrepresented.The sheet calls for valuing research approaches that address the particular challenges of intersectional work.Source: J. Implications for Social Groups · p. 27
  • Disaggregation can reinforce false beliefs about innate group differences or which traits define group membership, so findings need context.The paper notes that race is socially constructed while perceptions and behavior around race have real-world consequences.Source: J. Implications for Social Groups · p. 27
  • Collecting demographic information for disaggregated results raises questions about consent, privacy, autonomy, dignity, and how respondents’ data are later used.The sheet notes that downstream research can ignore self-descriptions or combine information beyond respondents’ control.Source: J. Implications for Social Groups · p. 27

4. In Summary

  • The paper synthesizes fifty ethical considerations for AER, emphasizing privacy and social groups alongside data, method, and evaluation choices.It aims to encourage early reflection on why to automate, how to automate, and how to judge success.Source: §4 · pp. 27–28

APPENDIX: Recommendations for Responsible AER

The recommendations treat AER as an imperfect account of what people convey or others perceive, not a measure of their full inner emotional experience. They urge context-sensitive, inclusive design across task framing, data, methods, deployment, and evaluation; following the guidelines does not guarantee ethicalness.

Task Design

  • Center affected people’s agency and power, consider neurodiversity, and seek ethics-board review when annotation work may affect annotator well-being.The Belmont Report’s principles of Respect for Persons, Beneficence, and Justice are offered as a starting point for human-subject research.Source: Task Design · pp. 28, 30
  • Specify the emotion task and its theoretical foundations, communicate its nuance, and avoid endorsing Basic Emotions Theory inadvertently.The recommendations warn that careless theoretical endorsement can reinforce the harmful idea that internal states can be determined from outward appearance.Source: Task Design · p. 29
  • AER captures what someone conveys or what an observer perceives, and is often inaccurate; it cannot capture a person’s full emotional experience.The recommendations caution against claiming that AER determines a person’s emotional state from speech, facial expression, or gait.Source: Task Design · p. 29
  • Keep AER from making fully automated decisions; use it as one input for human judgment, disclose automated decisions, and enable explanation and contestation.The recommendations also warn that AER can inappropriately bias human decision makers and can be misused.Source: Task Design · p. 29
  • Examine data sources and sensitivities, document them, study whose voices are missing, and limit how many instances come from each person.Relevant data dimensions include size, source, privacy sensitivity, language representation, and documentation.Source: Task Design · pp. 29–30
  • Because emotional expression and annotation can vary, document annotation conditions, preserve raw and aggregated labels, and consider multiple appropriate labels.The recommendations note that emotion and language data may have no single correct answer and that several answers can be appropriate.Source: Task Design · pp. 29–30
  • Check for systematic gaps, stereotypes, and annotation bias; assess representation and consider corrective action and more inclusive methods.The recommendations also call for diverse annotators, demographic reporting, and resources for work with marginalized communities and less-researched languages.Source: Task Design · pp. 29–31
  • Balance method choices across interpretability, privacy, energy, and data needs, while accounting for changing behavior, deployment context, data rights, and open-domain testing.For open-domain deployment, train on diverse datasets and test on datasets substantially different from training data; report efficiency measures such as costs per example and budget-accuracy curves.Source: Task Design · pp. 30–31

Impact and Evaluation

Evaluation should measure what matters, examine performance across groups and datasets, and make system behavior and potential harms inspectable. The recommendations also emphasize explainability and meaningful opportunities to contest system decisions.

Impact and Evaluation

  • Choose metrics that measure what matters, and report disaggregated accuracies across relevant groups and varied datasets.The recommendations note that some methods can be unreliable or systematically inaccurate for particular groups.Source: Impact and Evaluation · p. 31
  • Plan explainability from the outset using interpretable models, ablations, disaggregation, data perturbations, and adversarial tests as appropriate.These strategies are presented as examples of ways to examine a system’s behavior.Source: Impact and Evaluation · p. 31
  • Emotion visualizations should show broad trends while letting users inspect the source data driving them.Frequent emotion words are one suggested way to summarize the underlying data.Source: Impact and Evaluation · p. 31
  • Investigate misuse and harms arising from system biases and limitations, including harms that may occur when a system works correctly.The recommendations call for identifying steps to mitigate those harms.Source: Impact and Evaluation · p. 31
  • Make contestability support both challenges to decisions about people and participation in understanding how machine-learning systems work and their limitations.Source: Impact and Evaluation · p. 31

Implications for Privacy

The recommendations treat privacy as a matter of personal choice and warn that nonconsensual emotion inference and mass surveillance can threaten privacy and other rights. They also call for demographic-aware research that respects privacy, autonomy, and dignity.

Implications for Privacy

  • Treat privacy as personal choice and consider privacy-by-design principles.Source: Implications for Privacy · p. 31
  • Continuous emotion inference without meaningful consent is described as an invasion of privacy and harmful to individuals and society; mass surveillance also threatens multiple rights.The listed rights include freedom of expression, protest, protection against self-incrimination, and non-discrimination.Source: Implications for Privacy · pp. 31–32
  • Soft-biometrics raise group-privacy concerns, and many people disfavour this kind of profiling.Source: Implications for Privacy · p. 31
  • Match consent to context and data sensitivity; anonymization and differential privacy are possible harm-mitigation strategies, but their effectiveness varies.Source: Implications for Privacy · pp. 31–32
  • Plan for information security, obtain permission for secondary use or distribution, and assess who benefits when weighing privacy against benefits.The recommendations emphasize whether beneficiaries have more or less power and caution that privacy and benefits need not be a zero-sum tradeoff.Source: Implications for Privacy · p. 32

Implications for Social Groups

  • Use diverse annotators and report aggregate demographics; consider multiple label sets, and disaggregate hypotheses and system performance for relevant groups.Source: Implications for Social Groups · p. 32
  • Consider and report intersectional implications, while contextualizing disaggregation and the real-world consequences of perceptions and behavior around race.Source: Implications for Social Groups · p. 32
  • Collect demographic information with attention to meaningful consent, privacy, autonomy, deletion choices, and self-description.Source: Implications for Social Groups · p. 32
Loading 2109.08256v3…