Judgement and the Feeling of Agency When Instructing a Computer to Act

1. Introduction

• The sense of agency connects perceived control with ownership, responsibility, and causality, and is identified as a core subconstruct in two RDoC domains.The passage describes it as relevant to everyday life and how people perceive the world.Source: §1 · p. 3
• Agency can be malleable: the paper notes both disassociation from one's thoughts and actions and increased betting when gamblers initiate bets, despite unchanged odds.The cited work argues that this malleability presents opportunities and pitfalls for HCI.Source: §1 · p. 3
• Earlier HCI agency research largely focused on individual sensorimotor control, whereas this paper considers situations involving more than one actor.The paper positions this as a different approach to agency in HCI.Source: §1 · p. 3
• The paper applies human-human command-agent research to HCI, where a computer responds to a person's instruction and may set off further outcomes.Prior work found agency over another person's initial response to a command, but not necessarily the resulting outcome.Source: §1 · p. 4
• Two studies replace a human co-actor with a computer, then extend the paradigm with a more meaningful game task and measures of both agency feelings and judgements.The second study also examines instruction specificity.Source: §1 · p. 4
• People did not feel pre-reflective agency over outcomes of computer actions, but under some conditions judged themselves responsible for them.The authors argue that systems should encourage reflection and deliberative judgements about responsibility.Source: §1 · p. 4

2.1. The Sense of Agency

• A feeling of agency is a non-conceptual, pre-reflective experience, while a judgement of agency is conceptual.The distinction is attributed to Synofzik et al..Source: §2.1 · p. 5
• The feeling of agency is a pre-reflective experience that happens unconsciously when people feel in control of their voluntary actions and outcomes, letting them intuitively know “I did that”.It is a background experience, and people are often most aware of it when actions do not unfold as expected or fail to produce intended results.Source: §2.1 · p. 5
• A judgement of agency occurs consciously after an event, when a person evaluates actions and attributes responsibility or causality.The two levels are related but appear to involve different underlying brain processes.Source: §2.1 · p. 5

Judgement of Agency

• Researchers typically ask whether a person experienced agency, felt in control, or caused an outcome.These questions are asked after all actions and outcomes are complete.Source: Judgement of Agency · p. 5
• Participants usually report judgements on a Likert-style scale, though some studies use yes/no responses.The passage also notes questionnaires as a typical instrument.Source: Judgement of Agency · p. 5
• These measures assess a person's considered judgement after the event, rather than an in-the-moment report.The timing follows completion of the actions and outcomes.Source: Judgement of Agency · p. 5

Feeling of Agency

• Intentional binding is an implicit measure in which intentional action and outcome are experienced as closer together in time than they are.Interval estimation is less detailed than the Libet Clock method but quicker and better suited to complex tasks or multiple conditions, so the paper uses it.Source: Feeling of Agency · pp. 5–6

2.2. The sense of agency in HCI

• HCI research has examined how input and feedback modalities affect agency, alongside embodiment, augmentation, and measurement methods.Examples include skin-based input, voice commands, tactile and mid-air haptic feedback, and audio-based Libet Clock measurement.Source: §2.2 · p. 7
• Much HCI agency research focuses on individual sensorimotor experience; this framing does not cover situations where a computer is explicitly treated as a co-actor.Prior findings cited here associate high computer assistance with lower personal agency, even when it helps users reach their goals.Source: §2.2 · p. 7

2.3. Joint Action and the Sense of Agency

• Vicarious agency concerns agency over another person's actions, while joint agency concerns a shared sense of acting together.Studies report perceived control over another's hand movements, and intentional binding by both participants in a joint task despite explicit agency judgements by the initiator.Source: §2.3 · pp. 7–8
• Command-agent agency describes instructing another person to act; prior studies found agency over the agent's action but not the final outcome.A later study found greater agency over the ultimate outcome when acting oneself than when instructing an agent.Source: §2.3 · pp. 8–9

• The paper investigates how people experience agency when a computer follows their instruction and acts on their behalf.Such interactions can range from everyday commands to systems with significant consequences.Source: §3 · p. 9
• It asks whether people feel agency over a computer’s response and final consequence, and whether feeling differs from judgement.Source: §3 · p. 9
• The studies examine whether instruction specificity and more meaningful outcomes affect agency in computer-command situations.Source: §3 · pp. 9–10

4. Study 1

• Study 1 theoretically replicated Pfister et al. online, replacing a human co-actor with a computer and varying instruction specificity.The authors viewed this close replication as a rigorous foundation for subsequent studies.Source: §4 · p. 10

4.1. Procedure

• Each trial linked a keypress to a beep, a computer voice direction, and square movement, with interval estimates measuring agency at three stages.The general condition used the spacebar and a random direction; the specific condition used a matching arrow key and direction.Source: §4.1 · pp. 11–12, Figure 1 · p. 12

4.2. Apparatus

• The unsupervised online experiment used Testable and jsPsych, requiring participants to use a PC or laptop, keyboard, microphone, and speakers or wired headphones.The authors could not verify that participants followed the instructions for a calm, distraction-free setup.Source: §4.2 · p. 13

4.3. Participants

• Thirty-six participants were recruited through Testable, including 8 women and 28 men; most submissions came from India (26), and data from 35 participants were included and analysed.The 36 participants had a mean age of 26.44 (SD = 8.35).Source: §4.3 · p. 13, §4.4.2 · p. 15

4.4.1. Data Cleaning

• The analysis removed one participant with strongly low estimates and excluded trials with residual errors over 2.5 standard deviations, reducing data by 1.82%.The authors analyzed regression residuals rather than raw error scores because interval length predicted estimation error, though raw scores showed similar patterns.Source: §4.4.1 · pp. 13–14

4.4.2. Analysis

• The interval effect was significant, but condition and interval-by-condition effects were not; interval 3 errors exceeded intervals 1 and 2.Hypothesis 2 was not accepted because general and specific conditions did not differ significantly.Source: §4.4.2 · p. 15, §4.5 · p. 15

4.5. Discussion of Study 1

• Interval 1 estimates indicated agency over the initial beep, while interval 3 estimates indicated no agency over the final square movement, consistent with Pfister et al..Interval 2 was not significantly different from zero, complicating interpretation of the sequence.Source: §4.5 · pp. 15–16, §4.4.2 · p. 15
• Interval 2 remained difficult to interpret, and the intermediary beep and arbitrary square movement limited HCI realism; Study 2 addressed both issues.Source: §4.5 · p. 16

5. Study 2

• The game linked left/right character movements to collecting or losing points, with participants asked to maximize their accumulated total.The study aimed to make the task more meaningful than Study 1’s essentially random movements.Source: §5 · pp. 16–17, Figure 3 · p. 17

5.1. Procedure

• Each trial paired a keypress with a computer voice command and character movement; participants estimated one interval, then rated agency after 18 trials.Specific-condition participants chose a direction with an arrow key; general-condition participants received a random direction.Source: §5.1 · pp. 18–19

5.2. Apparatus

• The study used the first experiment’s software and was conducted online through Testable.Source: §5.2 · p. 19

5.3. Participants

• Fifty participants were recruited online, but the researchers could not verify whether they followed the environmental and audio instructions.The instructions requested a calm environment and wired headphones or speakers.Source: §5.3 · p. 19

5.4.1. Data Cleaning

• Cleaning removed one participant, two interval-estimate outliers, specific-negative entries, and 1.6% of entries with extreme residual errors.Specific-negative entries were excluded because fewer than five remained per participant.Source: §5.4.1 · pp. 19–20

5.4.2. Analysis

• Interval 1 errors were below zero (M = -41.90, SD = 100.52; p = .006), while interval 2 showed no binding (p = .059).The interval-by-condition analysis found no significant condition effect or interaction.Source: §5.4.2 · pp. 20–21

5.5. Discussion of Study 2

• Specific-positive produced mostly control judgments, while both general conditions produced mostly judgments of little control.Participants judged control based on being able to choose a specific action, rather than on whether the outcome was positive or negative.Source: §5.4.2 · p. 21, §5.5 · p. 23
• Participants felt agency over voice commands but not character movement; specificity affected explicit judgments, not pre-reflective interval estimates.The judgment of control was strongest when participants could choose a specific, meaningful action.Source: §5.5 · pp. 22–23

6.1. Theoretical Implications

• The absence of pre-reflective agency over computer-produced outcomes is consistent with human-human interaction findings and the Media Equation account of human-like responses to computers.The authors note that the transfer of human qualities to computers is not complete.Source: §6.1 · pp. 23–24

6.2. Design Implications

• Study 2 found no pre-reflective agency over final movement, but participants judged themselves to have caused it when the task was meaningful and they could specify the direction.Source: §6.2 · p. 24
• For systems where people should feel control and take responsibility for outcomes, the authors recommend encouraging reflection and deliberative judgments rather than relying on instinctive agency feelings.They suggest design friction may help prompt consideration of personal responsibility when instructing computers.Source: §6.2 · pp. 24–25

7. Limitations

• Both studies were online; the authors cite prior findings of no discernible data-quality difference from in-person research, including evidence on online intentional-binding studies.This prior evidence gave the authors confidence in their methods.Source: §7 · p. 25
• Study 1 followed a prior psychology protocol and was intended to provide robust foundations, but its HCI limitations were addressed in Study 2.The authors cite the absence of pre-reflective agency over final outcomes in both experiments as support for their results.Source: §7 · p. 25
• The authors cannot establish that findings from simple actions and outcomes apply to higher-consequence interactions such as remote surgery or autonomous command and control.They call for future studies involving instructions with greater consequences.Source: §7 · p. 25

8. Conclusion

• Participants felt agency over the computer’s action, but not the final movement; command specificity and outcome did not alter this pre-reflective pattern.The authors report that this replicated human-human findings in an HCI context.Source: §8 · p. 26
• Participants judged themselves to cause the movement when they could control the outcome, but not when the computer chose it.The authors argue this difference shows that pre-reflective feelings and post-reflective judgments reflect less-sensitive-to-different-processes measures.Source: §8 · p. 26
• The authors recommend designing systems that enable people to reflect on their actions and judge their responsibility.They call for future research on factors affecting each type of agency and which measure matters in each context.Source: §8 · p. 26