PaperlayerView exact revision on arXiv

Source-linked AI summary

Long-Term Personalization of an In-Home Socially Assistive Robot for Children With Autism Spectrum Disorders

Caitlyn Clabaugh, Kartik Mahajan, Shomik Jain, Roxanna Pakkar, David Becerra, Zhonghao Shi, Eric Deng, Rhianna Lee, Gisele Ragusa, Maja Matarić

arXiv:1911.07992v1cs.RO

TL;DR

Long-term socially assistive robot interventions need to adapt to children’s changing and individual learning needs, but autonomous personalization remains a challenge. This paper evaluates a hierarchical reinforcement-learning framework in month-long home interventions with children with ASD. The system personalized instruction and feedback, and all participants showed cognitive gains.

  • Problem

    Long-term SAR interventions need to adapt to children’s individual and changing needs, but autonomous personalization presents computational and engineering challenges.

    Source: §1 · p. 2

  • Method

    The hHRL framework uses five controllers and a reinforcement-learning meta-controller to personalize instruction challenge and robot feedback in autonomous in-home interventions.

    Source: §1 · p. 3

  • Results

    All participants improved in math skills, and the system personalized instruction and feedback over time; users were engaged for most of the intervention.

    Source: §1 · p. 3, §6 · p. 22, §7 · p. 23

  • Takeaways & Limitations

    The findings support the feasibility of in-home personalized SAR that engages and adapts to children with diverse needs over multiple weeks.

    Source: §7 · p. 23

    Sufficient video and audio for objective engagement analysis were available for only seven of the 17 participants.

    Source: §4.3.1 · p. 15

Abstract

from arXiv · show

Socially assistive robots (SAR) have shown great potential to augment the social and educational development of children with autism spectrum disorders (ASD). As SAR continues to substantiate itself as an effective enhancement to human intervention, researchers have sought to study its longitudinal impacts in real-world environments, including the home. Computational personalization stands out as a central computational challenge as it is necessary to enable SAR systems to adapt to each child's unique and changing needs. Toward that end, we formalized personalization as a hierarchical human robot learning framework (hHRL) consisting of five controllers (disclosure, promise, instruction, feedback, and inquiry) mediated by a meta-controller that utilized reinforcement learning to personalize instruction challenge levels and robot feedback based on each user's unique learning patterns. We instantiated and evaluated the approach in a study with 17 children with ASD, aged 3 to 7 years old, over month-long interventions in their homes. Our findings demonstrate that the fully autonomous SAR system was able to personalize its instruction and feedback over time to each child's proficiency. As a result, every child participant showed improvements in targeted skills and long-term retention of intervention content. Moreover, all child users were engaged for a majority of the intervention, and their families reported the SAR system to be useful and adaptable. In summary, our results show that autonomous, personalized SAR interventions are both feasible and effective in providing long-term in-home developmental support for children with diverse learning needs.

1 INTRODUCTION

Children’s varied and changing developmental trajectories make long-term personalization important, but autonomous adaptation in real-world SAR settings presents computational and engineering challenges. This work introduces an autonomous, reinforcement-learning-based framework and evaluates it in month-long in-home interventions with children with ASD.

1 INTRODUCTION

  • Children’s nonlinear, individual development means interventions need to adapt over time to each user’s needs and preferences, making autonomous long-term personalization a computational and engineering challenge.The passage notes that strategies must adapt in situ using untrained human feedback.Source: §1 · p. 2
  • The intervention addresses social and cognitive learning together, reflecting the role of social factors in young children’s learning.The authors frame embedding social contexts in learning as developmentally appropriate for children with ASD.Source: §1 · p. 2
  • Personalization is especially important in ASD, but human-in-the-loop and Wizard of Oz approaches are infeasible at scale amid diverse needs and noisy real-world settings.The text identifies reinforcement learning as a potential route to adapting SAR to users’ learning habits over time.Source: §1 · pp. 2–3
  • The hHRL framework uses five controllers and a reinforcement-learning meta-controller to personalize instruction challenge and robot feedback to each child’s learning patterns.It was evaluated in fully autonomous, session-based in-home interventions with 17 children with ASD aged 3 to 7 years.Source: §1 · p. 3
  • All participants improved in math skills and retained intervention content, while users were engaged for a majority of the intervention and most families found the system useful and adaptable.The study used space-themed mathematics problems in month-long in-home deployments.Source: §1 · p. 3

2 BACKGROUND

The background situates SAR for learning in embodied, situated, and engagement-oriented approaches, with particular relevance and challenges in ASD. It then contrasts rule-, model-, and goal-based personalization, positioning this work as a long-term reinforcement-learning approach to mathematics instruction and feedback.

2.1 SAR for Learning

  • SAR for learning draws on embodied cognition, situated learning, and user engagement, while real-world settings add environmental noise and unpredictability.SAR studies in real-world learning settings require robust platforms and experimental designs that account for spatial and social context.Source: §2.1 · p. 4
  • Learning support in ASD faces amplified SAR challenges, and individual variation in symptoms and severity underscores the need for personalization.The background also notes that time and financial costs can make personalized therapeutic and learning interventions inaccessible to many.Source: §2.1 · pp. 4–5

2.2 Personalization in SAR

  • Rule-based SAR personalization has used performance thresholds to adjust challenge, whereas this work uses a goal-based approach for challenge and feedback.Prior month-long work with children with ASD adapted activity challenge using past performance and fixed thresholds.Source: §2.2 · pp. 5–6
  • Model-based personalization such as Bayesian Knowledge Tracing estimates learner knowledge from response sequences and accounts for slips and guesses.The cited background describes BKT as a common SAR approach and notes its application to adapting to learner age and experience.Source: §2.2 · p. 5
  • Goal-based reinforcement learning selects actions according to progress rewards and develops personalized strategies, while noisy real-world use has required substantial data.Prior work includes RL for engagement and hierarchical RL for feedback in memory-based SAR interactions.Source: §2.2 · pp. 5–6
  • The paper positions its contribution as long-term SAR for abstract concept learning, using goal-based reinforcement learning to personalize mathematics instruction and feedback for children with ASD.The framework is presented as complementary to prior work on ASD interventions focused on social skills and engagement.Source: §2.2 · p. 6

3 FORMALIZING PERSONALIZATION IN SAR

The paper formalizes SAR personalization as hierarchical human-robot learning (hHRL), decomposing interventions into controllers and using reinforcement learning to personalize challenge and feedback. The framework is designed for long-term adaptation to each child’s needs.

3 FORMALIZING PERSONALIZATION IN SAR

  • Computational personalization in SAR had not yet been formalized, motivating hHRL as a controller-based approach to long-term personalization.The paper frames this as a way to address the challenge of adapting SAR in a principled way.Source: §3.1 · p. 6, §3 · p. 6

3.1 Human-Robot Learning

  • Human-Robot Learning (HRL) is the interactive, co-adaptive process of personalizing SAR, framed here around optimizing the robot’s actions toward intervention goals.Long-term success is assessed through intermediate progress measures; optimization is limited to robot actions, not human actions.Source: §3.1 · p. 6

3.2 Hierarchical HRL

  • The hHRL framework decomposes SAR interventions into computationally tractable state-action subspaces organized in two levels.A meta-controller uses high-level intervention state to activate a lower-level controller, which chooses a robot action from a simplified state representation, reward, and action category.Source: §3.2 · pp. 6–7, Figure 1 · p. 7
  • SAR acts are modeled as illocutions regardless of whether they are conveyed through speech or other communication modalities.The framework treats the robot as speaker and the user as hearer, with action-to-state or state-to-action direction of fit.Source: §3.2 · p. 7

3.3 Abstract Controllers

  • Five abstract controllers handle instructions, feedback, promises, disclosures, and inquiries.Instructions seek user actions toward the intervention goal; feedback expresses beliefs about past or current interactions; promises commit the robot to future actions; disclosures express beliefs about the robot’s past or current self; inquiries seek truths from the user.Source: §3.3 · p. 8

3.4 Computational Personalization

  • The domain-specific meta-controller activates one controller at a time, with prior study data informing the personalization design.The intervention included 10 games per session, with computational methods setting challenge and feedback levels based on a model developed from 31 typically developing preschool children’s single-session data.Source: §3.4 · pp. 8–9, Figure 2 · p. 9

3.4.1 Personalization of the Level of Challenge

  • For each game, Q-learning selects a personalized challenge level from five levels, seeking to maximize challenge without exceeding the mistake threshold.The challenge controller uses game and challenge level as state and action spaces, with 10 × 5 = 50 state-action pairs; when mistakes exceed five, the reward is the inverse of the challenge level.Source: §3.4.1 · pp. 9–10

3.4.2 Personalization of the Level of Feedback

  • The feedback controller uses Q-learning to choose among four personalized feedback levels, minimizing feedback level while avoiding excessive mistakes and penalizing help requests too heavily.The fifth feedback level is reserved for when the child exceeds the five-mistake threshold; the controller has 10 × 4 = 40 state-action pairs.Source: §3.4.2 · p. 10

4 PERSONALIZED SAR INTERVENTION DESIGN

The SAR personalization framework was instantiated for a month-long in-home intervention with children with ASD.

4 PERSONALIZED SAR INTERVENTION DESIGN

  • The SAR personalization framework was instantiated in a system designed for evaluation in children’s homes.Source: §4 · p. 11
  • The system was evaluated in a month-long in-home intervention with children with ASD.Source: §4 · p. 11
  • The study was approved under USC IRB UP-16-00755.Source: §4 · p. 11

4.1 System Design

The intervention paired a portable, child-friendly robot with developmentally tailored math games and feedback. Kiwi acted as a peer companion, while game difficulty and feedback were personalized to each child.

4.1 System Design

  • Kiwi was designed as a companion rather than a tutor, presenting tablet games and personalized feedback through a neutral, nonthreatening character.Its gender-neutral appearance let each child assign it a gender if desired.Source: §4.1 · p. 11, §4.1.1 · p. 11

4.1.1 Physical Design

  • The self-contained setup combined SPRITE with a container for its computer, power supply, speakers, and tablet, minimizing the system footprint for in-home use.SPRITE used a six-degree-of-freedom platform and an animated face controlled through CoR-Dial.Source: §4.1.1 · p. 11

4.1.2 Game Design

  • The game design was iteratively refined with educator feedback and preschool piloting, then deployed and validated in long-term home interventions with 17 children with ASD.Games used scaffolded difficulty aligned with developmental practices and math standards; numerical operations and math reasoning were also selected for pre-post assessment.Source: §4.1.2 · pp. 11–12

4.1.3 Child-Robot Interaction

  • Children helped Kiwi, a space-explorer peer, through 10 game types spanning five challenge levels and preschool or kindergarten math skills.Tasks included packing moon-rocks, comparing stars, ordering numbered pets, and completing patterns; game challenge levels were personalized to each child.Source: §4.1.3 · pp. 12–13, §4.1.2 · p. 12
  • After mistakes, Kiwi paired verbal hints with child-like movements that signaled whether the child was struggling or excelling.For a task requiring five crystals, feedback could restate the goal, prompt counting, or explain how many more were needed.Source: §4.1.3 · p. 13

4.2 Study Participants

The study enrolled 17 children with ASD, ages 3 years 4 months to 7 years 8 months, recruited through regional services and school districts. The sample and available assessments had several documented boundaries.

4.2 Study Participants

  • Eligibility required ages 3–8, stable physical, sensory, and medical health, English as the family’s primary language, and mild-to-moderate ASD.Families were recruited through regional centers and local school districts.Source: §4.2 · p. 14
  • The 17 participants were ages 3 years 4 months to 7 years 8 months; two were female, 15 male, and the sample included three sibling pairs.All had mild-to-moderate ASD diagnoses, established through clinical best estimate using multiple measures that varied by clinician and agency.Source: §4.2 · pp. 14–15
  • Personalization data were unavailable for P1 and P2 because the system was not yet fully developed, and P3 did not complete post-study assessments.The other participants completed the study; P3 participated for over a month and provided all other study data.Source: §4.2 · p. 14
  • The study had no control condition; it used pre/post comparisons as a baseline, consistent with the stated single-case study rationale for ASD research.Source: §4.2 · p. 14
  • Participants also received full-time educational and therapeutic services, and all had intelligence scores above 70 on Leiter-3 or Differential Ability Scales measures.Source: §4.2 · p. 14

4.3 Procedure

Families hosted the autonomous robot for at least 30 days, with deployment ending after 20 interactions. The study collected game, video, audio, assessment, and family-interview measures, though analyzable engagement recordings were limited.

4.3 Procedure

  • Deployments lasted at least 30 days and continued until 20 child-robot interactions were completed, averaging 41 days (SD 5.92); families could operate the autonomous system when they chose.Children were encouraged, but not required, to complete five sessions per week or play all 10 games in a session.Source: §4.3 · p. 15

4.3.1 Objective Measures

  • The study recorded game performance and interactions, while pre- and post-intervention WIAT II assessments measured achievement gains, focusing on numerical operations and math reasoning.For young children, those subtests cover early calculations and number discrimination, and quantity, order, word problems, and patterning, respectively.Source: §4.3.1 · pp. 15–16
  • Engagement was coded from video and audio, but technological challenges left sufficient data for only seven participants; annotator agreement was Fleiss’ kappa = 0.84.Source: §4.3.1 · p. 15

4.3.2 Subjective Measures

  • Families completed biweekly interviews rated on a 7-point Likert scale, with each child serving as their own baseline for usefulness and child-robot relationship questions.Questions covered whether Kiwi helped with tasks, how much parental involvement was needed, and whether the child viewed Kiwi as a friend or felt listened to and known.Source: §4.3.2 · p. 16

5 RESULTS

The results assess personalized challenge and feedback, family evaluations, engagement, and pre/post math learning across the in-home intervention. Children showed significant math gains, while engagement varied across children and challenge levels.

5.1 Personalized Level of Challenge

  • pLoC varied across children and personalized over time, but began converging only after more than 100 episodes (about 132 at the study’s average pace).The pLoC curve could not be interpreted as simply maximizing reward because reward reflected both challenge level and child performance.Source: §5.1 · pp. 16–17, Figure 5 · p. 17

5.2 Personalized Level of Feedback

  • pLoF adapted faster than pLoC, with curves varying by mistakes and intervention length; P10’s curve was longest, while P5 and P6 had higher rewards and shorter tails.The authors report that the pLoF model adapted to each child over time.Source: §5.2 · pp. 17–18

5.3 Participant SAR Evaluation

  • Survey responses reflected differences in perceived adaptability and usefulness: P1 and P17 were low usefulness outliers, and sibling ratings differed in ways the authors linked to age and fit.P17 also had the lowest average adaptability score; the authors suggest the maximum game difficulty was too easy for this high-performing participant.Source: §5.3 · pp. 18–19

5.4.1 Short-Term and Long-Term Engagement

  • Analyzed children averaged 65% engagement; continuous engagement lasted a median 13 seconds versus 5 seconds for disengagement, and engagement was higher shortly after robot speech.Engagement was about 70% in the minute after the robot spoke, versus under 50% when it had not spoken for over a minute.Source: §5.4.1 · p. 19

5.4.2 Engagement and the Level of Challenge

  • Participants accounted for 91% of engagement variance; higher optimal LoC correlated with greater engagement (rs = 0.84, p = 0.018), although pLoC did not necessarily maximize engagement.The authors hypothesize that slow convergence left some children with lower optimal LoC facing higher-challenge games before adaptation.Source: §5.4.2 · p. 20, Figure 9 · p. 20

5.5 Impact on Math Learning

  • p = 0.002 for numerical operations and p < 0.001 for math reasoning; scores increased significantly for all participants except P3, who did not complete post-study assessment.Effect sizes were d = 0.53 for numerical operations and d = 0.54 for math reasoning.Source: Figure 10 · p. 21, §5.5 · p. 21

6 DISCUSSION

The discussion reports math gains for every participant, alongside personalization that generally stabilized after substantial interaction. Survey and engagement findings suggest the system was useful and adaptable, while limited interaction constrained adaptation for some children.

6 DISCUSSION

  • After approximately 100 games, pLoC and pLoF stabilized for almost all participants, indicating adaptation to an appropriate challenge level for each student.The authors identify the study’s long-term nature as important for successful personalization.Source: §6 · pp. 21–22
  • P16’s pLoC and pLoF did not standardize with only 129 total interactions, while participants with sufficient interaction more often achieved consistent outcomes across both models.P16’s pLoC reward fluctuated by 2.6 points from episodes 61 to 129; the authors compare this with a 1.02-point fluctuation for P16, whose system ultimately converged after 592 games.Source: §6 · p. 22
  • Participants generally viewed the system as adaptive: four participants whose models adapted rated adaptiveness 6.25 on average, while P17 was the sole participant who disagreed.P17 had 112 feedback episodes over 298 games; usefulness questionnaire scores were similar across participants and were reported as related to adaptiveness.Source: §6 · p. 22
  • Engagement was lower for participants whose optimal challenge level was lower, and the authors suggest slow adaptation may have discouraged P16 from continuing.P16 had the second-lowest engagement and below-average episodes; the authors frame the explanation as likely rather than definitive.Source: §6 · p. 22
  • All participants improved in math, averaging 7.0 points on numerical operation and 7.125 points on math reasoning.P16, despite limited personalization, gained 15 points in numerical operation and 12 in math reasoning; P8, with the fewest episodes, had below-average gains in both assessments.Source: §6 · p. 22

7 CONCLUSION

The conclusion presents hHRL as a framework for computational personalization in long-term, real-world SAR interventions. In month-long in-home deployments, the system adapted to children with ASD and they showed cognitive gains.

7 CONCLUSION

  • The work addresses computational personalization for long-term, real-world SAR interventions, aiming to adapt robot interactions to individual users’ needs and preferences.Source: §7 · p. 23
  • The hHRL framework decomposes SAR intervention into a hierarchical decision-making problem and uses reinforcement learning to personalize challenge levels and feedback.The approach was instantiated in interactive games and tested in month-long in-home deployments with children with ASD.Source: §7 · p. 23
  • The study reports that hHRL engaged and adapted to children with diverse needs in math learning over multiple weeks, with participants demonstrating cognitive gains.The authors conclude that these findings highlight the potential of in-home personalized SAR interventions.Source: §7 · p. 23

CONFLICT OF INTEREST STATEMENT

The authors disclose their affiliations with Embodied, Inc.

CONFLICT OF INTEREST STATEMENT

  • Matarić co-founded Embodied, Inc. but is no longer involved; Clabaugh is a full-time employee and was not involved with the company during the reported work.Source: CONFLICT OF INTEREST STATEMENT · p. 23

DATA AVAILABILITY STATEMENT

The study dataset is not distributed because it contains identifiable information about children, families, and their homes.

DATA AVAILABILITY STATEMENT

  • The University IRB prohibits distribution of the dataset, which contains identifiable child and family video and audio, as well as images and video information representing their homes.Source: DATA AVAILABILITY STATEMENT · p. 24

Related papers

Loading 1911.07992v1…