PaperlayerView exact revision on arXiv

Source-linked AI summary

AdapTest: Supporting Designers in Authoring and Evaluating Environmentally Responsive User Interfaces for Extended Reality

Jungwoo Rhee, Guhn Lee, Yumin Jung, Mustafa Doga Dogan, Justin Moon, Andrea Bianchi

arXiv:2610.21386v1cs.HC

TL;DR

XR interfaces face changing distance, lighting, and occlusion, while existing workflows separate static variants from the conditions that should trigger them. AdapTest lets designers author and compare condition-specific rules and test them in place; participants covered more environmental factors and repeatedly tested and revised their designs, with workload and usability comparable to the baseline.

  • Problem

    XR interfaces can become difficult to use as distance, lighting, and occlusion change, while current workflows do not connect designer-made variants to testable environmental triggers.

    Source: §1 · p. 2

  • Method

    AdapTest lets designers define element-level IF/THEN changes for distance, lighting, and occlusion, compare previews, and test rules in place using live sensing or manual controls.

    Source: §8 · p. 23

  • Results

    All twelve participants authored working conditional rules and variants covering multiple environmental factors, repeatedly tested them, and reported workload and usability comparable to the baseline.

    Source: §1 · p. 3, §7 · p. 20

  • Takeaways & Limitations

    AdapTest integrates designer-controlled adaptation, comparison, and situated testing while accommodating testing woven through authoring or concentrated in later verification.

    Source: §8 · p. 23

    The tool comparison paired AdapTest with a home task and the baseline with an office task, so it was not a pure tool contrast; the study captured first-contact use in one study room.

    Source: §7.4 · p. 23

Abstract

from arXiv · show

Extended Reality (XR) headsets increasingly host familiar 2D user interfaces as windows floating in physical environments. However, their usability depends on environmental conditions that designers cannot fully control. Current design workflows often separate how a variant is designed from how it actually looks in the target environment. We present AdapTest, an in-situ authoring and testing tool that lets designers select interface elements, define when and how they change, compare the resulting variations, and test their activation without coding. Based on prior research and a formative study with 12 interface designers, the system targets three key environmental variables: distance, lighting, and occlusion. We evaluated AdapTest against a desktop design and in-headset preview workflow in a within-subjects study with 12 designers. Compared with the baseline, designers accounted for a wider range of environmental factors, addressed more overlapping conditions, and repeatedly tested and revised their designs in place; there were no significant differences in workload or usability despite longer sessions. Our work contributes an integrated, designer-centered approach to adapting, comparing, and validating environmentally responsive XR user interfaces.

1 Introduction

XR interfaces must remain usable as distance, lighting, and occlusion change, but current workflows separate static design variants from the conditions that trigger them. AdapTest brings rule authoring, variant comparison, and in-place testing into one designer-controlled workflow.

1 Introduction

  • AdapTest addresses workflows that either leave adaptation to automated systems or represent designer-made cases as static copies without condition-to-change mappings that can be tested in place.Confirming behavior in the room otherwise requires coding, rebuilding, and redeploying.Source: §1 · p. 2
  • Designers select interface elements, assign alternative styles for distance, darkness, or occlusion, preview each rule beside the default, then test and refine rules against room conditions.The workflow is designed to require no coding.Source: §1 · p. 2
  • All twelve AdapTest participants authored conditional rules, created variants covering multiple environmental factors, and repeatedly used Test Mode; workload and usability remained comparable to the baseline.Participants also reported that AdapTest made adaptive behavior a working rule that could be experienced and reused across screens.Source: §1 · p. 3
  • The paper contributes a characterization of environmental rule authoring, an integrated authoring and testing tool, and a study comparing it with a Figma-based workflow.Source: §1 · p. 3

2 Related Works

Prior XR work has explored runtime adaptation, rule-based authoring, design-variant comparison, and in-situ testing, but these practices leave gaps for designers authoring condition-responsive GUI elements. AdapTest makes environmental mappings explicit and editable, previews condition-specific variants, and tests the active rule in place.

2.1 Authoring Adaptive Behavior for XR Interfaces

  • Runtime XR systems adapt layouts, placement, or visual properties, but their condition–appearance mappings often reside in code, objectives, or optimization, limiting visual designers’ ability to inspect and adjust them without programming.Source: §2.1 · p. 3
  • Earlier rule-based XR tools support interaction-event triggers, while FrameKit maps viewer distance to 2D interface keyframes; rules attaching GUI elements to sensed XR environmental conditions remain an open opportunity.AdapTest lets designers select granular elements and define explicit, deterministic, editable condition–variant mappings without code.Source: §2.1 · p. 4

2.2 Comparing Design Variants

  • Existing design tools compare freely explored alternatives, whereas XR adaptation asks whether a design works across conditions; AdapTest previews environmental variants around the default, including combined conditions.Current duplicated artboards leave the condition-to-variant mapping in file names and memory.Source: §2.2 · p. 4

2.3 Testing XR Designs in Place

  • In-situ testing tools are limited to conditions available in the room, while AdapTest lets designers dial conditions or use live sensing to activate the currently matched rule.This reframes testing from re-enacting circumstances to querying the rule set.Source: §2.3 · p. 4

3 Formative Study

A formative study with twelve designers examined how screen-rooted designers adapt multi-screen interfaces in a physical space. It identified element-level strategies, difficulty managing condition coverage, and the need to judge thresholds and transitions in place.

3 Formative Study

  • The formative study observed twelve designers adapting multi-screen interfaces across three viewing distances, yielding findings about element-level strategies, compounding conditions, and in-place judgment.These findings informed design goals DG1–DG3.Source: §3 · p. 5

3.1 Method

  • The study staged three viewing distances in a maker’s lounge and had participants place screens, design from multiple zones, review them in XR, and iterate.The zones averaged 2.8m within-zone, approximately 11m adjacent-zone, and approximately 22m non-adjacent-zone; twelve designers completed the study, most without prior XR-interface authoring experience.Source: §3.1 · pp. 5–6

3.2 Finding 1: Adapting UIs on Element Level

  • Designers adapted individual elements through resizing, reducing block count, recoloring, repositioning, and occasionally rearranging; adapted states otherwise remained static copies with triggers held in memory.Scale adaptation appeared on 56% of screens, block-count control on 62%, and layout rearrangement on 10%.Source: §3.2 · pp. 6–7

3.3 Finding 2: Deciding Which Versions to Focus

  • As versions multiplied, designers struggled to prioritize conditions and apply consistent treatments, while their materials did not show which conditions were covered or missed.DG2 proposes previews beside the default to reveal coverage gaps and mismatches between sibling versions.Source: §3.3 · p. 7

3.4 Finding 3: Judging Transitions in Place

  • Designers’ choices about when an adaptation should fire and how it should change were difficult to settle away from the space, and participants worried about abrupt or continuous transitions.Some preferred discrete zone-based switching; physical movement also exposed occlusion problems.Source: §3.4 · pp. 7–8
  • Participants could create adapted states on paper, but validating trigger points and transitions required standing where the interface would be used.Distance and occlusion emerged from the study; lighting was included based on prior findings about legibility in dim and bright environments.Source: §3.4 · p. 8

4 AdapTest

AdapTest brings rule authoring, variant comparison, and in-environment testing into one workflow for adaptive XR interfaces. Designers specify element changes and environmental conditions, inspect overlaps and gaps, then test and refine behavior in the target space.

4 AdapTest

  • AdapTest unifies authoring adaptive interface rules, comparing condition-specific variants, and testing which variant activates in the intended physical environment.The resulting design pairs interface variants with rules defining when each adaptation occurs.Source: §4 · p. 8

4.1 DG1: Element-Targeted Rule Authoring

  • Designers target selected elements or whole screens with IF–THEN rules, choosing environmental measures, thresholds, and visual outputs such as resizing, hiding, or changing color.Rules can combine conditions, carry multiple outputs, and use priority ordering when several rules match.Source: §4.1 · p. 9, Figure 7 · p. 9

4.2 DG2: Multi-Condition Preview Comparison

  • Side-by-side previews, combination and conflict suggestions, and coverage strips make overlapping, conflicting, and uncovered conditions visible during authoring.Designers can accept or dismiss combination suggestions, resolve conflicting outputs, and add rules for measures with no coverage.Source: §4.2 · p. 10

4.3 DG3: Situated Testing and Refinement

  • Test Mode evaluates rules in the physical setting, showing active variants and environmental values; designers can manually set unreachable values, revise rules, and retest.A new match must hold briefly before activation to avoid flicker near a threshold.Source: §4.3 · pp. 10–11, Figure 8 · p. 10

4.4 System Implementation

  • The React web editor and native visionOS app share a project model and render the same HTML/CSS, with rule evaluation in both authoring previews and headset testing.End-to-end latency measurements used the first 100 valid environmental-change/UI-response pairs; brightness response was slower and more variable because lighting changes more slowly and its estimate settles gradually.Source: §4.4 · pp. 11–12, Table 1 · p. 12

5 User Study

The study compared designers’ adaptive-interface work with AdapTest and a Figma-based current-practice workflow in a within-subjects design. It examined rule specification, environmental-condition coverage, and in-place testing, using matched but distinct design tasks and logged behavior, surveys, and interviews.

5 User Study

  • A within-subjects comparison examined how designers specified adaptive behavior, covered environmental conditions, and tested adaptations with AdapTest versus a Figma-based baseline.The study used twelve designers and focused on three research questions concerning specification, condition coverage, and test-informed refinement.Source: §5 · p. 12

5.1 Participants

  • The twelve participants had UI design practice and HMD or AR/VR experience; all used Figma as a primary tool and reported high Figma fluency.Seven participants also reported XR UI design experience through coursework or research projects.Source: §5.1 · p. 12, Table 2 · p. 13

5.2 Study Procedure

  • Participants completed two counterbalanced tasks, received post-task surveys and artifact walkthroughs, and concluded with a semi-structured interview.The baseline needed no training, while AdapTest participants received a ten-minute walkthrough before the task.Source: §5.2 · p. 13

5.3 Conditions and Tasks

  • Matched but distinct three-screen tasks compared AdapTest with desktop Figma editing and in-headset static-image previews; the baseline could not express or execute environmental rules.The baseline workflow was made more convenient than the current practice it represented, and the task conditions had different time budgets.Source: §5.3 · pp. 13–14

5.4 Data Collection and Analysis

  • Researchers logged rule targets, factor coverage, test–refine cycles, and related actions, while surveys measured workload and usability and interviews supplied qualitative data.Baseline variants and intended triggers were coded from artifacts and walkthroughs; reasoning-sensitive measures were reported descriptively for first-block groups of six per condition.Source: §5.4 · pp. 14–15, Table 3 · p. 15

6 Results and Findings

In the 12-designer comparison, AdapTest supported explicit rule-based variants and in-place testing, while participants addressed more environmental factors and combined conditions. Workload and usability were comparable, and workflow preferences varied with the work.

6 Results and Findings

  • Across the different-budget tasks, AdapTest produced 55 working rules versus 38 static baseline variants; authoring rate, workload, and usability showed no reliable differences.The rate comparison was not reliable; the volume difference reflected longer authoring at a comparable rate.Source: Figure 10 · p. 15, §6 · p. 15

6.1 Writing Adaptive Behavior on Specific Elements

  • AdapTest attached IF/THEN rules to selected elements, while baseline trigger logic stayed in annotations or speech; rule communication ratings were 6.2 versus 4.4 of 7.Of AdapTest’s 55 rules, 44 targeted specific elements and 11 applied to a whole screen.Source: §6.1 · p. 16

6.2 Noticing and Covering Overlapping Conditions

  • Participants covered more staged factors with AdapTest than baseline (2.4 versus 1.4 on average), although the comparison differed in task domain and time budget.The within-subjects difference was reliable; six of twelve AdapTest participants covered all three factors, versus three of twelve baseline participants.Source: §6.2 · p. 16
  • Combined rules were authored by 9/12 AdapTest participants versus 2/12 baseline; five accepted combination suggestions supplied five of AdapTest’s twelve such rules.Eleven of the twelve combined rules remained in final projects.Source: §6.2 · p. 17

6.3 Testing Rules in the Real Environment

  • AdapTest participants ran Test Mode 14.3 times per session on average versus 2.7 baseline headset inspections, and 35% of style and condition edits were test-prompted.These are different activities, so their frequency comparison is descriptive; testing occupied 26% of active session time on average.Source: §6.3 · p. 18
  • Test Mode helped participants judge thresholds and styling through observation, but the physical room limited the distances and extreme conditions they could test.Participants described calibrating distance thresholds and revising styling during tests; some wanted virtual controls, while one preferred physical occlusion testing.Source: §6.3 · pp. 19–20

6.4 Preferences Between Workflows

  • Preferences split by task: immediacy and embodied judgment favored AdapTest, while learning cost and precise content edits favored the desktop workflow.Participants rated AdapTest higher on needing to learn a lot before getting started (3.0 vs. 2.1 of 5).Source: §6.4 · p. 20

7 Discussion

The discussion finds that AdapTest helped designers express adaptive intent as executable rules, inspect environmental coverage, and incorporate in-situ testing into authoring. It also identifies workflow trade-offs and limits in the study comparison and environmental sensing.

7.1 Authoring Adaptivity as Designer-Owned Rules

  • All twelve participants authored adaptive intent as working rules, and rated their ability to communicate change conditions higher with rules (6.2 vs. 4.4 of 7).Designers used concrete elements and continuous environmental triggers to author the condition-to-appearance mapping as part of design itself.Source: §7.1 · pp. 20–21
  • Six of twelve baseline participants authored triggers beyond the staged environmental factors, including time, touch, gaze, hand-reach, and user movement.Participants also imagined interfaces whose roles shift with placement and people nearby, such as a glanceable clock or a video-doorphone.Source: §7.1 · p. 21

7.2 Seeing Coverage Before Deployment

  • Designers covered more staged factors with AdapTest (2.4 vs. 1.4 of 3), nine of twelve authored combined rules, and previews were rated 6.9 of 7.Coverage remained incomplete: brightness and occlusion trailed distance, and previews did not ensure designers addressed every gap.Source: §7.2 · p. 21
  • Accepted combination suggestions seeded five of twelve combined rules, while one participant covered all three factors with self-authored rules after accepting none.Suggestions may serve as prompts for designers to author responses, not only as rules they adopt.Source: §7.2 · pp. 21–22

7.3 Testing as Part of Authoring

  • Testing occupied about a quarter of active session time, and 35% of edits followed a test within two minutes.Seven participants concentrated testing in later verification passes, while five interleaved it from the start; sessions ran longer without a reliable difference in authoring rate.Source: §7.3 · p. 22
  • AdapTest combines the desktop design source with in-place rule testing and visible coverage, while participants reported fine content editing and asset reuse as easier on desktop.The study did not evaluate a fully XR workflow, so this comparison rests on prior work and participants’ accounts.Source: §7.3 · p. 22

7.4 Limitations and Future Work

  • The study’s comparison is not a pure tool contrast because each tool was tested on its own fixed task domain.Its twelve single-session participants and room-bounded staged conditions support claims about authoring and testing processes and perceptions, not end-user usability or deployed adaptation.Source: §7.4 · p. 23
  • Brightness thresholds are relative to the session and camera exposure, while study occlusion sensing tracked hands only and was not instrument-validated.Thresholds are comparable within a session, not across rooms; continuous sensing in everyday spaces also raises privacy questions.Source: §7.4 · p. 23

8 Conclusion

AdapTest brings element-targeted rules, condition previews, and situated testing into a workflow for adapting existing 2D designs to XR environments. In a comparison with a familiar desktop-and-preview workflow, designers used the capabilities to cover more staged conditions and refine designs, with comparable workload and usability.

8 Conclusion

  • AdapTest lets designers adapt existing 2D interfaces through element-targeted rules, condition previews, and situated testing of active rules.Designers specify changes over distance, lighting, and occlusion, then compare preview outcomes and validate behavior using manual controls or live sensing.Source: §8 · p. 23

A Formative Study: Demographics

The appendix identifies a demographics table for the formative-study participants and provides the interview protocol used to elicit designers’ decisions about spatial interface design and environmental adaptation.

A Formative Study: Demographics

  • Table 5 is identified as presenting demographic and background information for all formative-study participants.The supplied caption does not state the table’s values.Source: Table 5 · p. 28

B Formative Study: Interview

  • The interview asked how the environment between user and interface influenced design decisions, including size, position, content display, and component types.It also probed transition factors, design challenges, potential beneficiaries, and participants’ final thoughts on spatial interface design.Source: B Formative Study: Interview · pp. 28–29
  • The interview protocol asked why participants chose particular numbers and complexity of interfaces and whether spatial structures or contexts influenced those choices.Follow-up questions considered how larger, more complex, or differently structured spaces might change designs.Source: B Formative Study: Interview · p. 28
  • Participants were asked to explain interface size and composition at each viewing location and identify the most important design factor.The protocol also asked whether they would change the number of interfaces and what determined that number.Source: B Formative Study: Interview · p. 28
  • Questions covered whether an optimal window size exists in physical dimensions or relative to the field of view.Source: B Formative Study: Interview · p. 28
  • The protocol asked how components such as videos, images, text, and buttons vary across viewing distances and why those variations were chosen.It also asked whether some component types work particularly well at specific distances.Source: B Formative Study: Interview · p. 28
  • Participants were asked why interfaces aimed at the same zone differed or resembled one another and how different viewing conditions might change them.Source: B Formative Study: Interview · p. 28

C User Study: Interview Protocol

The appendix introduces the full set of questions used in the user study.

C User Study: Interview Protocol

Post-task walkthrough (baseline block)

The baseline walkthrough asked designers to explain what they created and which situation it targeted, then distinguish what the artifact made inspectable from what remained imagined.

Post-task walkthrough (baseline block)

Post-task walkthrough (AdapTest block)

The AdapTest walkthrough asked designers to explain their final designs and important rules, and to reflect on revisions, combined-rule suggestions, and situations left unaddressed.

Post-task walkthrough (AdapTest block)

Semi-structured interview

The interview compared designers’ experiences with Figma and AdapTest, including rule authoring, expression limits, possible project fit, and desired changes before real use.

Semi-structured interview

  • Designers were asked what felt most different between working in Figma and AdapTest and what felt familiar or unfamiliar compared with their usual practice.Source: Semi-structured interview · p. 29
  • The interview probed which parts made specifying what changes and when easier or harder, and what designers found difficult to express in AdapTest.Source: Semi-structured interview · p. 29
  • Designers were asked what projects might suit a tool like AdapTest, when they would prefer not to use it, and what they would change before real use.Source: Semi-structured interview · p. 29
  • The interview also asked where the tool might fit in the design–development process.Source: Semi-structured interview · p. 29
Loading 2610.21386v1…