PaperlayerView exact revision on arXiv

Source-linked AI summary

UrbanGazeVis: A Visualization System for Analyzing Eye-Tracking Data on Urban Safety Perception

Andres De La Puente, Felipe Moreno, Luis Sante, Mauro Diaz, Jorge Poco

arXiv:2608.21686v1cs.HC

TL;DR

Existing image-level safety judgments provide limited insight into how attention to specific urban disorder cues shapes perceived safety. UrbanGazeVis links head-mounted eye tracking with semantic scene and disorder annotations to analyze these relationships, showing that local, context-specific patterns complement global analyses.

  • Problem

    The relationship between gaze behavior and specific urban disorder cues in perceived street safety remains poorly understood beyond image-level judgments.

  • Method

    UrbanGazeVis combines head-mounted eye tracking of 30 participants viewing 150 Rio de Janeiro scenes with semantic segmentation, disorder annotations, safety ratings, and interactive visual analytics.

  • Results

    Linking gaze, temporal attention, and urban elements reveals context-specific safety-perception patterns often hidden by global aggregation.

  • Takeaways & Limitations

    Integrating eye tracking, semantic scene understanding, and visual analytics supports more interpretable studies of urban perception and perception-aware urban design.

  • Takeaways & Limitations

    The study’s 30 participants came from a single institution, limiting representation of Rio de Janeiro’s socioeconomic diversity and generalizability.

Abstract

from arXiv · show

Perceived safety in streetscapes depends on where people look, yet how gaze relates to visual cues of urban disorder remains poorly understood. Prior work treats safety as an image-level label, offering little insight into how attention to specific elements (e.g, buildings, greenery, people, signs of decay) shapes these judgments. We present a head-mounted eye-tracking study in which 30 participants viewed and rated the safety of 150 street-view images from Rio de Janeiro using a HoloLens 2 headset. Gaze traces were mapped onto semantic segments and disorder cues (e.g., damaged walls, graffiti, overhead cables), yielding a multimodal dataset linking gaze dynamics, scene semantics, and safety scores. To analyze it, we introduce UrbanGazeVis, an interactive visual analytics system with image- and participant-centric views that connects the spatial, temporal, and semantic dimensions of gaze to perceived safety, supporting comparisons between safe and unsafe scenes, inspection of divergent ratings for similar images, and region-of-interest analysis via glyph-based summaries. Statistical models show that sustained attention to physical disorder is associated with lower perceived safety, while the visual analysis reveals context-specific effects often masked by global aggregation. Together, these analyses offer actionable insights for urban design and planning.

1. Introduction

UrbanGazeVis addresses the limited understanding of how visual attention relates to urban disorder and perceived street safety. The work combines head-mounted eye tracking, semantic and disorder annotations, and interactive visual analytics to connect gaze dynamics with safety judgments.

  • Introduction: Visible urban neglect can reduce perceived safety, yet the relationship between gaze behavior and disorder remains poorly understood.Examples include deteriorated façades and damaged infrastructure, with potential downstream effects on behavior, crime, and mental health.
  • Introduction: Static, aggregated fixation maps obscure the dynamic, temporal, and semantic dimensions of attention.This motivates moving beyond image-level safety scores toward spatiotemporal visual analytics.
  • Introduction: The study collects safety ratings for 150 Rio de Janeiro street-view scenes using a head-mounted optical see-through headset for controlled stimulus presentation and gaze capture.Gaze traces are projected onto semantic scene segments such as buildings, sidewalks, and trees, and annotated with disorder cues.
  • Introduction: The UrbanGaze dataset integrates street-view images, safety ratings, eye-tracking data, semantic segmentations, and participant metadata.The dataset is intended for urban perception research and is produced through calibration, trial design, filtering, and quality control.
  • Introduction: UrbanGazeVis interactively overlays dynamic gaze information on semantically segmented scenes to explore gaze paths and fixation patterns in relation to safety scores.The system supports domain-inspired case studies of how attention relates to perceived safety.

2. Related Work

Related work spans urban perception, environmental criminology, eye-tracking, immersive visualization, and visual analytics. UrbanGazeVis narrows this space by using a headset as an eye tracker and linking gaze to semantic scene content and fine-grained urban disorder in street-view imagery.

  • Urban Perception and Computer Vision: Urban perception research uses street-level imagery and crowdsourced judgments to study how built environments shape judgments such as safety.Examples include Place Pulse and StreetScore, which support questions about visual identity and perceived safety.
  • Environmental Criminology: Environmental criminology frames safety perception as a dynamic cognitive tension between physical infrastructure and social activity, complementing passive image-level computer-vision models.The paper invokes routine activity theory, crime pattern theory, and situational crime prevention as foundational perspectives.
  • Eye-Tracking for Urban Perception: Eye-tracking decomposes visual attention into fixations, saccades, and scanpaths, while head-mounted systems require calibration and geometric uncertainty modeling for reproducible datasets.Desktop and head-mounted systems trade off sampling rate, head-movement tolerance, calibration, and stimulus presentation.
  • Visual Analytics for Eye-Tracking Data: UrbanGazeVis uses the headset only as a self-contained eye tracker and projects gaze onto semantic segments and urban disorder cues in street-view imagery.Its visual analytics connect these encodings to time-resolved views and safety scores, enabling analysis of attention across safe and unsafe scenes.

3. System Overview

UrbanGazeVis is designed to support interactive analysis of relationships among gaze, scene semantics, and safety ratings across images and participants. Its overview-to-detail analytical tasks are implemented through complementary By-Image and By-Participant views.

  • Design rationale: UrbanGazeVis addresses the limited exploration of gaze–semantics–rating relationships by applying overview-first, zoom-and-filter, and details-on-demand principles.The design draws on established gaze-analysis task abstractions.
  • Design goals: Its design goals support image- and participant-centric analysis, characterize spatiotemporal gaze behavior, and connect gaze with semantic categories, disorder cues, and safety assessments.Consistent encodings support comparisons across individuals and scenes, while temporal analysis exposes dynamics hidden by static heatmaps and aggregates.
  • Analytical tasks: Spatial attention tasks summarize regions and semantic elements attracting attention for a selected image or across all images viewed by a selected participant.These tasks cover both image-level and participant-level perspectives.
  • Analytical tasks: Temporal tasks compare participants’ scanpaths within an image and track how one participant’s gaze evolves across a session to detect learning or fatigue effects.The tasks examine both within-image attention movement and across-session change.
  • Analytical tasks: UrbanGazeVis links fixations on semantic categories and disorder cues to image-level safety ratings, and compares participant focus, gaze coverage, and score distributions across similar images.These analyses inspect rating consistency and are supported by image-centric and participant-centric views.

4. UrbanGaze Dataset

UrbanGaze was constructed from 150 Rio de Janeiro street-view images, combining semantic and physical-disorder annotations with controlled eye-tracking safety ratings. Stratified ILP sampling and randomized block assignment supported balanced analysis while limiting participant fatigue.

  • Dataset curation: 150 Rio de Janeiro street-view images were curated from UrbanPD4k, with OneFormer segmentation used to obtain urban elements for fine-grained analysis.The dataset includes pixel-level annotations of physical disorder cues and defines ADE20K segmentation as a baseline.
  • Dataset curation: Multivariate stratified sampling formulated image selection as an Integer Linear Programming problem over baseline ADE20K segmentation layouts.The design balanced structural and social classes to reduce layout confounding and support stable regression models.
  • Experimental design: 30 participants evaluated randomized subsets of 50 images, with each of the 150 streetscapes rated by exactly 10 distinct participants.Participants were divided into three blocks of 10 raters, and each block received a unique subset.

5. Dataset Validation and Robustness

Dataset validation showed robust consensus despite sparse ratings and confirmed that image-level differences explain a substantial share of safety-score variation beyond participant-specific baselines.

  • Hardware robustness: Raw fixation points were modeled as attentional catchment areas to reduce coordinate-mapping uncertainty and segmentation-boundary noise.Gaussian smoothing used σ = 10.68 pixels, analytically derived rather than empirically tuned.
  • Rater reliability: A crossed-random-effects LMM partitioned 1,500 observations into scene signal, participant bias, and residual noise.This model-based approach was used to estimate reliability for the sparse, overlapping rating matrix.
  • Rater reliability: G(q, k) = 0.812 demonstrated high inter-rater reliability despite the sparse rating design.The analysis used k = 10 raters per image and q ≈0.67, with values above 0.80 indicating robust consensus.
  • Structural validity: ICC = 0.203 indicated that approximately one-fifth of safety-score variance was attributable to participant-specific baselines.The remainder was attributed to image-level differences and residual factors.

6. UrbanGazeVis

UrbanGazeVis is an interactive visual analytics system with coordinated By-Image and By-Participant modes for connecting spatiotemporal gaze behavior with semantic content and safety ratings. Its area-normalized Attention Intensity metric and linked views support comparisons across scenes, participants, and visual elements.

  • System overview: UrbanGazeVis provides By-Image and By-Participant modes that support image- and participant-centric analysis of gaze, semantics, and safety ratings.The modes address perspective switching, spatiotemporal gaze characterization, and relationships between gaze patterns, semantic content, and safety ratings.
  • Attention metrics: Area-normalized Attention Intensity discounts fixation time by semantic-class area, highlighting attention to small objects such as graffiti or cables.The metric mitigates the bias that large regions naturally receive more fixations because of their greater pixel area.
  • Attention metrics: τ = 0.005 (0.5%) excludes regions below the area threshold, treating them as indistinguishable from eye-tracker hardware noise.The lower bound prevents instability when normalizing by very small regions.
  • Interactive views: The By-Image view coordinates image selection, safety sorting, participant filtering, data toggles, and gaze overlays for streetscape analysis.The By-Participant view aggregates behavior across images, with semantic attention matrices and a similarity projection for comparing scenes and ratings.

7. Usage Scenarios

UrbanGazeVis supports urban-safety gaze analysis through participant-, image-, and region-focused views that connect attention patterns with perceived safety. Its usage scenarios reveal rating differences associated with disorder and distinct temporal strategies for attending to human presence.

  • Usage scenarios: Three usage scenarios demonstrate UrbanGazeVis workflows for analyzing eye-gaze data in urban safety studies.The scenarios use participant-centric, image-centric, similarity, attention, coverage, ROI, and temporal views.
  • Participant and image analysis: The By-Participant workflow compares Images 60, 38, and 109 by mean perceived safety, while AttIn highlights strongly attended elements with low spatial coverage.The images are sorted from unsafe to safe, and total-attention heatmaps emphasize large background elements such as construction and floor.
  • Temporal divergence: For Image ID 20, ROI Glyph and Temporal View distinguish early visits, late visits, and revisits to the person class during the 15-second trial.Participant 27 makes an early single visit, Participant 30 a late single visit, and Participant 16 repeatedly returns to the region; these patterns are not visible in conventional spatial heatmaps.
  • Similar-image comparisons: Participant comparisons show that concentrated gaze on graffiti coincides with lower ratings, whereas attention to construction and pedestrians can accompany higher or moderate safety judgments.Participant 10 rates Image 145 as 8 and Image 100 as 2; Participant 22 rates Image 72 as 6 and Image 38 as 3, with differing gaze distributions.

8. System Evaluation and Value-Driven Assessment

A study of 22 non-expert eye-tracking analysts assessed UrbanGazeVis through four visual analytics tasks and an ICE-T-inspired questionnaire. Results showed strong perceived confidence, essence, and usability, alongside cognitive-effort and discoverability limitations.

  • Evaluation Design: 22 participants with general data-analysis experience but no prior eye-tracking exposure evaluated UrbanGazeVis after a 10-minute interface introduction.The study emphasized accessibility for broader audiences and used ICE-T dimensions: Insight, Confidence, Essence, and Time.
  • Evaluation Design: Four tasks examined dominant attention elements, sequential gaze trajectories, localized urban disorder, and cross-image participant viewing patterns.These tasks operationalized analytical design tasks spanning spatial attention, temporal scanpaths, urban features, fatigue-related strategy shifts, and judgment consistency.
  • Quantitative Results: Confidence scored highest at 6.3, followed by Essence at 6.2, Time at 6.1, and Insight at 5.9, all above the neutral midpoint of 4.Insight responses were more varied, while coordinated views, consistent color encodings, concise summaries, and positive time ratings supported the other dimensions.
  • Limitations and Improvements: The By-Participant mode required more cognitive effort because of numerical density, while users requested broader search and attention-based sorting.Temporal visualizations also required slightly more cognitive effort, and the evaluation intentionally omitted task completion and error metrics to emphasize qualitative insight generation.
  • Qualitative Feedback: Participants identified the Participant Attention Summary and Temporal Participant Attention scarf plot as the most intuitive components.Cross-view filtering was considered essential for reducing clutter and supported comparisons, numerical interpretation, and chronological scanpath tracking.

9. Discussion

Discussion findings show that sustained attention to physical disorder is negatively associated with perceived urban safety, while fine-grained semantic representations expose effects that broad aggregation can obscure. UrbanGazeVis also reveals contextual moderation, including higher safety ratings when pedestrians are present despite visible disorder.

  • Physical disorder and safety: Sustained attention to broken or damaged brick walls was the strongest negative predictor of perceived safety (β = −0.442, p < .001).Damaged pavement (β = −0.229) and graffiti (β = −0.225) followed closely.
  • Semantic granularity: Fine-grained ADE20K+UrbanPD4k separates general infrastructure from decay signals, distinguishing blind walls (β = −0.326) from physically damaged wall cues.Raw ADE20K instead merges deterioration into broad categories such as wall and absorbs structural decay into background elements.
  • Semantic granularity: Grouped-ADE produced severe multicollinearity, with VIF values of 43.29 for Construction and 29.78 for Floor, making several predictors insignificant.Merging fine-grained infrastructure and ground elements into broad clusters inflated standard errors and destabilized safety analysis.
  • Contextual effects: Scenes with active human presence maintained substantially higher safety ratings than visually identical isolated spaces, even amid prominent disorder cues like graffiti.The comparison involved Image ID 72 with pedestrians and Image ID 10 as an isolated space.

10. Limitations and Future Work

The study’s findings are limited by participant and experimental-context constraints, visualization scalability, coarse semantic representations, correlational attention analyses, and the absence of macro-geographic mapping. Future work should pursue more diverse and scalable studies, finer-grained urban-disorder representations, causal analysis, and city-scale geospatial integration.

  • Participant and experimental context: The sample of 30 participants from a single institution does not represent Rio de Janeiro’s socioeconomic diversity, and the experiment used static street-view images viewed for 15 seconds in a controlled head-mounted setting.The sample was sufficient for methodological evaluation but limits generalizability.
  • Scalability and representation: UrbanGazeVis visualizations may require filtering or aggregation for larger cohorts, while Places365 embeddings may miss fine-grained disorder cues and t-SNE projections are sensitive to initialization.The system was designed for a moderate dataset of 150 images and 30 participants, with t-SNE using only 50 images per participant.
  • Causal interpretation: Attention to physical disorder is associated with lower safety ratings, but the relationship is correlational rather than causal.Future work should investigate scalable visualization, dimensionality-reduction strategies, and domain-specific representations for urban disorder cues.
  • Spatial mapping: UrbanGazeVis lacks a macro-geographic map layer because the dataset prioritizes structural diversity over geographic coverage, making sampled-location maps vulnerable to implying unsupported spatial trends.Future work should connect micro-level gaze analysis with city-scale geospatial representations for urban planning.

11. Conclusions · Appendix A. Multivariate Stratified Sampling Optimization · Appendix B. Randomized Image Assignment Matrix

UrbanGazeVis unifies gaze, scene semantics, disorder annotations, and safety ratings to reveal context-specific drivers of urban safety perception. Its appendices describe sampling and randomized assignment procedures for constructing and balancing the 150-image study subset.

  • 11. Conclusions: UrbanGazeVis integrates gaze dynamics, semantic segments, disorder annotations, and safety ratings from 30 participants viewing 150 Rio de Janeiro street-view images.The system provides a unified framework for exploring eye-movement behavior in urban safety perception.
  • 11. Conclusions: Image-level judgments alone do not fully explain perceived safety, because gaze and inspected elements expose context-specific patterns hidden by global aggregation.The analysis distinguishes contributions from disorder cues, infrastructure, vegetation, and human presence.
  • Appendix A. Multivariate Stratified Sampling Optimization: Multivariate stratified sampling preserves diversity in visual-element proportions by grouping each element’s pixel-ratio distribution into quartiles.For each visual element v, images are divided into quartile groups based on their proportions.
  • Appendix A. Multivariate Stratified Sampling Optimization: The optimization selects a binary keep-or-discard vector while enforcing a minimum number of images in every visual-element quartile.This procedure yields the final 150-image subset.
  • Appendix B. Randomized Image Assignment Matrix: The randomized assignment matrix requires every image to receive exactly KP distinct participant ratings and every participant to view exactly KI images.The quantities satisfy niKP = npKI, ensuring balanced participant-image assignments.
  • Appendix B. Randomized Image Assignment Matrix: The study targeted KP = 10 ratings per image and iteratively filled an np × ni binary matrix, where Vp,i = 1 means participant p views image i.The matrix encodes the randomized image assignment procedure.

Appendix C. Extended Experimental Protocol and Hardware Setup … Appendix I. Segmentation Model Comparison

The appendices document the study protocol, questionnaire, reliability analysis, statistical model variants, and segmentation-model comparison underlying UrbanGazeVis. Together, they specify controlled data collection and calibration procedures while reporting positive evaluations, high rating reliability, disorder-focused model outputs, and OneFormer’s comparative advantages.

  • Appendix C. Extended Experimental Protocol and Hardware Setup: The protocol used three phases within 25 minutes, including nine-point HoloLens 2 calibration, familiarization with safe and unsafe images, and buffered safety-rating trials.Each participant viewed 50 assigned images for 15 s, followed by a 5 s perceived-safety rating on a 1–10 Likert scale.
  • Appendix D. ICE-T Questionnaire and Open-Ended Questions: The ICE-T questionnaire assessed Insight, Confidence, Essence, and Time through two statements per dimension on a 7-point Likert scale, supplemented by open-ended questions.Questions addressed attention metrics, coordinated views, semantic-class colors, gaze overlays, temporal tools, filtering, navigation, and interface improvements.
  • Appendix E. ICE-T Results Per Participants: ICE-T scores per participant indicated consistently positive evaluations across all four dimensions.The results are presented in Table E.1.
  • Appendix F. Spatial Uncertainty and Kernel Calculation: The spatial-uncertainty procedure converted HoloLens 2 precision into a Gaussian heatmap kernel of σ ≈10.68 pixels and a conservative semantic-mask threshold of τ = 0.005 (0.5%).The device precision was 0.24◦ at 2.0 m, corresponding to approximately 44.5 Pixels Per Degree; the hardware noise floor occupied 0.3% of the visual field before padding.
  • Appendix G. Inter-Rater Reliability (IRR) and Rater Calibration: Given the ill-structured measurement design, crossed random-effects LMM estimation and the G(q, k) estimator separated scene signal, participant bias, and residual noise for partial-overlap ratings.The analysis used N = 1,500 observations, k = 10 raters per image, and q ≈0.67; reliability reached G(q, k) = 0.812.
  • Appendix H. Detailed Statistical Results for LMM; Appendix H.1. ADE20K Base Model Results; Appendix H.2. Grouped-ADE Model Results; Appendix H.3. ADE20K + UrbanPD4k Model Results; Appendix H.4. Grouped-ADE + UrbanPD4k Model Results; Var. SD ICC: The statistical appendices report fixed effects for ADE20K Base, Grouped-ADE, ADE20K + UrbanPD4k, and Grouped-ADE + UrbanPD4k, excluding broad contextual categories from summary tables.The baseline model benchmarks semantic granularity, Grouped-ADE exhibits multicollinearity in some categories, and hybrid models evaluate physical-disorder predictors including Broken/Damaged Wall and Graffiti.
  • Appendix I. Segmentation Model Comparison: OneFormer generally provided more accurate and versatile segmentation than SegFormer, PSPNet, and DeepLab, despite requiring more computational resources.The other models failed on light poles, distinctions among walls, sidewalks, and gates, or correct car segmentation.
Loading 2608.21686v1…