PaperlayerView exact revision on arXiv

Source-linked AI summary

Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning

Zhibo Yang, Lihan Huang, Yupei Chen, Zijun Wei, Seoyoung Ahn, Gregory Zelinsky, Dimitris Samaras, Minh Hoai

arXiv:2005.14310v2cs.CV

TL;DR

Free-viewing saliency models do not generalize to goal-directed visual search, and suitable search-fixation datasets have been limited. The paper uses IRL with dynamic contextual beliefs and introduces COCO-Search18; its model outperformed comparable methods in predicting search scanpaths.

  • Problem

    Free-viewing saliency models do not generalize to goal-directed tasks such as target-object search, and existing datasets have limitations for training search-fixation models.

    Source: §1 · p. 2

  • Method

    The paper applies IRL to recover a reward function and policy for search scanpaths, using dynamic contextual beliefs to update beliefs about objects as fixation locations change.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    The IRL model outperformed comparable models in predicting search scanpaths on COCO-Search18.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The authors suggest recovered reward maps may support visual-content indexing by likely attention and that better search prediction matters for robotic search and human-computer systems.

    Source: §5 · p. 8

Abstract

from arXiv · show

Being able to predict human gaze behavior has obvious importance for behavioral vision and for computer vision applications. Most models have mainly focused on predicting free-viewing behavior using saliency maps, but these predictions do not generalize to goal-directed behavior, such as when a person searches for a visual target object. We propose the first inverse reinforcement learning (IRL) model to learn the internal reward function and policy used by humans during visual search. The viewer's internal belief states were modeled as dynamic contextual belief maps of object locations. These maps were learned by IRL and then used to predict behavioral scanpaths for multiple target categories. To train and evaluate our IRL model we created COCO-Search18, which is now the largest dataset of high-quality search fixations in existence. COCO-Search18 has 10 participants searching for each of 18 target-object categories in 6202 images, making about 300,000 goal-directed fixations. When trained and evaluated on COCO-Search18, the IRL model outperformed baseline models in predicting search fixation scanpaths, both in terms of similarity to human search behavior and search efficiency. Finally, reward maps recovered by the IRL model reveal distinctive target-dependent patterns of object prioritization, which we interpret as a learned object context.

1. Introduction

Goal-directed visual search requires predicting target-guided scanpaths, which free-viewing saliency models do not reliably predict. The paper introduces an IRL model and COCO-Search18 to address the shortage of suitable search-fixation data.

1. Introduction

  • Most fixation-prediction models address free viewing, and their saliency-based predictions do not generalize to goal-directed tasks such as target-object search.Earlier search datasets were limited by task mismatch or small numbers of target categories, subjects, or fixations.Source: §1 · pp. 1–2
  • Rather than using a preset algorithm to prioritize image locations, the IRL approach learns search-fixation sequences as reward-bearing behavior and recovers a reward function and policy.The paper applies GAIL to learn from expert behavior represented as state-action pairs.Source: §1 · p. 2
  • COCO-Search18 provides high-quality search fixations from 10 people across 18 target-object categories, addressing limitations in earlier fixation datasets.The authors describe it as the world’s largest dataset of images annotated with human search fixations.Source: §1 · p. 2
  • Dynamic Contextual Beliefs (DCB) is a novel state encoder that updates beliefs about peripherally viewed objects as the simulated fovea moves.The authors identify integration of fixation-location changes with state-representation changes as a technical novelty.Source: §1 · p. 2
  • The IRL model outperformed other state-of-the-art methods in predicting search scanpaths, generalized to new subjects, and needed less data for good performance.The authors also report that its reward maps reflect target-dependent object prioritization.Source: §1 · p. 3

2. Scanpath Prediction Framework

The framework treats visual search as sequential, goal-directed fixation and learns a policy and reward from human behavior. Dynamic Contextual Beliefs represent accumulated visual information, while adversarial imitation learning trains the policy and reward function.

2. Scanpath Prediction Framework

  • The framework models search as sequential fixation: an agent receives a retina-transformed image and chooses actions to maximize learned rewards.The reward is treated as a function of state and action and learned jointly with the imitation policy.Source: §2 · p. 3

2.1. State Representation Modeling

  • Dynamic Contextual Beliefs combine foveal input, contextual object and background beliefs, and fixation-driven dynamics into the viewer’s evolving state.The state uses belief maps for 134 thing and stuff categories, augmented with a one-hot vector for the search task.Source: §2.1 · pp. 3–4
  • At each fixation, the state replaces low-resolution beliefs in the fixated region with corresponding high-resolution beliefs, retaining low-resolution beliefs elsewhere.The initial state is the low-resolution belief map; the fixation mask specifies the updated region.Source: §2.1 · p. 4

2.2. Reward and Policy Learning

  • GAIL trains a policy generator to imitate human state-action pairs and a discriminator-based reward function through adversarial optimization.The reward is the logarithm of the discriminator output, and the generator is optimized as an RL policy using PPO.Source: §2.2 · p. 4

3. COCO-Search18 Dataset

COCO-Search18 provides human fixation data for visual search across 18 target categories, with target-present and target-absent trials. The dataset uses controlled image-selection criteria and laboratory eye tracking, and the reported model data comprise a subset of target-present fixations.

3. COCO-Search18 Dataset

  • COCO-Search18 contains 6,202 images for 18 target categories, with equal numbers of target-present and target-absent images.The target-present images were selected under criteria governing target instances, size, position, and image ratio; target-absent images contained target siblings.Source: §3 · pp. 4–5
  • Ten participants viewed all images while their eye positions were recorded at 1,000 Hz across six laboratory sessions.Participants made a target-present or target-absent judgment for each image; sessions randomly interleaved both trial types.Source: §3 · p. 5
  • The study extracted 299,037 fixations, then retained 100,232 target-present search fixations for training and testing after exclusions.Evaluation used random within-category splits of 70% training, 10% validation, and 20% test; Figure 4 summarizes normalized gaze measures across categories.Source: §3 · p. 5, Figure 4 · p. 5

4. Experiments

The experiments evaluate IRL scanpath prediction against alternative methods, then test generalization, contextual guidance, state representations, and data efficiency. IRL leads the model comparison, while the analyses show that its guidance depends on target and context.

4. Experiments

  • Experiments compared IRL with heuristic and behavior-cloning baselines using target-present trials only; evaluation included scanpath similarity, target guidance, and search efficiency.The study reserved target-absent analyses for future work.Source: §4 · p. 5, §4.1 · pp. 5–6

4.1. Comparing Scanpath Prediction Models

  • IRL reward maps shifted with the target, prioritizing a laptop for mouse search, a monitor for TV search, and other cars rather than a stop sign for car search.The stop sign became strongly prioritized when it was the target.Source: §4.1 · p. 6, Figure 6 · p. 7
  • IRL outperformed the other methods on all scanpath metrics in the COCO-Search18 test comparison.The evaluation included scanpath similarity, search guidance, and search efficiency measures.Source: §4.1 · p. 6, Table 2 · p. 7

4.2. Group Model vs Individual Model

  • Group models trained on nine other subjects performed similarly to individual models trained on the held-out subject, suggesting generalization to new searchers.The reported performance gap was insignificant.Source: §4.2 · p. 7, Figure 7 · p. 7

4.3. Context Effects

  • Context objects could guide or distract search: pavement helped car search, trucks distracted, and fork distracted knife search, plausibly because of peripheral similarity.The analysis measured changes in target-fixation probability after switching off a context-object belief map.Source: §4.3 · p. 7, Figure 8 · p. 8
  • Context maps also encoded spatial relationships, with gaze directed toward the top of tables or refrigerators for microwave search and right of keyboards for mouse search.The maps compare policy probability maps with and without the context-object belief map.Source: §4.3 · pp. 7–8, Figure 9 · p. 8

4.4. Ablation Study on State Representation

  • Removing target and context object maps affected performance most, stuff mattered next, and saliency and history had weak effects; replacing DCB with CFI degraded IRL performance.DCB contains 136 belief maps across target, context-object, stuff, saliency, and fixation-history groups.Source: §4.4 · p. 8

4.5. Data Efficiency

  • IRL achieved comparable or better results than BC-CNN with fewer training images per category, a data-efficiency advantage the authors attribute likely to adversarial augmentation.The authors note that data efficiency matters when collecting fixations for new categories is costly.Source: Figure 10 · p. 8, §4.5 · p. 8

5. Conclusions

The paper concludes that IRL can predict visual-search scanpaths by recovering reward and policy with dynamic contextual beliefs, and reports that it outperformed comparable models. It also points to possible applications in attention-aware systems and visual-content analysis.

5. Conclusions

  • The proposed IRL model jointly recovers the reward function and policy used during visual search, using dynamic contextual beliefs, and outperformed comparable models on search scanpath prediction.The authors also introduced COCO-Search18, a large-scale dataset of goal-directed search fixations.Source: §5 · p. 8
  • The authors suggest that better search-behavior prediction could benefit robotic search and attention-aware human-computer systems.They also suggest using reward maps to annotate and index visual content by likely attention.Source: §5 · p. 8
  • The work addresses a behavioral-vision question: which visual features guide human goal-directed attention in real images.Source: §5 · p. 8

A. Appendix

The appendix provides supplementary dataset, model, implementation, experimental, ablation, and interpretation details.

A. Appendix

  • The appendix covers COCO-Search18, Dynamic Contextual Beliefs, implementation, and additional experimental, ablation, and interpretation results.Source: A. Appendix · p. 11

A.1. Details about COCO-Search18 Dataset

COCO-Search18 annotates COCO images with gaze fixations from target-present and target-absent visual-search trials. Its appendix describes image selection, participants, trial procedure, and sample scanpaths.

A.1. Details about COCO-Search18 Dataset

  • COCO-Search18 annotates COCO images with human gaze fixations from standard target-present and target-absent search trials.Source: §A.1 · p. 11
  • The dataset contains 3,101 target-present images across 18 categories and an equal number of target-absent images, totaling 6,202.Target-present images with highly occluded or difficult-to-recognize targets were excluded; target crops had to score above .99 on the classifier.Source: §A.1 · p. 11, Table 3 · p. 11
  • Ten university undergraduate and graduate students with normal or corrected-to-normal vision participated in gaze collection.Participants were naive to the experimental question and design.Source: §A.1 · p. 11
  • Trials presented randomly interleaved target-present and target-absent images, with participants judging target presence and responding as quickly as possible while remaining accurate.Eye movements were recorded with an EyeLink 1000; example scanpaths show fixation order for the 18 target categories.Source: §A.1 · pp. 11–12, Figure 11 · p. 12

A.2. Detailed Description of DCB

Dynamic Contextual Belief (DCB) combines image features at different blur levels with fixation history to represent the current state. The model also conditions its predictions on the target-object category.

A.2. Detailed Description of DCB

  • DCB combines high-resolution features from the original image with low-resolution features from a blurred image, using a fixation-history mask to select between them.The images are resized and passed through Panoptic-FPN; its feature maps are resized to 20×32.Source: §A.2 · p. 12
  • At each fixation, the resulting belief representation is concatenated with a one-hot task embedding to form a new state.The task embedding is spatially repeated and concatenated with inputs to each convolutional layer, equivalent to a task-dependent bias.Source: Figure 12 · p. 13, §A.2 · p. 13

A.3. Implementation details

The implementation discretizes fixations into a grid and uses policy, critic, and discriminator networks trained with PPO. Scanpaths are sampled from model predictions with inhibition of return, alongside fixed-map baselines.

A.3. Implementation details

  • The action space is a 20×32 grid of 640 patches, with each selected patch’s center used as the fixation location.The maximum discretization error is 1.75 degrees of visual angle.Source: §A.3 · p. 13
  • The IRL model uses a convolutional policy network, a critic that estimates state value, and a discriminator that distinguishes generated from human fixation pairs.The policy outputs a probability map; the critic estimates expected return and the discriminator shares the policy network’s structure with a sigmoid output.Source: §A.3 · p. 13
  • Training alternates discriminator updates on generated and human state-action pairs with joint policy–critic updates using PPO.Training used 20 epochs and took about 40 minutes on one NVIDIA Tesla V100 GPU; the policy and discriminator learning rate was 0.0005.Source: §A.3 · pp. 13–14
  • The four illustrated IRL scanpaths have sequence scores of 0.33, 0.40, 0.50, and 0.75, from top to bottom.The figure also plots the cumulative distribution of generated scanpath sequence scores.Source: Figure 13 · p. 14
  • The detector baseline predicts target-location probabilities, while the fixation-heuristics baseline is trained on behavioral fixation density maps.The fixation-density maps are generated from 10 subjects on the training images.Source: §A.3 · p. 14
  • At each scanpath step, predictive methods sample from a newly generated probability map and apply inhibition of return, while detector and fixation-heuristic maps remain fixed.Source: §A.3 · p. 14

A.4. Additional Experiment Results

Additional experiments compare state representations, training-data efficiency, state components, and reward policies. DCB improves search efficiency and scanpath similarity over CFI, while non-greedy policies better match human scanpaths than greedy policies.

A.4. Additional Experiment Results

  • 0.422: the IRL model’s average generated-scanpath sequence score; illustrated examples range from 0.33 to 0.75.Source: §A.4 · pp. 14–15
  • 45%: DCB reduces the search gap versus CFI and raises scanpath ratio from 61.9% to 82.6%, closer to the human behavioral ratio of 86.2%.The DCB policy network has 0.3M parameters versus 29.6M for CFI.Source: Table 4 · p. 15, §A.4 · pp. 15–16
  • Target-object and “thing” maps are the most critical DCB components for human-like scanpaths, followed by “stuff” maps; saliency and history have little impact.The ablation removes each map group in turn from the full representation.Source: Table 5 · p. 15, §A.4 · p. 16
  • Non-greedy policies generate more human-like scanpaths than greedy policies across all tested metrics.Greedy policies maximize immediate reward; non-greedy policies maximize total reward across a scanpath.Source: Table 6 · p. 15, §A.4 · p. 16
  • IRL achieves comparable or better results than CNN-based behavior cloning with less training data when both use DCB.Table 7 reports results across different numbers of training images per category.Source: Table 7 · p. 15, §A.4 · p. 16

Related papers

Loading 2005.14310v2…