PaperlayerView exact revision on arXiv

Source-linked AI summary

Poivre: Self-Refining Visual Pointing with Reinforcement Learning

Wenjie Yang, Zengfeng Huang

arXiv:2509.23746v1cs.CVcs.AI

TL;DR

Current VLMs fall short of human performance on visual pointing, which is typically completed in one step. Poivre lets a VLM mark and iteratively refine its prediction, using reinforcement learning with a process reward. Poivre-7B sets a Point-Bench state of the art and also generalizes to robotics tasks without task-specific training.

  • Problem

    Current VLMs remain below human performance on visual pointing and are usually constrained to produce a result in one step rather than observe and adjust.

    Source: §1 · p. 2

  • Method

    Poivre has a VLM mark an initial location and refine it over subsequent rounds, with reinforcement learning using a PBRS-inspired process reward.

    Source: §1 · p. 2, §4 · p. 7

  • Results

    2.7% in success rate over the strongest baseline on Point-Bench, Poivre-7B's state-of-the-art margin.

    Source: §5.2 · p. 9

  • Takeaways & Limitations

    Poivre-7B shows strong generalization to robotics tasks without task-specific training.

    Source: §6 · p. 12

    The study addresses visual pointing; extending the method to all vision tasks is beyond its scope.

    Source: C Limitations and Future Work · p. 18

Abstract

from arXiv · show

Visual pointing, which aims to localize a target by predicting its coordinates on an image, has emerged as an important problem in the realm of vision-language models (VLMs). Despite its broad applicability, recent benchmarks show that current VLMs still fall far behind human performance on this task. A key limitation is that VLMs are typically required to complete the pointing task in a single step, akin to asking humans to point at an object without seeing their own fingers. To address this issue, we propose a simple yet effective self-refining procedure: Point, Visualize, then Refine (Poivre). This procedure enables a VLM to first mark its estimated point, then iteratively refine the coordinates if necessary. Inspired by advances of reasoning models in the natural language domain, we employ reinforcement learning (RL) to incentivize this self-refining ability. For the RL training, we design a neat process reward that is not only empirically effective but also grounded in appealing properties. Our trained model, Poivre-7B, sets a new state of the art on Point-Bench, outperforming both proprietary models such as Gemini-2.5-Pro and large open-source models such as Molmo-72B by over 3%. To support future research, we release our training and inference code, dataset, and the Poivre-7B checkpoint.

1 Introduction

Current vision-language models (VLMs) lag human performance on visual pointing, which is commonly treated as a one-shot task. The paper proposes iterative self-refinement and reinforcement learning (RL) to address this gap, and reports strong benchmark and robotics results.

1 Introduction

  • Current VLMs fall short of human-level visual pointing, while their usual one-shot setting omits the iterative adjustment people use.The paper likens this to asking people to point without seeing their fingers.Source: §1 · p. 2
  • Poivre has a model predict a target location, mark that estimate on the image, and refine it in later rounds if needed.The paper says inference can extrapolate to more refinement steps than used in training.Source: §1 · pp. 2–3
  • RL with a PBRS-inspired process reward encourages improvement across refinement steps; Poivre-7B sets a Point-Bench state of the art.The reward targets stepwise improvement rather than only the final outcome.Source: §1 · pp. 2–3
  • The authors report experiments on reward effectiveness, refinement-step extrapolation, and robotics generalization, and release code, a dataset, and the Poivre-7B checkpoint.They identify visual test-time scaling and chain-of-thought prompting as future research directions.Source: §1 · p. 3

2 Related Work

Related work situates visual pointing among VLM and robotics research, noting that reinforcement learning for visual pointing remains limited. The paper connects self-refinement to iterative correction and test-time scaling while distinguishing it from prior work on other tasks or single-turn training.

2 Related Work

  • Visual pointing spans open-source VLMs and datasets, including Molmo and Pixmo [Deitke et al., 2025], and robotics work such as RoboPoint [Yuan et al., 2024a].Pixmo-Points contains 223k images and 2.3M question–point annotations; the paper says RL for visual pointing remains limited.Source: §2 · p. 3
  • VisionReasoner uses a grounding-point L1 reward but single-turn RL, whereas Point-RFT targets question answering rather than visual pointing [Ni et al., 2025].The paper also notes that related GUI-grounding studies do not adopt its self-refining VLM approach [Tang et al., 2025; Yuan et al., 2025; Zhou et al., 2025].Source: §2 · p. 4
  • The paper frames visual self-refinement as test-time scaling or visual chain-of-thought, building on iterative self-correction in language models [Kumar et al., 2025].It notes that visual self-refinement remains less mature than its natural-language counterpart.Source: §2 · p. 4

3 Preliminary

The preliminary section defines visual pointing as predicting coordinates for target objects from an image and language instruction. It adopts GRPO by default and emphasizes that rollout format and reward design should reflect the desired self-refining behavior.

3 Preliminary

  • Visual pointing supplies an image and natural-language description, then extracts predicted coordinates for one or more target objects from the VLM response.The setup follows Point-Bench [Cheng et al., 2025].Source: §3 · p. 4
  • Unless otherwise specified, the paper uses GRPO as its default reinforcement-learning algorithm.The authors leave evaluation of alternatives such as DAPO, GSPO, and GMPO for future work because of resource constraints.Source: §3 · pp. 4–5
  • GRPO samples multiple rollouts per input and optimizes a clipped objective using policy ratios, reward-based advantages, and KL regularization.The stated objective maximizes performance over rollouts sampled from the old policy.Source: §3 · p. 5
  • Rollout inputs should match inference format, and rewards should assess rollout quality so training favors the intended self-refining process and discourages reward hacking.The paper treats rollout sampling and reward computation as especially important components of GRPO in this setting.Source: §3 · p. 5

4 Method

Poivre has a VLM predict a target point, visualize it on the image, and refine it over further rounds. Its PBRS-inspired process reward scores both the initial point and subsequent refinements.

4 Method

  • Poivre mirrors inference during RL rollouts: the VLM predicts a point, sees that mark on the image, and can refine it over further rounds.The procedure generalizes to any desired number of rounds.Source: §4 · pp. 5–6
  • A final-prediction-only reward ignores intermediate predictions, treating random early guesses and progressive refinement alike when the final prediction is similar.The authors describe this as suboptimal for RL training.Source: §4 · pp. 6–7
  • The PBRS-inspired reward uses changes in outcome-reward potential between successive target distances, with the new prediction as the action.The states are distances to the target, and the potential is the outcome reward for that distance.Source: §4 · p. 7
  • The process reward combines an initial-pointing reward with rewards for subsequent refinements, which assess the quality of those refinements.The authors report that experiments show an advantage over the vanilla outcome reward.Source: §4 · p. 7
  • For usual parameter choices, the reward can be viewed as a weighted average across turns that emphasizes the initial and final estimates.The initial estimate sets the difficulty of refinement, while the final prediction reflects its ultimate quality.Source: §4 · p. 7

5 Experiments

The experiments evaluate Point-Bench performance, reward design, test-time refinement-round extrapolation, robotics transfer, single-turn RL, and real-world refinement behavior.

5.1 Experimental Setup

  • For RL training, the authors subsample 8,192 pairs from Pixmo-Points and fix the number of refinement rounds at T = 2.The subsampling is due to resource constraints; the authors note that training-data scale and scope could be expanded.Source: §5.1 · p. 8

5.2 Experimental Results

  • 2.7% in success rate over the strongest baseline makes Poivre-7B the new state of the art on Point-Bench.The comparison is reported in Table 1.Source: Table 1 · p. 9, §5.2 · p. 9
  • The PBRS-inspired process reward improves Point-Bench performance by about 1.3% over outcome reward alone.Table 2 compares the two reward strategies.Source: Table 2 · p. 10, §5.2 · p. 10
  • With training fixed at T = 2, Poivre-7B further improves performance when inference is extended to T = 3.The authors identify optimizing continued inference-time scaling as an open question.Source: Table 3 · p. 10, §5.2 · p. 10
  • Although not explicitly trained on robotics data, Poivre-7B’s pointing improvement transfers to the where2place robotics dataset.Table 4 reports performance on where2place.Source: Table 4 · p. 11, §5.2 · p. 11
  • VanillaRL improves initial performance but fails to develop self-refinement and is severely unstable during iterative inference.VanillaRL uses single-turn RL and is evaluated with the same iterative inference procedure.Source: §5.2 · p. 11, Figure 3 · p. 12
  • Real-world examples show Poivre-7B correcting initially incorrect coordinates through refinement to complete the pointing task.Figure 4 presents these examples.Source: §5.2 · p. 12

6 Conclusion

Poivre combines iterative visual refinement with reinforcement learning, and Poivre-7B achieves state-of-the-art Point-Bench results while generalizing to robotics tasks. The conclusion also reports continued improvement with inference rounds beyond training and releases code, data, and the model.

6 Conclusion

  • Poivre-7B sets a new state of the art on Point-Bench and generalizes to robotics tasks without task-specific training.It outperforms proprietary and large open-source models.Source: §6 · p. 12
  • Poivre-7B continues to improve when inference rounds exceed those used during training.Source: §6 · p. 12
  • Poivre lets a VLM observe and iteratively refine its predictions, with a PBRS-inspired process reward used to incentivize self-refinement.Source: §6 · p. 12
  • The paper releases its code, data, and model to support further community progress.Source: §6 · p. 12
  • Figure 4 shows real-world examples in which Poivre-7B completes pointing through refinement.Brown points mark visualized coordinates, and arrows show the shift from turn 1 to turn 2.Source: Figure 4 · p. 13

A Proof

The proof presents the PBRS-inspired process reward as a weighted average of point rewards across turns. Under the stated condition, the initial and final estimates receive the greatest weights for usual parameter choices.

A Proof

  • The PBRS-inspired process reward can be viewed as a weighted average of point rewards across turns, with the initial and final estimates most important for usual parameter choices.Source: A Proof · p. 17
  • The stated weight ordering holds when the number of turns satisfies T ≤ 1 + log(1−γ)/log(γ).Source: A Proof · pp. 17–18
  • With γ = 0.9, the stated condition is met for T ≤ 22.Source: A Proof · p. 18

B The Use of Large Language Models

The authors report using large language models only to polish the writing and state that they take full responsibility for the contents.

B The Use of Large Language Models

C Limitations and Future Work

The paper focuses on visual pointing rather than extending its method to all vision tasks. It identifies downstream applications such as robotics, assistive technologies, and education as future work.

C Limitations and Future Work

Loading 2509.23746v1…