Source-linked AI summary
The 2018 DAVIS Challenge on Video Object Segmentation
Sergi Caelles, Alberto Montes, Kevis-Kokitsi Maninis, Yuhua Chen, Luc Van Gool, Federico Perazzi, Jordi Pont-Tuset
TL;DR
Existing DAVIS setups use either detailed first-frame masks or no user input, while the paper identifies a need to assess interactive segmentation and response time. The challenge adds a scribble-based interactive track, and its baselines reach or exceed full-supervision performance in shorter times.
Problem
Method
Results
Scribble-OSVOS surpasses fully supervised OSVOS’s 57% at 1,200 seconds in 200 seconds; the support vector classifier matches 39.26% (110 seconds) in 13 seconds.
Source: §3 · p. 4Takeaways & Limitations
The challenge aims to encourage interactive segmentation research toward video object segmentation usable in realistic scenarios.
Source: §4 · p. 4The experiments examine the fastest OSVOS version, with fine-tuning taking 60 seconds, to keep waiting time between interactions low.
Source: §3 · p. 3
Abstract
from arXiv · showhide
We present the 2018 DAVIS Challenge on Video Object Segmentation, a public competition specifically designed for the task of video object segmentation. It builds upon the DAVIS 2017 dataset, which was presented in the previous edition of the DAVIS Challenge, and added 100 videos with multiple objects per sequence to the original DAVIS 2016 dataset. Motivated by the analysis of the results of the 2017 edition, the main track of the competition will be the same than in the previous edition (segmentation given the full mask of the objects in the first frame -- semi-supervised scenario). This edition, however, also adds an interactive segmentation teaser track, where the participants will interact with a web service simulating the input of a human that provides scribbles to iteratively improve the result.
1 INTRODUCTION
The 2018 DAVIS Challenge keeps the previous edition’s semi-supervised main track and adds an interactive teaser track. The new track addresses evaluation challenges by simulating iterative human input and measuring result quality alongside response time.
1 INTRODUCTION
- The 2018 main track retains the 2017 semi-supervised setup, giving algorithms the object mask in the first frame and evaluating segmentation in later frames.The dataset and its partitions also remain as in the 2017 edition.Source: §1 · p. 1
- The teaser track adds interactive video segmentation, where users iteratively provide quick inputs to refine results.The paper motivates this setting as a middle ground between pixel-accurate first-frame masks and no user input.Source: §1 · p. 1
- The evaluation measures result quality over wall time, accounting for both the number of user interactions and the algorithm’s response time.The paper describes this as more realistic than semi- and unsupervised setups because it uses natural inputs and accounts for time.Source: §1 · p. 1
- The interactive evaluation simulates human corrections and uses a web service to protect test annotations while monitoring quality and computation time.The service receives inputs and segmentation results from a local process during a session.Source: §1 · p. 2
- The paper proposes interactive segmentation baselines and evaluation metrics to demonstrate the framework’s usefulness.The challenge is intended to promote research and help laboratories and industry discover algorithms for realistic scenarios.Source: §1 · p. 2
2 SEMI-SUPERVISED VIDEO OBJECT SEGMENTA-
The 2018 semi-supervised track retains the 2017 dataset and splits, with public training and validation annotations and restricted test annotations.
2 SEMI-SUPERVISED VIDEO OBJECT SEGMENTA-
- The track keeps the 2017 dataset and splits: 60 training, 30 validation, 30 test-dev, and 30 test-challenge sequences.The first-frame object mask is given to algorithms, which produce segmentations for the remaining frames.Source: §2 · p. 2
- Training and validation images and full annotations are public, while only first-frame annotations are public for both test sets.Test-dev accepts unlimited submissions; test-challenge allows five submissions within two weeks.Source: §2 · p. 2
- Detailed evaluation metrics are documented in the 2017 edition, while dates and instructions are available on the challenge website.Source: §2 · p. 2
3 INTERACTIVE VIDEO OBJECT SEGMENTATION
The section presents interactive video segmentation as a middle ground between pixel-wise masks and no user input, using iterative scribbles to refine results. It describes simulated feedback, web-service evaluation, time–accuracy metrics, and two baseline methods.
Interactive video object segmentation
- Users can add scribbles to erroneous regions on any frame and iteratively refine predictions, reaching fully supervised performance in less labeling time.Current foreground predictions are treated as no-care areas during Scribble-OSVOS retraining to avoid noisy labels.Source: §3 · pp. 2–3
Evaluation as a Web Service
- A web service evaluates client predictions, generates further simulated scribbles, and keeps test annotations private while supporting large-scale evaluation.The server supplies crowd-sourced initial scribbles, then evaluates returned predictions and generates the next interaction.Source: §3 · p. 2
Scribble simulation
- The scribble simulator removes small spurious detections, selects incorrect connected components, and uses simplified skeleton paths in false-positive and false-negative regions.The figure describes true positives, false negatives, and false positives before skeleton paths are selected.Source: §3 · p. 2, Figure 2 · p. 3
Evaluation metrics
- Interaction time combines user feedback time and algorithm response time, while quality and speed tracks measure J&F under a time budget or time to a minimum quality.The quality-versus-time figure plots J&F on the DAVIS 2017 validation set against available time.Source: §3 · p. 3, Figure 3 · p. 4
Baseline methods
- Scribble-OSVOS adapts OSVOS to scribbles using dilated foreground labels and a no-care band, but its fastest tested fine-tuning takes 60 seconds.The experiments use one object per fine-tuning step; scribbles for other objects are treated as background when available.Source: §3 · p. 3
- The second baseline extracts pixel features once and adapts an object-specific support vector classifier after new scribbles, avoiding repeated CNN feature training.It uses Deeplabv2 with ResNet-101 and 2048-feature vectors per pixel.Source: §3 · pp. 3–4
Evaluation results
- Manual and simulated scribbles produce similar conclusions across both baselines, supporting use of the simulator in the challenge.Scribble-OSVOS builds performance marginally faster with manual scribbles, while the support vector classifier shows the opposite trend.Source: §3 · p. 4
- Fully supervised OSVOS achieves 57% in 1200 seconds, surpassed by Scribble-OSVOS in 200 seconds; the scribble-supervised support vector classifier reaches fully supervised 39.26% in 110 seconds in 13 seconds.Only the 200-second duration is specified as wall time.Source: §3 · p. 4
4 CONCLUSIONS
The 2018 DAVIS Challenge retains the previous edition’s dataset and semi-supervised main track, while adding an interactive video segmentation teaser track. Its web service supplies scribbles iteratively, with response speed and segmentation quality treated as a trade-off.
4 CONCLUSIONS
- The main track retains the previous edition’s dataset and semi-supervised setup, using the perfect first-frame mask to segment the remaining frames.Source: §4 · p. 4
- The new teaser track evaluates interactive video segmentation through a web service that supplies scribbles and receives participants’ resulting segmentations.Source: §4 · p. 4
- The teaser track measures the trade-off between response speed and segmentation quality to encourage research toward video segmentation usable in realistic scenarios.Source: §4 · p. 4