Source-linked AI summary
The VIA Annotation Software for Images, Audio and Video
Abhishek Dutta, Andrew Zisserman
TL;DR
Manual annotation of images, audio, and video is a common processing stage requiring spatial or temporal regions and textual descriptions. The paper presents VIA, an offline, standalone browser-based tool for these tasks, and reports broad adoption across academic and industrial settings, including more than 1,000,000 uses by July 2019.
Problem
Manual annotation requires people to define and describe spatial regions in images or video frames and temporal segments in audio or video.
Method
VIA provides a standalone, offline browser application for spatial and temporal annotation, with workflows supporting human review of automatically generated image annotations.
Results
As of July 2019, VIA had been used more than 1,000,000 times, including 220,000 unique pageviews, and was adopted across academic disciplines and industrial sectors.
Takeaways & Limitations
VIA functions as an open source research-support tool for manual annotation across many academic disciplines and industrial settings.
Abstract
from arXiv · showhide
In this paper, we introduce a simple and standalone manual annotation tool for images, audio and video: the VGG Image Annotator (VIA). This is a light weight, standalone and offline software package that does not require any installation or setup and runs solely in a web browser. The VIA software allows human annotators to define and describe spatial regions in images or video frames, and temporal segments in audio or video. These manual annotations can be exported to plain text data formats such as JSON and CSV and therefore are amenable to further processing by other software tools. VIA also supports collaborative annotation of a large dataset by a group of human annotators. The BSD open source license of this software allows it to be used in any academic project or commercial application.
1 Introduction
VIA is a lightweight, standalone browser tool for manually annotating spatial regions in images and temporal segments in audio or video. Its offline, installation-free design supports rapid adoption and broad use.
- Manual annotation defines spatial regions in images or video frames and temporal segments in audio or video, described with textual metadata.Supported spatial shapes include rectangles, circles, ellipses, points, polygons, and polylines; temporal segments use start and end timestamps.
- VIA runs offline in modern web browsers without installation or setup and fits in a single self-contained HTML page smaller than 400 kilobyte.The software is written solely using HTML, Javascript and CSS.
- A minimalistic interface and rigorous testing made VIA configurable, simple, and easy to use for manual annotation.Testing was conducted internally and by the open source community.
- As of July 2019, VIA had been used more than 1,000,000 times, including 220,000 unique pageviews.
- The paper presents use cases, software design principles, the open source ecosystem, project impact, and planned extensions.
2 Image Annotation
VIA lets annotators define image regions with multiple geometric shapes and describe them using textual metadata. Predefined input lists help maintain consistent labels across annotators.
- VIA supports six image-region shapes: rectangle, circle, ellipse, polygon, point, and polyline.Rectangles commonly define object bounding boxes, polygons capture complex boundaries, and points mark feature locations such as facial landmarks or microscopy particles.
- Textual descriptions record visual content associated with each annotated region.
- Figure examples span actor faces, arbitrarily shaped microscope objects, 15th-century illustrations, and cryo-electron microscopy particles.The examples use rectangles, circles, polygons, and points across academic applications.
- Predefined checkbox, radio, image, and dropdown inputs maintain label-naming consistency between human annotators.
3 Image Group Annotation
VIA supports group annotation by combining automatically generated image annotations with human review, filtering, selection, and updates. Its image grid view helps annotators manage consecutive-frame face tracks and remove erroneous images.
- The two-stage workflow uses computer vision for preliminary annotation, followed by human filtering, selection, and updating of retained annotations.This approach reduces the annotation burden by limiting manual work to reviewing automatically produced annotations.
- For face tracks, detectors and tracking systems generate annotations that are imported into VIA for human review.Face tracks delineate and identify an individual’s face across consecutive video frames.
- In 165 consecutive BBC Sherlock frames, annotators assign face-track metadata such as track quality and name using the image grid view.Incorrect detections can be filtered by setting “is good track” to “No”.
- The grid view enables annotators to remove erroneous images from a group and supports retraining an existing image classifier using incorrectly classified images.
4 Audio and Video Annotation
VIA enables annotators to mark temporal segments in audio and video and describe them with metadata. One application is collaborative speaker diarisation, where speech segments are manually assigned to individuals.
- VIA allows human annotators to define and describe temporal segments in audio or video using textual metadata.
- Collaborative annotators use VIA to identify video segments containing each speaker’s speech for speaker diarisation.The annotations are used to assess the accuracy of automatic speaker-diarisation tools.
- Figure 4 shows human annotators manually identifying video segments containing speech from individual participants.
5 Software Design
VIA uses a minimal, browser-native design to provide lightweight offline annotation without installation or external libraries. Its familiar interface and self-contained implementation reduce deployment barriers for non-technical users.
- VIA uses standard HTML components, CSS, and browser features to provide a minimalist interface focused on annotated visual content.
- The software contains nearly 9000 lines of JavaScript but depends solely on standard modern-browser features and no external libraries.
- Browser-based offline deployment avoids installation and setup barriers that can affect non-technical users across computing systems.
6 Open Source Ecosystem
VIA’s open-source ecosystem extends the software through user feedback, bug fixes, feature contributions, and commercial adaptation under a flexible BSD license.
- Users report issues, request features, suggest fixes, and contribute code through VIA’s GitLab issue and merge-request portals.
- The BSD open-source license enables commercial organizations to seek advice and adapt VIA for internal or commercial applications.
7 Impact on Academia and Industry
VIA has been used as a research support tool across diverse academic disciplines and adapted by industrial organizations for specialized workflows.
- Academic impact: Academic users apply VIA to annotate historical illustrations, cosmological diagrams, microscopy particles, medical images, and computer-vision datasets.
- Industrial impact: Industrial organizations have integrated or adapted VIA for internal workflows and large-scale collaborative annotation.
8 Summary and Future Development
VIA is maintained as an evolving open-source annotation project, with planned development focused on collaboration and computer-vision assistance. A basic collaborative annotation implementation is already available.
- Summary and future development: VIA is continually developed to support manual annotation tasks across academic and industrial settings.
- Collaborative annotation: Collaborative annotation is intended to let multiple annotators independently and incrementally label large image and video collections.
- Plugins: Computer-vision plugins are planned to seed annotations or assist tasks such as segmentation, tracking, text reading, and keypoint detection.
- The work was funded by the EPSRC Seebibyte programme grant.