Source-linked AI summary
A Short Note on the Kinetics-700-2020 Human Action Dataset
Lucas Smaira, João Carreira, Eric Noland, Ellen Clancy, Amy Wu, Andrew Zisserman
TL;DR
The paper addresses the need to maintain a large, diverse human-action video dataset as YouTube clips disappear. It replenishes rare classes in Kinetics-700-2020, analyzes the resulting dataset, and benchmarks RGB I3D training with varying data amounts. The release has at least 700 clips per class, is more balanced by clip count, and shows improved I3D classification performance with more training clips.
Problem
Maintaining large-scale YouTube action datasets requires addressing video disappearance while preserving diverse clips for human-action modeling.
Method
The paper replenishes the 123 rarest classes, expands multilingual query searches, analyzes geography, and benchmarks an RGB-only I3D model with varying examples per class.
Results
Every class has at least 700 clips, and I3D top-1 and top-5 accuracy improve steadily with more examples per class.
Takeaways & Limitations
Kinetics-700-2020 provides a more balanced dataset for human-action modeling and demonstrates the value of additional training clips for I3D classification.
Abstract
from arXiv · showhide
We describe the 2020 edition of the DeepMind Kinetics human action dataset, which replenishes and extends the Kinetics-700 dataset. In this new version, there are at least 700 video clips from different YouTube videos for each of the 700 classes. This paper details the changes introduced for this new release of the dataset and includes a comprehensive set of statistics as well as baseline results using the I3D network.
1. Introduction
Kinetics is a family of large-scale YouTube video datasets for modeling human actions, but video disappearance requires periodic replenishment. Kinetics-700-2020 replenishes clips without increasing the number of action classes.
- Kinetics datasets contain diverse human-action video clips for training and exploring neural network architectures.
- The three editions contain 400, 600, and 700 human-action classes, respectively, with 10-second clips from distinct YouTube videos per class.
- Video disappearance, including user deletion or privatization, is a central challenge in building realistic YouTube datasets.
- Kinetics-700-2020 replenishes clips for each class while keeping the class count unchanged.
- The dataset release provides URLs and temporal intervals for Kinetics videos, plus additional annotations for AVA-Kinetics and Countix.
2. Data Collection Process
The collection process targets the 123 rarest classes, expands multilingual search queries to improve yield, removes duplicates, and analyzes geographic distribution. Geographic diversity increased slightly, but North America still accounted for more than half of uploads.
- The replenishment targets 123 classes with fewer than 700 clips and tops each up to at least 700 clips.
- Expanded query wording across English, French, Spanish, and Portuguese improved candidate yield and content relevance for rare classes.
- Duplicate clips arise from repeated uploads or compilations and are filtered through clustering and final class verification.
- Geographic analysis assigns video locations by upload continent using information available for around 90% of videos.
- Geographic diversity increased slightly, especially in Latin America, while more than half of videos were still uploaded from North America.
3. Benchmark Performance
The benchmark trains an RGB-only I3D model from scratch with different numbers of examples per class and evaluates it on validation and test sets. Top-1 and top-5 accuracy improve steadily as training examples increase.
- The RGB-only I3D baseline is trained from scratch using 100, 200, 300, 400, 500, 600, or all available examples per class.
- Top-1 and top-5 accuracy improve steadily with more examples per class.
- I3D has around 12M parameters, providing the benchmark with a relatively small model baseline.
- Evaluation uses eight linearly spaced segments per clip for the RGB I3D model.
- Training uses a 32-device TPU pod and 32-frame clips, while testing uses eight clips of 32 frames.
4. Conclusion
Kinetics-700-2020 is more balanced by clip count, with every class containing at least 700 examples, and additional training clips improve I3D classification performance. The paper also situates Kinetics as both a development resource and a possible downstream task.
- Every Kinetics-700-2020 class has at least 700 examples, making the dataset more balanced than current Kinetics-700 by clip count.
- The benchmark demonstrates improved I3D classification performance when more training clips are available.
- Table 3 reports geographic data distribution per continent.
- Table 4 reports validation and test top-1/top-5 accuracy for different training-example counts using eight regularly spaced clips.
- Kinetics was introduced to support spatio-temporal architecture development and model pre-training for downstream tasks.
A. Yield success rate per class
The ranked list reports, for each replenished class, candidate positivity and post-filter publication probabilities. These yields vary substantially across classes, with the listed examples spanning high and low success rates.
- Yield definitions: Each class is ranked by two probabilities: candidate clips voted positive by at least three annotators, and examples published after deduplication and final filtering.The first probability measures positive voting; the second measures final publication after dataset filtering.
- Higher-yield classes: 38.68% positive voting and 37.93% publication are reported for stacking dice.The publication probability is close to the positive-voting probability for this class.
- Lower-yield classes: 2.82% positive voting and 2.11% publication are reported for coughing.This is the lowest pair among the supplied class entries.