Source-linked AI summary
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
Khurram Soomro, Amir Roshan Zamir, Mubarak Shah
TL;DR
Existing action-recognition datasets have few classes or controlled recording conditions, limiting coverage of real-world actions. This paper introduces UCF101, a large dataset of unconstrained web videos spanning 101 action classes, and reports a standard bag-of-words baseline with 44.5% overall accuracy.
Problem
Existing action-recognition datasets typically cover few action classes and use unrealistically controlled recording environments.
Method
The paper constructs UCF101 from unconstrained web videos, organizing 101 action classes across five action types.
Results
44.5% overall accuracy was achieved by a standard bag-of-words action-recognition baseline on UCF101.
Takeaways & Limitations
UCF101 is presented as a challenging action-recognition dataset because it combines many classes with unconstrained videos and over 13k clips.
Takeaways & Limitations
Reported tests use a recommended 25-fold cross-validation setup using all videos in the dataset.
Abstract
from arXiv · showhide
We introduce UCF101 which is currently the largest dataset of human actions. It consists of 101 action classes, over 13k clips and 27 hours of video data. The database consists of realistic user uploaded videos containing camera motion and cluttered background. Additionally, we provide baseline action recognition results on this new dataset using standard bag of words approach with overall performance of 44.5%. To the best of our knowledge, UCF101 is currently the most challenging dataset of actions due to its large number of classes, large number of clips and also unconstrained nature of such clips.
1. Introduction
UCF101 addresses the limited class coverage and controlled recording conditions of prior action-recognition datasets with unconstrained web videos. These videos include camera motion, varied lighting, partial occlusion, and low-quality frames.
- Motivation: Existing action-recognition datasets typically contain few action classes and are recorded in unrealistically controlled environments.Examples include KTH, Weizmann, UCF Sports, and IXMAS, with 6, 9, 9, and 11 classes respectively.
- Dataset characteristics: UCF101 is composed of web videos recorded in unconstrained environments.
- Dataset characteristics: UCF101 videos typically include camera motion, various lighting conditions, partial occlusion, and low-quality frames.
2. Dataset Details
UCF101 contains 101 action classes organized into five predefined types, with clips grouped by shared backgrounds or actors. The dataset documents class-specific clip distributions, standardized video formatting, and a folder-and-filename convention identifying each action class, group, and clip.
- Action Classes: 101 action classes are divided into five types: Human-Object Interaction, Body-Motion Only, Human-Human Interaction, Playing Musical Instruments, and Sports.Figure 2 shows one sample frame for each action class, with border colors indicating the predefined action type.
- Action Classes: 51 new action classes are introduced in UCF101 as an extension of the 50 classes included in UCF50.The new classes include actions such as Apply Eye Makeup, Archery, Basketball Dunk, Playing Cello, Surfing, and Writing On Board.
- Clip Groups: 25 groups divide the clips within each action class, with 4-7 clips per group sharing features such as background or actors.Figures 3 and 4 describe the number, duration distribution, average length, and total duration of clips across action classes.
- Video Format: 25 FPS and 320 × 240 are the fixed frame rate and resolution for all clips, which are stored as DivX-compressed .avi files.The videos were downloaded from YouTube, irrelevant clips were manually removed, and audio was preserved for clips from the new 51 actions.
- Naming Convention: 101 folders organize the dataset by action class, while each filename encodes the action class label, group, and clip number.For example, v ApplyEyeMakeup g03 c04.avi identifies clip 4 of group 3 for the ApplyEyeMakeup class.
3. Experimental Results
The experiments establish a bag-of-words baseline for UCF101 using spatiotemporal interest-point descriptors and a nonlinear multiclass SVM under leave-one-group-out 25-fold cross-validation. The method achieves 44.5% overall accuracy, with Sports actions performing best among the predefined action types.
- Experimental setup: The baseline used Harris3D corners, 162 dimensional HOG/HOF descriptors, and a k=4000 codebook clustered from 100,000 space-time interest points.Descriptors were assigned to their closest codebook centers to form histogram representations.
- Overall baseline: 44.5% overall accuracy was achieved using a nonlinear multiclass SVM with histogram intersection kernel across 101 action classes.The classifier used histogram vectors derived from query and training videos.
- Experimental setup: The evaluation followed a leave-one-group-out 25-fold cross validation scenario, training on histogram vectors from the training folds.The authors recommend using all dataset videos under this setup for consistency in reported UCF101 tests.
- Results: Sports actions achieved the highest predefined-type accuracy at 50.54%, followed by Human-Human Interaction at 44.14% and Human-Object Interaction at 38.52%.The reported accuracies were also 37.42% for Playing Musical Instrument and 36.26% for Body-Motion Only.
4. Related Datasets
UCF101 is the latest in UCF’s chronological series of four action datasets, each incorporating its precursor. Compared with existing action-recognition datasets, UCF101 is notably larger, with minor modifications to its UCF50-video subset.
- UCF Sports, UCF11, UCF50, and UCF101 form UCF’s four chronologically compiled action datasets, with each dataset including its precursor.
- The UCF101 subset containing UCF50 videos fixes the number of groups at 25 for every action, with up to 7 clips per group.
- UCF101 is remarkably larger than the other existing action-recognition datasets listed in Table 2.Table 2 provides detailed characteristics for the existing datasets.
5. Conclusion
The paper introduces UCF101 as a large, challenging action-recognition dataset built from unconstrained YouTube videos and reports baseline results using a standard bag-of-words method.
- Dataset contribution: UCF101 contains 101 action classes and over 13k clips, making it substantially larger than existing datasets.The dataset is described as outstandingly larger than other datasets.
- Dataset contribution: The dataset comprises unconstrained YouTube videos with poor lighting, cluttered backgrounds, and severe camera motion.These characteristics create challenging conditions for action recognition.
- Baseline evaluation: The paper provides baseline action-recognition results on UCF101 using a standard bag-of-words method.