Source-linked AI summary
RED: Reinforced Encoder-Decoder Networks for Action Anticipation
Jiyang Gao, Zhenheng Yang, Ram Nevatia
TL;DR
Action anticipation must predict actions before they occur, but prior representation-based methods use a single past frame and a fixed future time. RED encodes multiple history representations, predicts a future sequence, and adds reinforcement-based sequence supervision; it achieves state-of-the-art performance on the evaluated datasets.
Problem
Action anticipation requires predicting actions before they happen, while prior representation-based anticipation uses a single past frame and only a fixed future time.
Method
RED encodes multiple history representations, predicts a sequence of future representations for classification, and uses reinforcement learning to provide sequence-level supervision favoring early correct predictions.
Results
RED achieves state-of-the-art performance on TVSeries, THUMOS-14, and TV-Human-Interaction for action anticipation and online action detection.
Takeaways & Limitations
The experiments support the effectiveness of RED's reinforcement module and encoder-decoder network for action anticipation.
Abstract
from arXiv · showhide
Action anticipation aims to detect an action before it happens. Many real world applications in robotics and surveillance are related to this predictive capability. Current methods address this problem by first anticipating visual representations of future frames and then categorizing the anticipated representations to actions. However, anticipation is based on a single past frame's representation, which ignores the history trend. Besides, it can only anticipate a fixed future time. We propose a Reinforced Encoder-Decoder (RED) network for action anticipation. RED takes multiple history representations as input and learns to anticipate a sequence of future representations. One salient aspect of RED is that a reinforcement module is adopted to provide sequence-level supervision; the reward function is designed to encourage the system to make correct predictions as early as possible. We test RED on TVSeries, THUMOS-14 and TV-Human-Interaction datasets for action anticipation and achieve state-of-the-art performance on all datasets.
1 Introduction
Action anticipation detects actions before they happen, supporting applications such as surveillance intervention and robot planning. RED addresses limits in prior representation-based anticipation by using history information to predict continuous future representations.
- Motivation: Action anticipation detects an action before it happens and supports predictive applications in surveillance and robotics.Surveillance systems can raise alarms before accidents, while robots can use anticipated human actions for planning and interaction.
- Motivation: Anticipation must capture historical and contextual information while handling the discriminative challenges of action detection and irrelevant background variation.
- Limitations of Prior Work: Prior deep methods anticipated future representations from a single past frame and predicted only one fixed future time, despite actions being better modeled across multiple frames.
- Proposed Approach: RED uses continuous history representations to anticipate a sequence of future representations, which are then classified into actions, with reinforcement learning providing sequence-level supervision.
2 Related Work
Related work spans early and online action detection, offline temporal localization, future action anticipation, and reinforcement learning for visual prediction and control.
- Early and Online Action Detection: Early and online action detection methods include structured-output SVMs, LSTM ranking models, and a TV-series dataset with temporal annotations for 30 action categories.
- Offline Action Detection: Offline action detection methods use proposal generation, temporal coordinate regression, and language-conditioned temporal localization to detect actions in complete videos.
- Action Anticipation: Action anticipation research has used hierarchical representations, stochastic context-sensitive grammars, intention inference, and CNNs trained on unlabeled video to predict future actions or representations.
- Reinforcement Learning in Vision: Vision-based reinforcement learning has applied REINFORCE to selecting observation locations, image regions, and sequence-level training objectives.
3 Reinforced Encoder-Decoder Network
RED encodes multiple historical visual representations to anticipate a sequence of future representations and classify future actions. Reinforcement learning adds sequence-level supervision that rewards correct predictions made earlier.
- RED combines video representation extraction, an encoder-decoder, action classification, and reinforcement modules for anticipation.
- Encoder-Decoder Network: The encoder reads a sequence of historical chunk representations, while the decoder predicts representations for the following anticipation steps.The input spans [t−Tenc,t), and the output spans [t,t+Tdec).
- Cross-entropy loss evaluates action classification, while squared loss trains future-representation regression.
- Classification Network: The anticipated representation sequence is processed by a classification network that outputs action-category distributions using two fully connected layers.
- Reinforcement Module: Cross-entropy alone misses sequence-level distinctions in which one anticipation sequence predicts the action correctly earlier than another.For ground truth 011111, 001110 is considered better than 000111 because it anticipates the action earlier.
- Reinforcement Module: The reinforcement module treats anticipation and classification as an agent and uses rewards whose cumulative objective favors correct predictions made earlier.Correct predictions at the transferring time receive the largest reward, and the reward decays with time.
4 Evaluation
RED is evaluated against progressively stronger anticipation baselines on three datasets, across fixed and varying anticipation times. Results show benefits from history encoding, sequence prediction, reinforcement supervision, and strong performance for online action detection.
- Experiment setup: The evaluation uses TVSeries, THUMOS-14, and TV-Human-Interaction, with anticipation times from 0.25s to 2.0s where applicable.TVSeries and THUMOS-14 support model training and testing; TV-Human-Interaction uses stage-1 models trained on the larger datasets.
- Comparison of action anticipation methods: At 1s, RED outperforms FC on both TVSeries and THUMOS-14, while comparisons with FC and EFC show that encoding multiple history representations improves anticipation.Table 1 reports TVSeries cAP and THUMOS-14 per-frame mAP with two-stream features.
- Comparison of action anticipation methods: ED anticipates continuous future representations more accurately than EFC, which predicts only one future representation.ED and RED use encoder-decoder prediction over multiple continuous future steps, whereas EFC uses a single anticipated representation.
- Comparison of action anticipation methods: Both RED-VGG and RED-TS outperform Vondrick et al. on TV-Human-Interaction at Ta = 1s, although stronger features may explain some of the gain.The authors also report using a much smaller THUMOS training subset and state that network design accounts for a significant part of the improvement.
- Varying anticipation time: Reinforcement consistently improves ED across anticipation steps on TVSeries and THUMOS-14 with both two-stream and VGG features.The reward function supplies sequence-level supervision that encourages correct predictions as early as possible.
- Comparison of online action detection: RED-VGG reaches 71.8 and RED-TS reaches 79.2 on TVSeries online action detection, exceeding the cited LSTM result of 64.1.The paper treats minimum anticipation time, Ta = 0.25s, as online action detection and reports that RED also outperforms prior methods on THUMOS-14.
5 Conclusion
RED uses multiple history representations to anticipate a sequence of future representations for action classification, with reinforcement-based sequence-level supervision encouraging correct predictions as early as possible.
- RED takes multiple history representations as input and anticipates a sequence of future representations for action categorization.
- The reinforcement module provides sequence-level supervision through a reward designed to encourage correct predictions as early as possible.
- RED is jointly optimized with cross-entropy loss, squared loss, and the reward function through a two-stage training process.