PaperlayerView exact revision on arXiv

Source-linked AI summary

Attention-based Convolutional Neural Network for Weakly Labeled Human Activities Recognition with Wearable Sensors

Kun Wang, Jun He, Lei Zhang

arXiv:1903.10909v2cs.LGstat.ML

TL;DR

Wearable-sensor activity data is costly to annotate precisely, while conventional HAR methods depend on strictly labeled datasets. The paper proposes an attention-based CNN for weakly labeled sequences, achieving improved recognition and locating labeled activities through compatibility density.

  • Problem

    Accurately segmenting activities in long wearable-sensor sequences is laborious, and conventional HAR methods require labeled datasets for ground-truth annotation.

  • Method

    An attention-based CNN recognizes activities from weakly labeled sequences, focuses on salient activity regions, and converts compatibility scores into compatibility density for localization.

  • Results

    The attention model achieves significant accuracy improvement on weakly labeled data compared with classical deep learning models.

  • Takeaways & Limitations

    The method can reduce time-consuming annotation and facilitate wearable-sensor data collection while locating labeled activities in long sequences.

Abstract

from arXiv · show

Unlike images or videos data which can be easily labeled by human being, sensor data annotation is a time-consuming process. However, traditional methods of human activity recognition require a large amount of such strictly labeled data for training classifiers. In this paper, we present an attention-based convolutional neural network for human recognition from weakly labeled data. The proposed attention model can focus on labeled activity among a long sequence of sensor data, and while filter out a large amount of background noise signals. In experiment on the weakly labeled dataset, we show that our attention model outperforms classical deep learning methods in accuracy. Besides, we determine the specific locations of the labeled activity in a long sequence of weakly labeled data by converting the compatibility score which is generated from attention model to compatibility density. Our method greatly facilitates the process of sensor data annotation, and makes data collection more easy.

I. INTRODUCTION

The paper addresses activity recognition from weakly labeled wearable-sensor sequences, where activities are identified without precise temporal segmentation. It proposes attention-based CNN methods to recognize activities and locate them in long sequences.

  • Wearable sensors produce time-series data, but accurately segmenting activities from long sequences is laborious.
  • Weakly labeled data records whether an activity occurred in a sequence, without specifying its exact time.
  • Traditional HAR methods use supervised learning, while deep CNNs still require labeled datasets for ground-truth annotation.
  • The proposed attention-based CNN recognizes activities from weakly labeled sensor data and determines the locations of labeled activities.

II. RELATED WORKS

Earlier wearable-sensor HAR methods relied on handcrafted features and supervised learning, motivating deep models that learn representations directly. The paper places its attention-based approach within this shift toward weakly supervised recognition and localization.

  • Handcrafted-feature methods extracted statistics from accelerometer data and supplied them to conventional classifiers.
  • Feature engineering performed poorly on similar activities and made feature selection and extraction difficult.
  • Deep learning methods use convolutional networks to learn features and classifiers simultaneously, avoiding manually designed features.
  • The proposed model combines CNN pipelines with attention to identify salient activity areas and suppress irrelevant information in weakly labeled sequences.

A. Attention Submodule

The attention submodule compares local convolutional features with a global descriptor, normalizes compatibility scores, and uses them to form weighted feature summaries for classification.

  • At each convolutional layer, local feature vectors are combined with the global feature vector through a compatibility function.
  • Compatibility can be computed by combining local and global features with a learned weight vector or by their dot product.
  • The resulting compatibility scores are normalized with softmax or tanh to produce attention weights.
  • Normalized scores produce one vector per layer through element-wise weighted averaging of local feature vectors.
  • The resulting layer vectors replace the original global descriptor, are concatenated, and feed the linear classification step.

B. Fundamental CNN

The fundamental model is a deep CNN composed of convolutional, pooling, and fully connected layers, with a specified sequence ending in softmax classification. ReLU transforms each layer’s output, while early pooling choices preserve higher-resolution local feature maps for attention.

  • B. Fundamental CNN: The CNN comprises convolutional, pooling, and fully connected layers, followed by a softmax classifier.Its shorthand architecture is C(32)-C(64)-C(128)-P-C(128)-P-C(128)-P-FC(128)-softmax.
  • B. Fundamental CNN: ReLU activation transforms the output of each layer.
  • B. Fundamental CNN: Removing pooling after the first two convolutional layers preserves higher-resolution local feature maps for attention estimation.
  • B. Fundamental CNN: Local and global features are deliberately assigned the same dimensionality, 128, to avoid extra mapping cost.

C. Location Function

The location function converts local–global compatibility scores into a compatibility density over spatial locations. Density peaks identify likely labeled-activity points, and surrounding windows define the corresponding activity areas.

  • C. Location Function: The attention method places filters early in the CNN to learn mappings compatible with the global descriptor produced by the original architecture.
  • C. Location Function: Local feature vectors are matched with global feature vectors through a compatibility function, producing one compatibility score for each spatial location.
  • C. Location Function: A varied-width sliding window sums compatibility scores around each spatial location to compute a location score.
  • C. Location Function: The resulting compatibility density assigns peaks to locations most likely to contain the labeled activity.The density is represented as D = {d1, d2, ..., dn}.
  • C. Location Function: For a density peak at location i, the activity area is defined as the surrounding section [i − w/2, i + w/2].

IV. EXPERIMENTS

The experiments evaluate whether the attention model supports conventional activity recognition and weakly supervised recognition. They use both a simple CNN and the DeepConvLSTM model as experimental models.

  • IV. EXPERIMENTS: The experiments have three parts: validating traditional recognition, evaluating weakly supervised recognition, and using the weakly labeled sensor dataset.
  • IV. EXPERIMENTS: The conventional recognition experiment uses the public UCI HAR Dataset.
  • IV. EXPERIMENTS: Both experiment types use a simple CNN and the DeepConvLSTM model proposed in.
  • IV. EXPERIMENTS: Networks with attention at the last, last two, and last three levels are named Net-att, Net-att2, and Net-att3, respectively.

A. Dataset

The experiments use a manually labeled smartphone-sensor dataset and a weakly labeled dataset collected from participants. The weakly labeled data contain long sequences in which walking is background activity for four target activities.

  • A. Dataset: The UCI HAR Dataset contains 12 daily activities recorded by a Samsung Galaxy smartphone at 50 Hz.
  • A. Dataset: The UCI experiment uses six activities: three static and three dynamic activities.
  • A. Dataset: The weakly labeled dataset contains walking, jogging, jumping, going upstairs, and going downstairs.
  • A. Dataset: Walking serves as the background activity for the other four activity types in the weakly labeled dataset.
  • A. Dataset: The weakly labeled data comprise 76,157 sequences sampled in fixed windows of 40.96 seconds, split into 70% training, 10% validation, and 20% testing.

B. Experiments on UCI HAR Dataset

On the UCI HAR dataset, the attention model was evaluated against the fundamental CNN and two classical deep learning models for classification accuracy and prediction efficiency.

  • All models perform comparably well in classification accuracy on the UCI HAR dataset.This indicates that the attention model also performs well on traditional supervised training tasks.
  • The attention approach predicts more than 5000 sequences per second on the UCI HAR dataset.Its efficiency is slower than CNN but ahead of DeepConvLSTM.

C. Experiments on Weakly Labeled Dataset

On weakly labeled data, the attention model improves recognition over several CNN baselines, while its compatibility scores can become concentrated at only a few points.

  • 3.65%, 4.21% and 3.79% improvements are achieved by Net-att3-pc-tanh over the fundamental CNN, classical CNN and DeepConvLSTM, respectively.These improvements are reported for weakly labeled data recognition.
  • The attention mechanism focuses on labeled-activity areas while ignoring background activity information.This behavior is illustrated by the weakly labeled sensor sample and generated compatibility score.
  • The dot-product and softmax combination on long weakly labeled sequences produces an abnormal compatibility score with only a few excessively high-value points.The resulting feature vectors lack crucial information outside those valid points.
  • Att-pc-tanh addresses this issue by generating compatibility scores with lower disparity and independently normalizing them instead of jointly applying softmax.

D. Location Experiments

The location experiment evaluates whether compatibility representations can identify labeled-activity positions in weakly labeled sensor sequences.

  • Converting compatibility scores into compatibility density provides clearer labeled-activity locations than using compatibility-score peaks.Compatibility-density peaks concentrate where the labeled activity occurs more intensively.

V. CONCLUSION

The paper concludes that attention-based CNNs support weakly labeled activity recognition and activity localization from wearable sensors. The approach improves weakly labeled recognition and is intended to simplify annotation and data collection.

  • The proposed attention model achieves significant accuracy improvement on the weakly labeled dataset compared with classical deep learning models.On the traditional UCI HAR dataset, its performance is comparable with classical human activity recognition methods.
  • Compatibility density is used to determine the specific location of labeled activity in long weakly labeled data sequences.The location function is presented as a basis for collecting specific segments in future work.
  • The method can use weakly labeled data to avoid time-consuming traditional annotation without unduly increasing computational cost compared with the classical CNN model.
Loading 1903.10909v2…