Sequence Recognition in Bharatnatyam dance

• Adavus comprise 15 classes and 58 variations, each built from ordered postures and movements.Source: §1 · p. 2
• Key postures are stationary holds, while motions connect one key posture to the next; this work addresses visual, not audio, information.The visual sequence alternates between key postures and motions, which may repeat.Source: §1 · p. 2
• The proposed approach classifies key postures and motions, then uses their sequence to recognize the Adavu.It uses CNN and SVM approaches for key-posture recognition and SVM for motion recognition.Source: §1 · p. 2

2 Related Work

• The survey separates Adavu recognition into key-posture, motion, and sequence-recognition subproblems.Source: §2 · p. 2

2.1 Recognition of Key Postures

• Prior Bharatanatyam key-posture work used SVM with bone-angle or RGB-HOG features; combined the model with Edit Distance to recognize Natta and Mettu at 99.38% accuracy, without considering motion.[21] reported 99.56% with ResNet on 23 classes using RGB data.Source: §2.1 · p. 3

2.2 Recognition of Motion

• Motion-recognition studies explored features and classifiers including STIP, HOOF, joint velocities, and MHI; this paper uses HOG features from MHI.Reported results include 92.6% with Treebagger and 85% with DTW or kNN.Source: §2.2 · pp. 3–4

2.3 Recognition of Adavus

• Earlier Adavu work often omitted motions and covered few variations; this paper includes both postures and motions and considers 13 Adavus and 51 subclasses.The paper also analyzes time complexity and designs a CNN described as taking less time.Source: §2.3 · p. 4

3 Data Set

• After erroneous data were discarded, the dataset contained 13 Adavus and 51 variations, with 182 key-posture classes considered for recognition.Although 184 key postures and 334 motion classes were identified, motions with 12 or fewer samples and two key postures were excluded.Source: §3 · p. 5

3.1 Data Set for Adavu

• Candidate sequences are formed from annotated posture and motion frames, with the least-sampled key posture limiting the number available for an Adavu.For Paikkal-1, the example gives 11 valid subsequences, limited by the 11 frames for P134; the initial ready-to-dance posture is discarded.Source: §3.1 · pp. 5–6

• The workflow converts video frames to grayscale and uses a depth-derived binary mask to remove the background.Source: §4 · p. 6, Figure 2 · p. 7
• After preprocessing, CNN classifies key postures from background-subtracted frames, while SVM classifies motions using HOG-MHI features.Source: §4 · p. 6
• The recognized key-posture and motion sequences are matched to Adavus using Edit Distance.Source: §4 · p. 7, Figure 2 · p. 7

5 Feature Extraction

• Frames are converted to grayscale and background-subtracted before feature extraction; annotations distinguish KP frames from motion frames.Source: §5 · p. 7

5.1 Feature Extraction for Motion

• Motion History Images are built by accumulating thresholded differences between consecutive frame contours, then resized and converted to HOG features for motion recognition.The MHI starts as a 480 × 640 zero matrix; it is resized to 120 × 160, producing a 9,576-element HOG feature vector.Source: §5.1 · p. 7

5.2 Feature for Adavu

• Adavu recognition uses the KP and motion features in their ordered sequence, with classifiers predicting the constituent classes.The resulting sequence is used to recognize the Adavu.Source: §5.2 · pp. 7–8

6 KP Recognition

• The proposed CNN uses two convolutional and three fully connected layers, with dropout and a 15-epoch limit to address overfitting.The final fully connected layer outputs 182 feature arrays, converted to class scores by softmax.Source: §6 · p. 8
• The proposed CNN was reported as the best performer on the dataset among the proposed model, AlexNet, VGG16, and ResNet.The comparison also reports execution time for training and testing; the authors note that deeper models take longer to execute.Source: §6 · p. 9, Table 3 · p. 9

6.1 KP Recognition: Result Discussion

• Common confusions include P8 with P3 and P94 with P95; Kinect hand-tracking failures can also make P28 resemble P3.P8 may resemble P3 when a leg stretch is insignificant; P94 and P95 differ in whether the foot is grounded or on the toes.Source: §6.1 · p. 9, Figure 4 · p. 10

6 KP Recognition

• 99.46% recognition accuracy on 8,943 test samples; average precision was 99.95%, recall 99.15%, and F1 score 99.51%.The result is reported for the proposed CNN; 158 KPs achieved 100% accuracy, and the rest scored above 95% except for a few.Source: §6 · p. 8, Table 4 · p. 9

7 Motion Recognition

• SVM classifies motions from HOG features extracted from MHIs; the approach addresses motion complexity, occlusion, and variable frame counts.Each MHI is converted to a 9576-element HOG feature vector.Source: §7 · p. 10

7.1 Motion Recognition– Result Discussion

• 22 motion classes reach 100% accuracy; the reported classes have 0, 1, and 2–3 mispredictions in counts of 22, 20, and 12, respectively.Most classes have only one or two samples misclassified.Source: §7.1 · p. 10
• Similar motions can be confused when their MHI patterns are close, and lower-body occlusion can obscure leg movement.The paper also notes that minimal motions may be difficult to distinguish when they last only a few frames.Source: §7.1 · pp. 10–11
• The SVM achieves 95.58% recognition accuracy across 12 motion classes; the passage also says its accuracy is higher than that of the SVM using at least 12 samples.Source: §7.1 · p. 11
• 84.86% is reported for the trained SVM covering 54 motion classes with at least 12 samples, and this model is used for Adavu recognition.The section also reports 271 of 320 samples correctly classified, or 84.08%.Source: §7 · p. 10, §7.1 · p. 11

8 Adavu Recognition

• The recognizer matches an unknown KP-and-motion sequence against a dictionary of labeled Adavu sequences, selecting the sequence with the least Edit Distance score.The dictionary records each Adavu's ground-truth label and its KP and motion IDs.Source: §8 · p. 11, §8.1 · p. 12

8.1 Result Discussion

• Approach 2 covers all Adavus by accounting for motions excluded from motion recognition because of insufficient samples; it evaluates 1645 sequences.Only 54 of 334 motion classes are included in motion recognition.Source: §8.1 · p. 12
• 77.51% accuracy is reported for the KP-only approach, with 1275 of 1645 sequences correct; errors include Tatta-2 through Tatta-8 being classified as Tatta-1.Those Tatta variations share the same KP sequence, so the cross-class errors occur within Tatta.Source: §8.1 · p. 12
• Except for Tatta, the other Adavus reach 100% accuracy in both approaches, even when some KPs or motions are mispredicted.The paper attributes this to the lack of densely repeated KPs or motions in those sequences.Source: §8.1 · p. 12
• 98.66% accuracy is reported for the KP-and-motion approach, with 1623 of 1645 Adavu sequences predicted correctly.Table 6 presents the Adavu recognition result comparison.Source: §8.1 · p. 12, Table 6 · p. 13

8.2 Performance Analysis Based on Time

• Approach 1 is approximately 76 times faster than, Approach 2 approximately 40 times faster, and Approach 1 approximately twice as fast as Approach 2.The comparison uses total prediction time for all sequences and is reported for the stated Windows 10 system.Source: §8.2 · p. 13

9 Conclusion

• 98.66% recognition accuracy outperforms previous approaches when considering the amount of data.The paper also reports good accuracy on more sequences.Source: §9 · p. 14
• The approach recognizes Adavu sequences using both KPs and motions, unlike earlier approaches that considered only KPs.Source: §9 · p. 14
• CNN-based KP recognition performs better than SVM-based recognition, as reported in earlier works.Source: §9 · p. 14
• Prediction-time analysis found the proposed approach more efficient than earlier similar work.Source: §9 · p. 14
• The CNN model's robustness on other datasets remains untested; the authors propose trying RNNs for the KP-and-MHI time-series sequence.Source: §9 · p. 14