Seeing Seizures: A Large-Scale Empirical Study of Video and EEG Deep Learning for Automated Rodent Seizure Detection and Severity Grading

Motivation and gap

• Severity labels reflect observable motor behavior on a soman-specific scale covering Stages 2–5, from mild tremors to full-body convulsions.Stage 1 is excluded because it is rare; standard Racine scores do not directly map to these chemically induced seizures.Source: §1 · p. 2
• Video seizure research has largely focused on detection, leaving the ability of modern video models to recover behaviorally defined severity unclear.The study compares video and EEG models on the same synchronized events.Source: §1 · p. 2

Study scope

• The target is retrospective analysis of recorded seizures, not pre-ictal forecasting, preventive intervention, or demonstrated reductions in annotation time.Predictions are framed as potential support for expert review and behavioral scoring.Source: §1 · p. 2
• The study compares ten video architectures and seven single-channel EEG models on matched events, examining label granularity, temporal sampling, capacity, and generalization.The cohort comprises 24,131 clips from 601 sessions across 20 rats.Source: §1 · p. 2

Synchronized video and EEG

• Paired video and EEG provide complementary views of physical behavior and electrical activity, illustrated by synchronized Stage 3 seizure streams.Figure 4 pairs four video frames with the corresponding single-channel EEG trace and highlighted active period.Source: Figure 4 · p. 3, §2 · p. 3

EEG-only and video-only systems

• EEG models provide a strong baseline for detection but do not capture the physical movements that define behavioral severity.Prior video work draws on action recognition and animal behavior analysis, while animal seizure-video studies have mostly addressed detection.Source: §2 · p. 3
• RodEpil reported around 97% F1 for video-only detection, but prior animal-video studies had not systematically tested multi-level seizure severity grading.The cited RodEpil study used 13,053 clips from 19 rats and a TimeSformer model [44].Source: §2 · p. 3

Synchronized video and EEG

• Animal studies using mixed video and EEG have generally addressed binary detection; this study instead evaluates each modality separately on the same events.Separate evaluation permits comparison from detection through severity grading before any future mixed-model work.Source: §2 · p. 3

3.1. Detection and severity grading

• Labels distinguish non-seizure clips from soman-specific behavioral Stages 2–5, with Stage 1 excluded for rarity and lack of reliable visible motor signs.The scale should not be conflated with the classical Racine scale.Source: §3.1 · p. 3
• Three nested tasks map the same labels to binary detection, three-class grouped severity, or five classes retaining each represented seizure stage.The three-class task groups Stages 2–3 and 4–5; video and EEG classifiers are trained and evaluated independently on the same events.Source: §3.1 · p. 4

3.2. Conditional ordinal severity formulation

• The proposed conditional ordinal extension first separates seizure occurrence from severity, but all reported experiments use flat classifiers.It would encode severity with cumulative probabilities constrained to be monotonic; this formulation remains unevaluated.Source: §3.2 · p. 4

Dataset and annotation

• The cohort contains 24,131 paired clips across 601 sessions from 20 soman-exposed rats recorded in home cages with synchronized video and EEG.Natural seizures were tracked 10–18 weeks after exposure; raw EEG was sampled at 1 kHz and downsampled to 125 Hz.Source: §4 · p. 4

Model evaluation context

• Table 2 compares ten video models on identical splits under a 12-epoch training limit, with detection scores varying by only 0.03.Scores are means ± standard deviations over five test runs, except SlowFast detection, which uses n=4.Source: Table 2 · p. 5

Dataset and annotation

• Researchers reviewed synchronized streams to mark event boundaries and assign behavioral severity labels, so reference labels are not modality-independent.Formal inter-rater agreement was not reported, leaving reproducibility across independent raters unquantified.Source: §4 · pp. 4–5

Labels and class distribution

• The dataset is heavily imbalanced across severity stages and rats; Stage 1 was dropped after only six clips were captured.The working data include 1,435 Stage 2, 9,256 Stage 3, 1,050 Stage 4, and 193 Stage 5 clips, alongside 12,197 normal clips.Source: §4 · p. 5

5.1. Video capacity and input requirements

• Across video architectures, binary-detection macro-F1 ranges from 0.945 to 0.974; X3D-M (4M parameters) scores 0.969 versus 0.971 for R(2+1)D-18 (33M).Their five-class macro-F1 scores differ more substantially, at 0.586 and 0.667, respectively.Source: §5.1 · p. 5
• Using 16 frames raises video macro-F1 from 0.569, 0.417, and 0.314 to 0.970, 0.780, and 0.662 across two-, three-, and five-level tasks.Performance peaks around 32 frames; 64 frames provides no detection gain and lowers five-class F1.Source: Table 3 · p. 5, §5.1 · p. 6

5.2. Cross-modal comparison and generalization

• Video and EEG macro-F1 favor different modalities by task: GRU scores 0.974 versus video’s 0.968 for detection, while video scores 0.657 versus GRU’s 0.560 for five-class grading.These are session-exclusive reference-model results; the comparison is conditional on the evaluated architectures, inputs, training procedures, and annotation protocol.Source: §5 · p. 5, §5.2 · p. 6
• Subject-exclusive five-class macro-F1 falls to 0.475 for video and 0.404 for EEG, compared with session-exclusive values of 0.657 and 0.560.The video lead narrows from 0.097 to 0.071; Figure 6 displays per-fold detection macro-F1 of 0.948 ± 0.016 for video and 0.849 ± 0.055 for EEG.Source: §5.2 · p. 7, Table 5 · p. 7, Figure 6 · p. 7

5.3. Severe class imbalance

• Macro-F1 declines as label granularity increases, and video F1 for Stages 4 and 5 is 0.36 and 0.42 versus EEG F1 of 0.10 and 0.03.Stage 5 contributes only 193 clips; weighted cross-entropy was used, but no unweighted ablation isolated its effect.Source: §5.2 · p. 7, §5.3 · pp. 7–8, Figure 7 · p. 7

5.4. Discussion

• The modality comparison does not establish video’s general superiority over multichannel EEG, and accuracy alone does not establish reduced cost, improved welfare, or real-time suitability.Labels came from integrated video and EEG review, so the effects of EEG-assisted verification and visually defined severity cannot be separated.Source: §5.4 · p. 8

6. Limitations and Future Work

• The cohort contains 20 rats from one laboratory and one soman-induced epilepsy model, limiting conclusions about other animals, recording environments, and seizure models.Five-class video macro-F1 decreases from 0.657 under session-exclusive evaluation to 0.475 under subject-exclusive evaluation, and Stages 4 and 5 remain difficult.Source: §6 · p. 8
• Annotations are not modality-independent, formal inter-rater agreement is unreported, and EEG evaluation uses one channel; neither multichannel EEG nor fusion was evaluated.RodEpil provides binary-detection evidence, not external validation of severity grading.Source: §6 · p. 8
• Extracted-clip classification does not establish continuous-monitoring performance, which requires separate evaluation of event boundaries, false alarms per hour, latency, resources, and annotation-workload reduction.The conditional ordinal formulation also remains unevaluated, and imbalance treatments were not compared against resampling or focal loss.Source: §6 · p. 8

7. Conclusion

• GRU scores 0.974 versus R(2+1)D-18’s 0.968 for detection macro-F1, while video scores 0.657 versus EEG’s 0.560 for five-class behavioral grading.The study characterizes behavior-defined severity labels within the evaluated protocol; it does not demonstrate forecasting, fusion benefits, or readiness for continuous monitoring.Source: §7 · p. 8

A.1. Scope and Taxonomy

• Paired animal video-EEG remains the least explored area, particularly for severity-aware grading and open, shared benchmarks.The survey organizes prior work by data type and test subject.Source: §A.1 · p. 12, A Survey of Video and Video-EEG… · p. 12

A.2. EEG-Only Automated Detection

• EEG-only seizure analysis spans hand-crafted features and deep learning; reported rodent examples include Bergstrom et al.’s 99% accuracy and 91% precision and Kotloski’s 95.6% sensitivity.Recurrent models in a chemically diverse rat cohort achieved GRU recall above 90% for most animals.Source: §A.2 · p. 12

A.3. Video-Only Seizure Analysis

• Video seizure analysis draws on action-recognition backbones and computational ethology, including tools for raw-video behavior classification, pose patterns, and animal tracking.The survey also covers clinical and animal video systems, including seizure screening and severity-related behavior tracking.Source: §A.3.1 · pp. 12–13, §A.3.2 · p. 13

A.3.1. Video-Understanding Backbones

• R(2+1)D factorizes spatial and temporal convolutions, while SlowFast separates slow appearance processing from a fast motion pathway.The surveyed video backbones also include multiscale attention, global attention, and self-supervised pretraining approaches.Source: §A.3.1 · p. 13, §B.1 · pp. 14–15, Table 9 · p. 15

A.3.2. Human and Animal Seizure Video

• The study directly compares video and synchronized EEG on the same events, addressing the limited systematic testing of video-based multi-level severity grading.The appendix also surveys animal video systems and paired video-EEG work, which has largely focused on binary detection.Source: §A.3.2 · p. 13, §A.4 · p. 13, Table 8 · p. 14

A.4. Multimodal Video-EEG Fusion

• Paired video-EEG fusion is rare in animal studies, and the reviewed animal fusion systems focus on binary seizure detection rather than behavioral severity grading.The study evaluates each modality independently as a baseline before future combined systems.Source: §A.4 · p. 13

A.5. The Data Bottleneck

• Matched, expertly labeled video-EEG data are scarce, while animal studies face logistical challenges including invasive sensors, variable seizure rates, and differing camera setups.The appendix identifies self-supervised pretraining and transfer learning as strategies that can reduce manual labeling needs, without replacing paired benchmarks.Source: §A.5 · p. 14

B. Model Architectures and Training Details

• The study evaluates ten video backbones spanning 4M to 121M parameters; their detection macro-F1 scores cluster within a 0.03 margin, and parameter count alone does not explain performance.Video models use Kinetics-400 pretraining; EEG comparisons include recurrent, convolutional, attention-based, and feature-driven models, with TCN strongest by macro-F1 in the reported tests.Source: §B.1 · pp. 14–15, §B.2 · pp. 15–16

B.4. Metric definitions

• The confusion matrix C records actual category i predicted as j in Cij, and the defined counts support calculation of TP, FP, FN, and TN.For each category, precision, recall, and F1 are calculated; macro-F1 is not computed by combining macro-precision and macro-recall, and RodEpil reports positive-class scores to match its original publication [44].Source: §B.4 · pp. 16–17
• Macro-F1 averages category-specific F1 scores evenly, whereas MCC uses all clips directly and is therefore driven more heavily by frequent categories.Failing a rare category reduces macro-F1 by 1/K, while MCC may barely register the error; the metrics nevertheless indicate the same performance shift here.Source: §B.4 · pp. 16–17
• Across M = 5 runs, scores are calculated independently before averaging rather than pooling predictions into one confusion matrix.This aggregation applies to reported means across runs or folds.Source: §B.4 · p. 17
• For five-class grading, video scores 0.657 macro-F1 and 0.831 MCC versus baseline EEG GRU scores of 0.560 and 0.728, while both metrics show the video–EEG gap increasing across task complexity.The macro-F1 gap changes from −0.006 for detection to +0.030 for three-class grading and +0.097 for five-class grading; MCC follows the same upward trend.Source: §B.4 · p. 17

B.5. EEG preprocessing sensitivity

• Log-mean pooling outperformed simple averaging for long, varied seizure clips, leading to the final EEG setup of 6-second windows, 3-second stride, and 125 Hz sampling.These sensitivity checks preceded the finalized EEG dataset, so their scores were excluded from the final video–EEG comparisons; the authors note that optimizing preprocessing would require rerunning them on the final data.Source: §B.5 · pp. 17–18

C.1. Acquisition and synchronization

• The chronic-phase dataset combines synchronized video and EEG from soman-exposed rats, with simultaneous timestamps and cage-specific crops used to standardize views.Recordings captured recurring seizures roughly 10–18 weeks after exposure; EEG was downsampled from 1 kHz to 125 Hz.Source: §C.1 · p. 18

C.2. Annotation and event boundaries

• Reviewers verified seizure boundaries and behavioral grades against synchronized video and EEG, with clips padded by 10 seconds before and after each event.Labels are not independent across modalities: EEG helps flag events, while video determines severity; matched normal clips come from the same rat and session, reducing but not eliminating visual shortcuts.Source: §C.2 · p. 18

C.3. Behavioral severity scale

• The custom five-point severity scale is modified from, but distinct from, the Racine scale; the experiments also group stages differently from the clinical convulsive split.Stages 1–2 are clinically non-convulsive and 3–5 convulsive, while the experimental three-class system groups stages 2 and 3 together and separates them from stages 4 and 5.Source: §C.3 · pp. 18–19

C.4. Rare stages and long-tailed burden

• The 24,131-clip dataset contains 12,197 normal clips and 11,934 seizures, including 9,256 Stage 3 events but only 193 Stage 5 events.Stage 1 was excluded because only six clips were available; four rats contribute nearly half the data, and individual totals range from 11 to 3,091 clips. Training uses inverse-frequency weighting and macro-averaging, with rat-exclusive splits; weighting is presented as a baseline, not a perfect fix.Source: §C.4 · p. 19

C.5. Cross-subject appearance variation

• Cage, camera, lighting, and coat differences create cross-subject visual variation that subject-exclusive evaluation is designed to capture.Custom crops were applied to each cage’s video before clips were combined.Source: §C.5 · p. 19

C.6. Availability, de-identification, and ethics

• Raw video and EEG recordings cannot currently be shared, while the cohort breakdown, task definitions, and data-splitting formulas are being released in de-identified form.The analysis is retrospective, and the original animal procedures followed approved protocols and NIH ARRIVE guidelines.Source: §C.6 · pp. 19, 21

D.1. Failure cases

• The video model’s reported failure cases are subtle Stage 2 movements and animals with few seizure clips, often recorded under grainy infrared lighting.The authors connect these errors to natural biological variation and camera limits, motivating animal-by-animal reporting alongside aggregate results.Source: §D.1 · p. 21

D.2. Grad-CAM interpretability

• Grad-CAM attention falls on the animal in illustrated seizure clips across Stages 2, 3, and 5, while normal-period attention scatters into background noise.The authors treat these heatmaps as suggestive rather than definitive evidence of how predictions are made.Source: §D.2 · pp. 21–22, Figure 15 · p. 24

D.3. What the shortcut evidence does and does not establish

• Session-matched clips, cage-boundary crops, and a failed single-frame baseline address obvious visual shortcuts, but subtler cues remain possible.The authors say ruling out reliance on shortcuts would require targeted tests, such as erasing the animal or training on empty-cage backgrounds; current evidence is supportive but not causal proof.Source: §D.3 · p. 22

E.1. Data efficiency and pretraining

• With R(2+1)D, detection macro-F1 rose from approximately 0.79 with two animals to 0.957 with ten and 0.970 with twenty; Kinetics pretraining yielded 0.970 versus 0.952 from random initialization.The ten-to-twenty-animal gain was 0.013 with non-monotonic intermediate results; the pretraining comparison is specific to R(2+1)D under the reported protocol.Source: §E.1 · p. 22, Table 11 · p. 23

E.2. RodEpil encoding audit and re-encoding protocol

• Uniform re-encoding of RodEpil standardized resolution, frame rate, frame count, and codec, after raw files showed class-correlated frame counts and cropping.The sanitized R(2+1)D result was 0.979 subject-wise seizure F1, 0.999 AUROC, and 0.972 MCC, aligning with the dataset creators’ 0.97 baseline [44]; the pipeline cannot guarantee removal of every digital artifact.Source: §E.2 · pp. 22–23

E.3. All ten backbones on RodEpil

• On sanitized RodEpil, R(2+1)D scored 0.978 seizure-class F1, while model rankings differed from the native dataset and traditional convolutional models performed strongly.Several newer attention-based models struggled under the fixed training budget; the perfect SlowFast score was flagged for further investigation rather than treated as proof of behavioral recognition.Source: §E.3 · pp. 24–25, Table 12 · p. 25

E.4. Backbone, temporal, and spatial ablations in detail; E.5. Ranking quality: AUROC

• Across video backbones, performance spans only a 0.03 margin, and rankings can shift with training data; resolution increases also reduce performance in the reported ablation.The authors report that AUROC is secondary because it saturates: detection AUROC across video backbones is 0.984–0.995, and five-class AUROC is similar for EEG GRU and video R(2+1)D despite a macro-F1 gap.Source: §E.4 · p. 25, §E.5 · p. 26

E.4. Backbone, temporal, and spatial ablations in detail

• On video detection, 4 frames achieve 0.940 accuracy and 32 frames 0.972, while 64 frames provide no detection benefit and reduce five-class accuracy to 0.632.The 16-frame default gives up just 0.002 detection and 0.005 grading accuracy versus 32 frames while halving computational cost; increasing resolution beyond 112 px hurts performance.Source: §E.4 · p. 25, Table 13 · p. 25

E.6. Robustness of the cross-modal ordering

• On five-class grading, video exceeds EEG by 0.097 macro-F1; with a TCN, the same ordering persists, while detection slightly favors EEG and three-class grading is effectively tied.The TCN scores 0.978 ± 0.001, 0.767 ± 0.007, and 0.563 ± 0.012 macro-F1 across detection, three-class, and five-class tasks; corresponding video scores are 0.968, 0.780, and 0.657.Source: §E.6 · p. 26

E.7. Subject-exclusive transfer

• On subject-exclusive five-class grading, video reaches 0.475±0.024 macro-F1 and EEG 0.404±0.042, while both retain strong binary detection.Subject-exclusive detection macro-F1 is 0.948 ± 0.016 for video and 0.972 ± 0.015 for EEG; three-class grading falls to 0.720±0.028 and 0.656 ± 0.022, respectively.Source: §E.7 · pp. 26–27

E.8. Long-tail behavior and weighting

• On five-class grading, weighted video reaches 0.657 macro-F1 versus 0.136 for a majority-class baseline, but EEG has only 0.202 and 0.123 F1 on Stages 4 and 5.The dataset contains 193 Stage 5 clips, and the authors use inverse-frequency-weighted cross-entropy to address rare classes.Source: §E.8 · pp. 27–28

E.9. Where each modality fails, and why

• For detected events, video errors are mostly adjacent severity levels, whereas EEG often makes non-adjacent errors; EEG has higher Stage 2 recall, 0.863 versus video’s 0.650.EEG misses 10.1% of major Stage 3 convulsions, compared with 2.3% for video; the modalities also have largely non-overlapping detection errors.Source: §E.9 · p. 28

F. Extended Interpretation

• Video’s behavioral alignment supports severity grading, while EEG remains important for internal electrographic anomalies without overt symptoms.The authors describe video as measuring behavioral ground truth directly and EEG as providing an indirect electrographic proxy.Source: F. Extended Interpretation · pp. 28–30
• A lightweight 4M-parameter X3D-M nearly matches a 33M-parameter R(2+1)D model, but streaming latency and continuous false-positive rates were not evaluated.The authors suggest that dense temporal tracking matters more than parameter count in the reported comparison.Source: F. Extended Interpretation · p. 30
• Video and EEG have non-overlapping failure regimes, so relying on either sensor alone leaves distinct diagnostic blind spots.The video model detects seizures missed by EEG across 14.2% of the dataset, while EEG uniquely detects active events on 4.9% of clips.Source: F. Extended Interpretation · p. 30

• Subject-exclusive evaluation preserves binary detection at 0.968 →0.948±0.016 but reduces granular grading from 0.657 to 0.475 ± 0.024, without establishing cross-laboratory transfer.The cohort remains localized to one laboratory, and the authors identify hardware, induction-protocol, and genetics differences as untested boundaries.Source: G. Limitations and Future Work in Detail · p. 30
• Rare Stage 4 and Stage 5 episodes have disproportionately high error rates, while equal-penalty cross-entropy ignores the ordered S2-to-S5 progression.The proposed ordinal-objective direction and conditional framework still require empirical validation; inverse-frequency weighting is only a practical baseline.Source: G. Limitations and Future Work in Detail · p. 30
• The EEG baseline uses one channel, and the modalities were evaluated in isolation, leaving dense-montage performance and quality-aware video-EEG fusion unresolved.The authors note that a dense multi-electrode montage could narrow or eliminate the visual model’s current grading advantage.Source: G. Limitations and Future Work in Detail · p. 30
• Uniform RodEpil re-encoding reduces identified encoding differences but does not rule out residual shortcuts, and absent synchronized telemetry prevents direct multimodal comparison.The authors call for an open-source dataset pairing synchronized video with dense electrophysiology and subject-exclusive partitions.Source: G. Limitations and Future Work in Detail · p. 31
• Pre-segmented clips leave continuous-stream event detection, low-latency edge inference, and near-zero long-duration false-positive rates unaddressed.Lightweight backbones suggest edge deployment may be feasible, but practical continuous-monitoring engineering remains undeveloped.Source: G. Limitations and Future Work in Detail · p. 31
• Human review of both streams entangles annotation sources: EEG drives event boundaries, while visual phenotype determines severity grades.The authors propose segregated modality-specific expert annotations validated by inter-rater consensus to separate model performance from annotation mechanics.Source: G. Limitations and Future Work in Detail · p. 31