PaperlayerView exact revision on arXiv

Source-linked AI summary

BioSync: Transformer-Based Cross-Modal Fusion for a Multimodal Physiological Digital Biomarker

Seyed Mahmoud Sajjadi Mohammadabadi

arXiv:2609.04504v1cs.AIcs.IR

TL;DR

Wearable and mobile measures provide partial views of physiological state, while multimodal applications to wearable cognitive-decline biomarkers remain limited. BioSync combines modality-token attention with a linear branch; in two synthetic cohorts, it exceeded concatenation on selected metrics, though validation on real cohorts remains necessary.

  • Problem

    Multimodal wearable cognitive-decline biomarkers remain limited, and external validation is uncommon in the digital-biomarker literature.

    Source: §1 · p. 2

  • Method

    BioSync combines self-attention over modality tokens with a parallel linear branch whose hypothesis class includes feature concatenation, producing a continuous BSI.

    Source: §1 · p. 3, §4.4 · p. 8

  • Results

    Cognitive-cohort AUC was 0.928 versus 0.926 for concatenation, and metabolic-cohort accuracy/F1 was 0.764/0.766 versus 0.756/0.758.

    Source: §8 · p. 16

  • Takeaways & Limitations

    The BSI correlated with latent severity in both synthetic cohorts, addressing graded validity beyond separating discrete classes.

    Source: §3.4 · p. 6, §8 · p. 16

    All quantitative results come from synthetic cohorts rather than patients, raw AI-READI records, or prospective wearable recordings.

    Source: §7 · p. 15

Abstract

from arXiv · show

Cardiac, neural, behavioral, and speech measurements from wearable and mobile devices provide partial, noise-sensitive views of physiological state. BioSync combines these measurements into the \textbf{BioSync Index (BSI)}, a continuous composite digital biomarker defined under the BEST framework. The model applies multi-head self-attention to modality tokens and adds a linear branch whose hypothesis class includes standard feature concatenation. This architecture is motivated by latent-variable measurement theory and by the possibility that joint observations contain information unavailable from individual modalities. We evaluated BioSync on two literature-informed synthetic cohorts: a four-modality cognitive-decline cohort using HRV, EEG, actigraphy, and speech, and a metabolic-autonomic cohort structured around the public AI-READI wearable schema. In the cognitive cohort, BioSync and concatenation obtained AUCs of 0.928 and 0.926, respectively. In the metabolic cohort, BioSync obtained accuracy/F1 of 0.764/0.766, compared with 0.756/0.758 for concatenation. The BSI correlated with latent severity in both cohorts ($r=0.91$ and $r=0.68$). A pure-attention ablation obtained cognitive-cohort AUC 0.911, locating the increase to 0.928 in the combined wide-and-deep architecture. With matched modality-dropout training, BioSync led concatenation at five of six cognitive-cohort corruption rates and at the highest metabolic-cohort rate. Its cognitive-cohort AUC was also higher than five published digital-biomarker reference values, although differences in datasets and tasks preclude a controlled benchmark claim. Comparison with single-modality, early-fusion, and late-fusion designs across six prespecified criteria identifies the model's computational properties; validation on real cohorts remains necessary.

1. Introduction

Wearable and mobile measures offer a route to repeated monitoring, but multimodal cognitive-decline biomarkers and external validation remain limited. BioSync combines modality-specific attention with a linear branch to produce a continuous index, tested in two synthetic cohorts.

1. Introduction

  • Multimodal biomarkers appeared in only 24 of 431 reviewed studies, and fewer than 3% of studies performed external validation.Clinical assessments and biomarkers such as cerebrospinal fluid assays, amyloid PET, and structural imaging are not well suited to frequent longitudinal monitoring of MCI in daily life.Source: §1 · p. 2
  • HRV, EEG, actigraphy, and speech provide cardiac-autonomic, neural, behavioral, and acoustic or linguistic measures relevant to cognitive screening.Prior multimodal studies support combining signals across these pathways.Source: §1 · p. 3
  • Wearable cognitive-decline applications of cross-modal attention remain limited, and few studies evaluate computational constraints for wearable or edge deployment.Common alternatives concatenate features before classification or combine modality-specific classifiers by voting, neither producing subject- and session-specific modality weights.Source: §1 · p. 3
  • BioSync treats modalities as tokens, uses multi-head self-attention to estimate subject-level contributions, and defines its continuous BSI as a composite digital biomarker under the BEST framework.A parallel linear branch retains the decision functions available to feature concatenation; the design is related to latent-variable measurement and potentially joint-modality information.Source: §1 · p. 3
  • The two literature-seeded synthetic cohorts test cross-modal weights, BSI association with latent severity, and performance degradation under modality corruption, but do not support clinical inference.The cognitive cohort uses HRV, EEG, actigraphy, and speech; the metabolic-autonomic cohort follows the public AI-READI wearable-activity-monitor schema, without analyzing patient or raw AI-READI data.Source: §1 · p. 3

2. Related Work

Prior work establishes digital and wearable measures across several domains, while multimodal cognitive biomarkers, external validation, and deployment-focused evaluation remain limited. Existing fusion and privacy-preserving approaches provide context for BioSync’s design.

2.1. Digital biomarkers for Alzheimer’s disease and MCI

  • A review of 431 studies found only 24 multimodal digital-biomarker studies, with external validation and calibration rarely reported; a late-fusion study reported 91.8% accuracy for cognitive-impairment screening.The reported hard-voting rule remains fixed across participants.Source: §2.1 · p. 4

2.2. Cardiac-autonomic, neural, and behavioral digital signals

  • HRV, EEG, and actigraphy capture complementary autonomic, neural, and behavioral signals, while reported HRV associations vary and EEG measures have replicated MCI associations.A study combining EEG and HRV features outperformed either signal alone for MCI detection.Source: §2.2 · p. 4

2.3. Multimodal fusion architectures

  • Early fusion concatenates features and late fusion combines modality-specific classifiers, while attention-based and other fusion methods have been applied across physiological and clinical tasks [22].Prior examples include EEG–peripheral fusion, ECG–phonocardiogram fusion, bidirectional EEG–ECG/HRV attention, and kernel-based discriminant correlation fusion.Source: §2.3 · p. 4

2.4. Efficient and privacy-preserving on-device learning

  • Wearable and edge deployment constrain computation and data transfer; low-rank adaptation and federated learning are relevant approaches, and BioSync outlines a federated-training arrangement.Federated learning can train across institutions without centralizing raw data and has been applied to stress detection from smart-band heart-activity data.Source: §2.4 · p. 4

3. Theoretical Rationale

The rationale combines noisy-view measurement and joint-modality information with attention’s capacity for data-dependent weighting and token interactions. It also defines six criteria for judging composite biomarkers beyond classification accuracy.

3.1. A latent-variable measurement model

  • Under conditional independence of modality errors given latent state, combining noisy modality estimates by inverse-variance weighting yields lower variance than an individual estimate.Modality noise may vary by subject and session, as with EEG motion artifact or a loose wristband during actigraphy.Source: §3.1 · p. 5

3.2. Information-theoretic complementarity and synergy

  • Partial information decomposition distinguishes redundant, unique, and synergistic information; synergy can occur only in joint observations and may be missed by single-modality or additive analyses.Late voting does not model feature-level cross-modal interactions, while linear early fusion represents additive effects unless interaction terms are supplied explicitly.Source: §3.2 · p. 5

3.3. Why a Transformer specifically

  • Self-attention turns modality embeddings into tokens, computes pairwise interactions, and uses softmax-normalized weights for data-dependent reweighting.The same encoder can process the two- and four-token configurations evaluated here; adding a modality still requires an input encoder and retraining.Source: §3.3 · p. 5

3.4. A multi-criteria definition of what makes a composite biomarker good

  • The six criteria are graded validity, multimodal information capture, adaptive interpretability, graceful degradation, architectural generality, and on-device and federated feasibility.The comparison includes single-modality, early-fusion, late-fusion, and BioSync designs because accuracy alone does not assess these properties and does not consistently favor BioSync in the synthetic experiments.Source: §3.4 · p. 6

4. Proposed BioSync Framework

BioSync represents modality-specific measurements as tokens whose attention-derived fusion produces a continuous risk score and per-subject relevance weights, alongside a linear path that retains concatenation’s decision functions.

4.1. Problem formulation

  • The BSI is a continuous risk score, paired with attention weights summarizing each modality’s contribution for a participant.It is intended as a susceptibility/risk and monitoring biomarker, not a diagnostic label.Source: §4.1 · p. 6

4.2. Two application domains and their modality-specific features

  • The cognitive-decline domain uses HRV, EEG, actigraphy, and speech; the metabolic-autonomic domain groups AI-READI wearable variables into cardiac-autonomic and behavioral tokens.The metabolic study uses public documentation and synthetic data rather than controlled-access AI-READI records.Source: §4.2 · pp. 6–7

4.3. Transformer fusion encoder

  • BioSync projects each modality into a shared token space; a learned fusion token attends across modality tokens, yielding the fused representation and normalized per-subject relevance weights.The same attention block is used for both domains, with separate modality-specific input projections.Source: §4.3 · pp. 7–8

4.4. A wide-and-deep hybrid containing the concatenation baseline

  • The wide-and-deep model sums a linear path over concatenated features with the Transformer path; removing either path recovers concatenation or pure attention.Its hypothesis class contains concatenation’s linear decision functions, though finite-sample optimization and regularization can still lower held-out performance.Source: §4.4 · p. 8

4.5. Toward federated, privacy-preserving training

  • Because modality encoders process local feature vectors, BioSync is compatible in principle with federated training, but federated training was not implemented or evaluated.The proposed setup would share parameter updates rather than centralize raw physiological recordings.Source: §4.5 · pp. 8–9

5. Experimental Validation

Across two synthetic cohorts, BioSync’s strongest results varied by metric, while matched modality corruption favored it more consistently in the cognitive cohort. The experiments assess model behavior under literature-seeded assumptions, not clinical diagnostic performance.

5.1. Rationale and scope

  • The evaluation used two synthetic, literature-informed cohorts; no patient data were used, and reported metrics do not estimate clinical diagnostic performance.Source: §5.1 · p. 9

5.2. Cohort construction

  • Each cohort contained 360 profiles with severity-linked synthetic features; the continuous severity values were reserved for correlation analysis rather than used as training targets.The cognitive cohort modeled four modalities, while the metabolic cohort grouped five features into two AI-READI-schema tokens.Source: §5.2 · p. 9

5.3. Experimental setup

  • Models were compared using stratified five-fold cross-validation; BioSync used one encoder block and plain gradient descent for 500 epochs with weight decay.Accuracy, F1, and AUC were averaged across folds; severity correlations used pooled out-of-fold BSI values.Source: §5.3 · p. 9

5.4. Main results

  • 0.928 cognitive-cohort AUC (BioSync) exceeded 0.926 (concatenation); metabolic accuracy/F1 were 0.764/0.766 versus 0.756/0.758.BioSync ranked first for cognitive AUC and metabolic accuracy and F1; several clean-data differences were within or near one cross-validation standard deviation.Source: Table 1 · p. 10, §5.4 · p. 10

5.5. BSI and simulated severity

  • r=0.905 in the cognitive cohort and r=0.681 in the metabolic cohort measured correlations between pooled out-of-fold BSI and latent severity, which was not a training target.Direct severity regression produced higher correlations; near-uniform mean attention weights in the synthetic cohorts alone do not demonstrate adaptive interpretability.Source: §5.5 · p. 10

5.6. Robustness to heterogeneous modality reliability

  • Five of six cognitive corruption rates favored BioSync over concatenation, while metabolic results showed no monotonic ordering.With matched modality-dropout training, cognitive AUC at p=0.5 was 0.891 versus 0.865; metabolic results were less stable.Source: §5.6 · p. 11

5.7. Architecture ablation

The ablations show that the wide-and-deep model exceeded pure attention and concatenation on cognitive-cohort AUC, while results against literature and alternative designs are qualified by comparison scope and metric differences.

5.7. Architecture ablation

  • Cognitive-cohort AUC was 0.928 for wide-and-deep BioSync versus 0.911 for pure attention and 0.926 for concatenation; accuracy did not improve.Source: §5.7 · p. 11
  • With matched modality-dropout training, BioSync led at 5 of 6 cognitive corruption rates and at the highest metabolic rate tested.Source: Figure 6 · p. 13
  • Across the d/H grid, accuracy ranged 0.831–0.858 and AUC 0.915–0.932; d=8, H=2 was retained to avoid post hoc selection.Source: §5.7 · p. 11, Figure 7 · p. 13

5.8. Positioning against published digital biomarkers

  • BioSync’s synthetic cognitive-cohort AUC of 0.928 is numerically above five plotted literature values, but mixed metrics and datasets make the ordering descriptive.Source: Figure 8 · p. 14, §5.8 · p. 14

5.9. Comparison against alternative fusion designs

  • BioSync alone satisfies all six architectural criteria, but concatenation exceeds it on cognitive accuracy/F1 and metabolic AUC.Source: §5.9 · p. 14, Table 2 · p. 15

6. Discussion

Clean-data rankings depend on the metric, while matched corruption tests favor BioSync more consistently in the cognitive cohort. The discussion also distinguishes attention-based relevance from causal or calibrated sensor interpretation and identifies limits of the synthetic, summary-feature design.

6. Discussion

  • BioSync ranked first on cognitive-cohort AUC and metabolic-cohort accuracy/F1, while concatenation ranked first on the other three cohort–metric combinations.All differences were small relative to fold-level variation; adding the linear path raised cognitive AUC from 0.911 to 0.928.Source: §6 · p. 15
  • BioSync led at five of six cognitive corruption rates with an increasing margin, but the two-token metabolic results were nonmonotonic.The high-variance noise replacement is a simplified model of modality failure.Source: §6 · p. 15
  • Subject-specific modality weights are attention-based relevance scores, not causal contributions or calibrated sensor reliabilities.The BSI correlated with latent severity at r = 0.905 and r = 0.681, while direct severity regression produced higher correlations.Source: §6 · p. 15
  • Hand-crafted summary features limit what the attention path can learn; temporal CNN or GRU encoders could admit windowed signals while retaining summary features in the wide branch.Source: §6 · p. 15
  • Only 24 of 431 reviewed AD digital-biomarker studies used multimodal signals, and external validation was uncommon; BioSync’s clinical value cannot be determined from synthetic data.Source: §6 · p. 15

7. Limitations and Future Work

The reported results are bounded by synthetic cohorts, summary-feature inputs, and simplified corruption assumptions; the proposed federated extension is not yet tested. Future work outlines real-cohort validation, temporal encoders, empirical failure models, and deployment studies.

7. Limitations and Future Work

  • All quantitative results come from synthetic cohorts, so the reported metrics characterize literature-seeded assumptions rather than diagnostic performance.Clean-data differences from concatenation are small, and larger real cohorts are needed for statistical comparisons.Source: §7 · p. 15
  • The deep path uses hand-crafted summaries rather than raw or windowed signals, and the high-variance-noise corruption test does not capture all real failure patterns.Actual failures may be partial, temporally structured, or correlated; the proposed federated-training extension has not been implemented or measured.Source: §7 · pp. 15–16
  • Prospective cognitive validation is planned using Empatica wristbands, Muse EEG, and passive smartphone speech; gait or motor features could form a fifth token.Source: §7 · p. 16
  • The metabolic-autonomic cohort uses the public AI-READI wearable schema, not its controlled-access JSON records.The validation plan proposes windowed temporal CNN/GRU encoders, empirical artifact and missingness models, federated experiments, and on-device processing tests.Source: §7 · p. 16

8. Conclusion

BioSync combines attention over modality tokens with a linear path to produce a continuous BioSync Index. Synthetic-cohort results motivate real-cohort validation rather than clinical conclusions.

8. Conclusion

  • The BioSync Index is a continuous composite biomarker whose attention path models interactions among modality tokens and whose linear path contains concatenation’s decision functions.The design is motivated by precision-weighted latent-variable measurement and information that may arise from joint modality configurations.Source: §8 · p. 16
  • BioSync achieved cognitive-cohort AUC 0.928 versus 0.926 for concatenation and metabolic-cohort accuracy/F1 of 0.764/0.766 versus 0.756/0.758.Concatenation remained higher on cognitive accuracy/F1 and metabolic AUC.Source: §8 · p. 16
  • The synthetic-cohort findings are hypotheses for real-cohort validation, not clinical conclusions.Dataset, task, and metric differences prevent a controlled benchmark claim against published reference values.Source: §8 · p. 16

CRediT authorship contribution statement

Seyed Mahmoud Sajjadi Mohammadabadi is credited with conceptualization, methodology, software, formal analysis, investigation, writing, and visualization.

CRediT authorship contribution statement

  • The author is credited with conceptualization, methodology, software, formal analysis, and investigation.The statement also credits the author with writing the original draft, review and editing, and visualization.Source: CRediT authorship contribution statement · p. 16

Funding

The study reports no specific grant funding from public, commercial, or not-for-profit organizations.

Funding

  • The research received no specific grant from public or commercial agencies or not-for-profit organizations.Source: Funding · p. 17

Data availability

Synthetic data-generation code and analysis scripts used for Table 1 are available from the corresponding author upon reasonable request; no real patient data were used.

Data availability

  • The synthetic data-generation code and Table 1 analysis scripts are available from the corresponding author upon reasonable request.No real patient data were used.Source: Data availability · p. 17

Related papers

Loading 2609.04504v1…