Mask-Induced Displacement in Audio XAI via Logit Trajectory Decomposition

Problem

• Masking substitutes a fill for removed features, so classifier output reflects the joint response to retained and replacement signals.The paper frames this as missingness bias and a risk of out-of-distribution responses.Source: §1 · p. 1

Motivation

• Faithfulness evaluation can be circular: fill choice and hyperparameters can alter rankings, and mask shape can cue class predictions.The authors therefore investigate output geometry under partial masking rather than evaluating faithfulness.Source: §1 · p. 1

Method

• Logit-space trajectory decomposition separates displacement along the fully occluded-to-original reference axis from perpendicular displacement.The analysis covers three fills, three classifiers, and 1,100 AudioSet clips.Source: §1 · p. 1, Figure 1 · p. 1

Contributions

• 41–77% of partial-mask output displacement is off-axis, and no tested fill is an information-absent baseline under full occlusion.Each fill activates a distinct class profile.Source: §1 · p. 1

Audio XAI

• Perturbation methods have been applied across audio tasks, while receiving comparatively little study in audio relative to vision.Examples include singing voice detection, music tagging, automatic speech recognition, audio content analysis, and speech emotion recognition.Source: §2 · p. 1

Alternative approaches

• Learned-surrogate approaches train decoder networks on classifier representations, gaining faithfulness at the cost of surrogate interpretability.Source: §2 · p. 2

Fill and baseline choice

• Prior work reports that fill choice can dominate faithfulness rankings, while baseline choice also shapes integrated-gradients attributions.Related work also examines feature removal and fill non-neutrality.Source: §2 · p. 2

3.1. Models & Experiment Setup

• The evaluation uses three AudioSet-trained mel-spectrogram classifiers; the two PANNs variants differ only in SpecAugment, while AST+SA is evaluated with SpecAugment.The 1,100 clips come from 50 clips in each of 22 AudioSet evaluation classes, selected to span acoustic types rather than sampled randomly.Source: §3.1 · p. 2

3.2. Spectrogram Segmentation, Sampling & Perturbation

• RISE-style masks use 25 redrawn contiguous blocks, 300 seeds, and 32 samples per seed, yielding 9,600 partial masks per clip with retention fractions centered near 0.5.The three fill strategies are ZERO, MEAN, and GAUSSIAN NOISE; MEAN substitutes time-averaged magnitude, while Gaussian noise is scaled by each clip’s average spectral magnitude per frequency bin.Source: §3.2 · p. 2
• With fixed grids, output variation at a given retention fraction partly reflects which blocks contain important cells, limiting tests of whether α represents proximity to the original signal.Masks are deterministic, enabling direct comparison of fill effects across conditions.Source: §3.2 · p. 2

3.3. Logit-Space Trajectory Analysis

• The logit trajectory is decomposed relative to the clip-specific difference between original and fully occluded outputs; τ measures projection along that axis, while d⊥ measures perpendicular displacement.The projection is anchored at τ = 0 for full occlusion and τ = 1 for the original output; the additive surrogate assumption requires τ ≈ α.Source: §3.3 · p. 2

3.4. Structure of the Off-Axis Residual

• Within each retention bin, covariance of pooled off-axis residuals estimates their effective dimensionality and tests whether the leading residual direction remains stable across retention fractions.Residuals are pooled across clips and 22 evaluation classes; the analysis uses 20 bins and reports participation ratio, pairwise direction similarity, and variance explained by the leading direction.Source: §3.4 · pp. 2–3

4.1. Fill Non-Neutrality Under Full Occlusion

• Under full occlusion, ZERO elicits Silence for AST+SA and PANNS-SA but Music for PANNS+SA, while GAUSSIAN NOISE elicits broadband-noise classes across all three models.MEAN responses also differ by architecture, and MEAN and GAUSSIAN NOISE entangle fill properties with source structure across retention fractions.Source: §4.1 · p. 3

4.2. Logit-Space Trajectories Under Partial Masking

• 41–77% of partial-mask output displacement is off-axis and invisible to scalar target-class methods, with its magnitude and direction shaped by model–fill combinations.LIME’s high-α weighting aligns with lower off-axis displacement for AST+SA, while KernelSHAP downweights the mid-α peak seen in both PANNs variants.Source: §4.2 · pp. 3–4, Figure 2 · p. 4
• Both PANNs variants have an inverted-U off-axis profile peaking near mid-retention, where Bernoulli block sampling concentrates masks and KernelSHAP assigns lower regression weights.For AST+SA, LIME’s high-retention weighting coincides with lower off-axis displacement; PANNS-SA’s Gaussian and mean fills can track τ=α while off-axis displacement remains substantial.Source: §4.2 · pp. 3–4

4.3. Structure of the Off-Axis Component

• PANNS-SA under ZERO has a structured off-axis residual: PR=8.4, mean directional similarity 0.95, and 32.1% of off-axis variance in one eigenvector.The leading direction’s consistency and low effective dimensionality make this profile a tractable target for future correction strategies.Source: §4.3 · p. 4, Table 3 · p. 4
• PANNS+SA’s leading off-axis direction is less stable across retention fractions, with mean similarity at or below 0.82 and a minimum as low as 0.02 under GAUSSIAN NOISE.Direction instability makes post-hoc correction less tractable.Source: §4.3 · p. 4
• AST+SA’s leading off-axis direction remains stable across fills, but its residual is more diffuse than PANNS-SA under ZERO, with PR 13.4–19.1 and 18.5–23.5% of variance in the leading direction.Its directional consistency therefore does not imply a compact residual.Source: §4.3 · p. 4, Table 3 · p. 4

• Across three architectures and 1,100 AudioSet clips, 41–77% of partial-mask output displacement is orthogonal to the attribution axis, and no tested fill is information-absent.Source: §5 · p. 4
• Logit-space geometric decomposition provides a black-box diagnostic of how fill and attribution method shape model outputs.Source: §5 · p. 4
• For certain model–fill combinations, off-axis displacement concentrates in a stable low-dimensional subspace that affords mitigation, but correction evaluation requires fill-dependent faithfulness metrics.Source: §5 · p. 4

• The study is computational and uses the publicly available AudioSet dataset.Source: §6 · p. 5
• No human subjects were involved in the study.Source: §6 · p. 5
• The study did not require ethical approval.Source: §6 · p. 5