PaperlayerView exact revision on arXiv

Source-linked AI summary

What Selects, What Reconstructs: Repairing Exemplar-Based Complex-Spectrum Separation

Maxime Baelde

arXiv:2609.04756v1eess.SPcs.SDeess.AS

TL;DR

The paper asks when exemplar-based residual selection genuinely distinguishes candidates rather than merely rewarding a chosen regulariser. It proves that interpolation makes selection vacuous, proposes a rigid selector with richer reconstruction, and reports substantially smaller oracle-to-criterion gaps while retaining a quantified limitation against the masking ceiling.

  • Problem

    Exemplar methods use one deformation class both to select stored spectra and reconstruct the mixture, but the criterion’s ability to distinguish candidates requires a class that can fail to fit.

  • Method

    The paper analyzes interpolation by parameter counting, then ranks atoms with one complex gain and one pure delay while rebuilding aligned local atoms through joint complex least squares.

  • Results

    Under the rigid selector, the oracle-to-criterion gap falls from 6.23–7.70 dB to 0.48–2.80 dB across the reported sources and atom counts.

  • Takeaways & Limitations

    Residual-based model selection is empty whenever its fit class interpolates the observation, so selection and reconstruction should use different-capacity classes.

  • Takeaways & Limitations

    The repaired rule remains 10.0 dB below the masking-class ceiling on vocals, and the selection score retains a cross-term from the other source.

Abstract

from arXiv · show

Exemplar methods separate a mixture by picking one learned spectrum per source and deforming it until it explains the observation, making one deformation class both reconstructor and selector. We show that the second role is empty as soon as the class can interpolate: the rule then ranks candidates on its regulariser, a choice made before the data, and the estimates sum back to the mixture whichever candidate wins. The condition is a parameter count, so the diagnosis runs before any experiment. On free per-bin deformation of complex spectra it explains the observed pathologies at once: a criterion that ranks candidates by their loudness, and half an output that is a mask on the mixture rather than an exemplar. The same theorem prescribes the repair, a selection class poorer than the reconstruction class: one complex gain and one pure delay rank the candidates, and a local combination of the best-aligned atoms, fitted jointly in closed form, rebuilds them. On MUSDB18 against the exact ceiling of the masking class, the distance between the criterion and an oracle inside its own candidate pool falls under the rigid selector from 6.2-7.7 to 0.5-2.8 dB, though only 0.9-1.2 dB of that reaches the output, and the per-frame latency of the deployed rule by a factor of 47 to 806. One lock remains, quantified: atoms are scored against the mixture, so the score carries a term for the other source that absorbs the capacity the reconstruction class gains, leaving the output 10.0 dB under the ceiling. Ranking hypotheses by the residual of a fit free enough to interpolate ranks them on the regulariser alone.

I. INTRODUCTION

Exemplar separation can fail because an overly flexible deformation class makes residual-based selection vacuous: every candidate fits the mixture, so the regulariser alone ranks candidates. The paper separates selection from reconstruction by using a rigid selector and a richer joint reconstruction class.

  • I. INTRODUCTION: The method’s criterion, not merely dictionary content or search speed, is identified as the source of the observed separation failures.The paper distinguishes insufficient exemplars from a broken selection rule and targets the latter.
  • I. INTRODUCTION: Residual-based selection becomes empty when the deformation class interpolates every candidate pair, because no observation can reject a hypothesis.The condition is checkable by parameter counting before experiments.
  • I. INTRODUCTION: When interpolation holds, the criterion ranks candidates solely by the regulariser, a choice fixed before observing the data.The retained estimates still sum to the mixture for whichever pair wins.
  • I. INTRODUCTION: The proposed repair ranks candidates with one complex gain and one pure delay, then reconstructs with a jointly fitted local combination of aligned atoms.The ranking class is deliberately poorer than the reconstruction class, and both stages remain non-iterative.
  • I. INTRODUCTION: The selection score remains limited because scoring atoms against the mixture introduces a cross-term for the other source, which absorbs capacity as reconstruction flexibility grows.This preference moves selected spans toward explaining the mixture rather than resembling the source.
  • I. INTRODUCTION: Free per-bin complex deformation carries 4F real parameters per pair against 2F−2 frame constraints, yielding an affine feasible set of dimension at least 2F+2 under full row rank.The count explains why the Def-MAP class interpolates broadly rather than by chance.

A. The Method Under Study

The method under study applies a free per-frequency-bin deformation to complex spectral atoms and penalises deviation from a reference transform. Its reference transform preserves the learned real channel while imposing no prior phase information.

  • A. The Method Under Study: Def-MAP uses a short-time Fourier transform with a periodic Hann window at half-window hop and applies separate deformation components to the two sources.The method is instantiated as a residual selection rule for a specific deformation class and regulariser.
  • A. The Method Under Study: The deformation class is free in every frequency bin, scaling real and imaginary channels by independent real vectors rather than applying a complex multiplication.The action is defined bin by bin on the complex spectrum.
  • A. The Method Under Study: The regulariser penalises squared distance from a reference transform whose real channel is one and imaginary channel is zero.With a uniform first-frame index prior, this penalty alone separates candidates.
  • A. The Method Under Study: The reference transform is the MAP prior mean, not the identity: it preserves an atom’s real part while supplying no prior phase information.Applied through the stated deformation, its unpenalised action returns the atom’s real component.

B. The Criterion in Closed Form

The closed-form criterion solves equality-constrained least-squares problems independently in the real and imaginary channels. Its optimal deformation and score are determined by the candidate components and the mixture at each frequency bin.

  • B. The Criterion in Closed Form: At each fixed bin, the optimal real-channel deformation distributes the mixture residual across candidate real components under the quadratic penalty.The residual is ε = ax−a1−a2, with the candidate energies setting the allocation.
  • B. The Criterion in Closed Form: The resulting criterion sums the real residual penalty and the imaginary-channel contribution over frequency bins and sources.The closed-form expressions are obtained by solving two equality-constrained least-squares problems.
  • B. The Criterion in Closed Form: The optimal imaginary-channel deformation distributes the mixture’s imaginary component across candidate imaginary components under the same constrained least-squares structure.The target for this channel is zero in the regulariser, while the equality constraint matches the observed imaginary component.

C. Degeneracy of the Criterion

Because the free deformation class can interpolate the mixture, every candidate pair is accepted and the criterion ranks pairs through its regulariser rather than data fit. This yields exact source summation, loudness bias, and a mixture mask in one output channel.

  • Exact interpolation: Every non-degenerate pair produces estimates summing exactly to the mixture, so candidate choice cannot change that identity.This coupling also predicts equal criterion–oracle gaps across the two sources within the same class.
  • Exact interpolation: 4F free parameters against 2F −2 constraints make the deformation interpolate every candidate pair and observation.The resulting constraint set has dimension at least 2F + 2 when its constraint matrix has full row rank.
  • Regulariser-driven ranking: The criterion therefore measures distance from the reference transform, not how well a pair explains the mixture.The regulariser rewards properties of the candidates in the absence of any data term.
  • Output pathologies: The imaginary output channel becomes a ratio mask on the mixture and retains only the sign of the mixture’s imaginary part.It discards other dictionary phase features, making half the output subject to the masking ceiling.
  • Output pathologies: The criterion prefers loud atoms because its imaginary penalty decreases with candidate imaginary energy, regardless of atom shape.Equal-energy pairs differ only through where the mixture places its own energy; the real-term defect is likewise attenuated by candidate energy.
  • Computational consequence: The original rule evaluates every candidate pair with a per-pair solve over all frequency bins, giving O(N^2F) complexity.This computational burden accompanies a criterion that cannot reject candidates.

IV. SEPARATING SELECTION FROM RECONSTRUCTION

The repair separates selection from reconstruction: a rigid class ranks candidates, while a richer local complex-gain model rebuilds the retained atoms. The two stages are deliberately assigned different deformation capacities.

  • Two-stage design: Selection uses one complex gain and one pure delay per atom, while reconstruction jointly fits complex gains for the k best-aligned atoms from each source.The ranking class is intentionally poorer than the rebuilding class so its residual remains informative.
  • Two-stage design: The proposed rule therefore uses two stages that must not share a deformation class.The rigid selector is also reported alone as a control when tables evaluate its reconstruction directly.

A. Parameter Counts of the Two Stages

Parameter counts determine whether a deformation class can interpolate: the rigid selector lies below the constraint threshold, whereas the rich reconstruction class lies above it. The method then aligns atoms, selects locally, and fits their gains jointly.

  • A. Parameter Counts of the Two Stages: A class with 2p < q has negligible interpolating observations, so its residual recovers a data term.This converts selector design into a parameter-count inequality.
  • A. Parameter Counts of the Two Stages: The rigid class has 2p = 6 against q = 2F −2 and selects, while the rich class never selects and only rebuilds.Def-MAP reverses the inequality with 2p = 4F, producing interpolation.
  • B. Rigid Class: Gain and Pure Delay: The physical selector models exemplar mismatch with a level difference and subframe delay, represented by one complex gain and a linear phase ramp.The delay is estimated from cross-correlation and refined to sub-sample precision, with a model-based cap.
  • B. Rigid Class: Gain and Pure Delay: The deployed rigid selector fits candidate gains by a regularised 2 × 2 normal equation, whose residual measures misfit because the class cannot interpolate.The guarantee concerns non-degeneracy of the searched class; the deployed delay and gain estimates approximate the exact minimiser.
  • C. Rich Class: Local Combination of Aligned Atoms: For reconstruction, the method aligns both dictionaries, retains the k best atoms per source, and fits all k1 + k2 complex gains by one least-squares solve.At k = 1, this fit coincides with the pair rule on the same atoms.
  • C. Rich Class: Local Combination of Aligned Atoms: Figure 1 compares atom and mixture phase before alignment, after ramp removal, and after exact gain-and-delay removal.The remaining residual is what the rigid class cannot explain and what makes its criterion informative.
  • C. Rich Class: Local Combination of Aligned Atoms: The rich local class has 2k complex gains, or 4k real parameters per pair, so a small k can match the capacity of free per-bin deformation.Its capacity is evaluated against aligned atoms and compared with the free class’s ceiling.

D. Partial Reabsorption of the Residual

Partial residual reabsorption restores exact mixture summation at α = 1 but introduces a masking-type component, so only α = 0 isolates the exemplar model. The selection problem remains separately governed by scoring atoms against the mixture rather than the sources.

  • The reabsorbed residual is split per bin according to the estimates’ own energies and therefore adds a real per-bin gain of masking type.
  • At α = 0 the estimates are the pure model, whereas α = 1 restores estimates that sum exactly to the mixture.
  • Selection remains rigid at α = 0 even when the reconstruction model reabsorbs residual energy.
  • Only the α = 0 column supports claims about the exemplar model because the reabsorbed residual carries information absent from the dictionary.
  • Scoring atoms against the mixture adds a cross-term that favors explaining the mixture over resembling either source, with the gap widening as k grows.
  • Two approximations that penalize overlap improve vocals from +5.57 to +6.20 and +6.54 dB at k = 16 and 300 atoms, against a capacity of +12.99 dB.

F. Complexity of the Two Rules

The original rule searches every atom pair with costly per-pair deformation, whereas the rigid rule reuses alignment and the local combination removes pair search altogether. The protocol uses paired MUSDB18 measurements with explicit masking-class references and controlled nested dictionaries.

  • The original method evaluates N1N2 candidate pairs with a closed-form deformation over F bins, costing O(N^2F).
  • The rigid pair rule aligns the dictionary once and scores pairs through 2 × 2 normal equations, yielding O(NF log F) alignment plus BLAS-based pair scoring.
  • The local combination drops pair search and solves a joint system of size k1+k2, giving O(k^2F + k^3) locally and O(NF log F) overall.
  • The two papers share the measurement layer, making their reported numbers directly comparable.
  • MUSDB18 uses vocals and accompaniment, track-level held-out test material, nested 50-, 100-, and 300-atom dictionaries, and paired means over fifty test tracks.
  • Evaluation uses time-domain SDR after windowed overlap-add resynthesis, with edge trimming to prevent reconstruction boundaries from dominating error.
  • Every measurement includes mixture, ideal-ratio-mask, Wiener, and best-real-mask references, including the exact ceiling of the masking class.

B. Separating the Rule From the Dictionary

Using the same candidate pairs separates dictionary adequacy from criterion failure: the original criterion loses selection quality as dictionaries grow, while the rigid class sharply reduces that gap. The remaining gap still grows with dictionary size because selection is driven toward explaining the mixture rather than either source.

  • The oracle replays each criterion’s fitted transforms while choosing the candidate pair by true reconstruction error within the same pool.
  • 6.23, 6.57, and 7.70 dB are the original criterion’s oracle-to-criterion gaps at 50, 100, and 300 atoms per source.
  • The original criterion’s quality peaks at 100 atoms, reaching +5.84, +6.07, then +5.71 dB above the mixture as dictionary size increases.
  • The original gap is symmetric between sources, whereas the rigid-class gap is asymmetric by a factor of three and a half to four and a half.
  • Under the rigid class, the gap falls to 2.13, 2.29, and 2.80 dB on vocals and 0.48, 0.55, and 0.80 dB on accompaniment.
  • Two thirds to nine tenths of the loss is attributable to the selection-class parametrisation, without changing the dictionary, reconstruction class, or metric.
  • Between 50 and 300 atoms, the residual gap grows by 0.67 dB on vocals and 0.32 dB on accompaniment even as the same-class oracle improves.

D. Quality Against Capacity as k Grows

As k grows, the reconstruction class gains capacity but the deployed selection rule does not deliver it: the gap between capacity and quality widens, leaving the output nearly flat and far below the masking ceiling.

  • Quality Against Capacity as k Grows: 2.75, 3.31, 4.12, 5.34 and 7.42 dB separate capacity from quality on vocals as k increases from 1 to 16.On accompaniment, the corresponding gaps are 0.85, 1.06, 1.53, 2.56 and 4.53 dB.
  • Quality Against Capacity as k Grows: The hatched selection lock absorbs the entire 4.4 dB capacity gain between k = 1 and k = 16.Capacity is chosen against the truth, whereas quality is produced by the deployed rule.
  • Quality Against Capacity as k Grows: +8.56 to +12.99 dB capacity gains above the mixture while delivered quality remains between +5.57 and +6.11 dB.Quality peaks at k = 4 and falls at k = 16, so increasing k does not control accuracy under the present selection rule.
  • Quality Against Capacity as k Grows: The best repaired rule remains 10.0 dB below the masking-class ceiling on vocals, while supervised NMF is the comparable-latency baseline.The table reads absolute quality against m⋆, not the ideal ratio mask.
  • Quality Against Capacity as k Grows: At k = 1, increasing the vocals dictionary from 50 to 300 atoms raises quality by 0.34 dB but capacity by 1.00 dB.At k = 16, delivered quality is non-monotone across dictionary sizes: +5.56, +5.84, +5.57 dB.

G. Measured Latency

The repaired local combination is substantially faster than the original pair-search rule, while real-time feasibility depends on dictionary size and the paper does not claim blanket real-time performance or state-of-the-art quality.

  • Measured Latency: Latency depends on (N1, N2, F), is measured on random spectra, and excludes diagnostic rules that a deployed separator never evaluates.The reported timings are best-of-five on unoptimised single-frame NumPy using one CPU core.
  • Measured Latency: 47 to 806 times faster is the measured latency improvement of the local combination over Def-MAP.The rigid pair rule improves by 45 to 328 times and only ranks candidates.
  • Measured Latency: At 300 atoms per source, the frame duration is exceeded by a factor of 1.9, so real-time feasibility holds only at smaller measured dictionary sizes.The frame duration is 23 ms under the reported measurement convention.
  • Measured Latency: Each atom’s delay is estimated against the mixture and is contaminated by the other source.The paper states that decoupling the gain phase from delay requires another refinement rule.
  • Measured Latency: The paper does not claim state-of-the-art quality, dominance over its NMF baseline, exemplar-based accompaniment separation, or blanket real-time performance.These scope limits are stated explicitly alongside the remaining quality gap.
Loading 2609.04756v1…