Source-linked AI summary
Learning to deform the matched filter
Paul Anthony Haigh
TL;DR
The paper addresses how to retain the interpretability of analytical matched filters while adapting to hardware and channel distortions that violate their assumptions. It constrains learning to steer bounded deformations around an explicit matched filter, showing transfer across signalling and conditions with condition-dependent controller advantages and mostly sustained gains over a span-matched FSE after convergence.
Problem
Analytical filters are interpretable but rely on assumptions that practical hardware and channels violate, while learned replacements can discard that structure.
Method
Learning causally steers a bounded, inspectable deformation of an explicit matched filter using blind state descriptors, without replacing the waveform-processing path.
Results
Receiver deformation transferred across PR4 and PAM-2 signalling and held-out distortion conditions, generally outperforming fixed matched filtering and the span-matched FSE; controller rankings remained condition dependent.
Takeaways & Limitations
Deformable matched filtering provides an adaptive platform in which classical receivers learn continuously without surrendering their analytical identity.
Takeaways & Limitations
Evidence is limited to generalisation across held-out waveforms and impairment regimes within one optical wireless platform, using constrained bandwidth and memoryless nonlinear impairments.
Abstract
from arXiv · showhide
Analytical signal-processing blocks are interpretable and reliable, but their optimality depends on assumptions that practical hardware and channels violate. Learned replacements can adapt, but often discard the structure that makes the original solution understandable. Here we introduce deformable matched filtering, a platform in which learning steers a bounded deformation of an explicit matched filter instead of replacing the waveform-processing path. In a hardware-in-the-loop optical wireless link, blind state descriptors drive causal, pilotless updates while payload samples, transmitted references, and condition labels remain outside the controller. Receiver deformation generalises across held-out signalling and channel conditions and remains ahead of a span-matched fractionally spaced equaliser after stationary convergence in most tested regimes. A transferable transmitter deformation reduces error-vector magnitude in all 144 held-out evaluations, whereas its additional value after receiver adaptation emerges principally under severe combined distortion. KAN, MLP, and linear controllers provide different condition-dependent advantages within the same filter structure. During 24 hours of uninterrupted changing-condition operation, bounded deformable receivers recover their original operating regime after severe intervening distortion while the persistent conventional equaliser accumulates destructive state. These results establish deformable analytical filters as a reusable middle ground between fixed theory and end-to-end learned signal processing.
RESULTS
The matched filter remains the explicit signal-processing structure while causal controllers steer bounded deformations from blind receiver-state descriptors. Receiver deformation generalises across held-out signalling and impairments, while transmitter deformation is broadly transferable but adds value mainly under severe combined distortion.
- Matched filters adapt without being replaced: The receive filter remains anchored to the nominal matched filter, with learning restricted to a constrained deformation rather than an independently learned receiver.The zero-deformation state is exactly the conventional matched filter, and the deformation is applied causally between successive blocks.
- Matched filters adapt without being replaced: KAN, MLP, and linear controllers map the same blind 26-element state vector to the same 64-mode deformation family around a 301-tap matched filter.This keeps the waveform-processing path fixed while varying only the controller mapping.
- Matched filters adapt without being replaced: All three deformable PR4 receivers improved over the fixed matched filter across held-out roll-offs, while the preferred controller depended on the physical distortion.KAN was particularly strong under clean and nonlinear conditions; linear control often led under bandwidth-limited and combined conditions, with MLP often close behind.
- Matched filters adapt without being replaced: PAM-2 showed the same qualitative pattern: receiver deformation reduced EVM and generally remained ahead of the span-matched FSE, with controller ordering changing by condition.The shared advantage was attributed to constraining adaptation to an explicit family of matched-filter deformations, not to one universally superior controller.
- Transmitter deformation is largely static: The frozen transmitter deformation produced lower EVM than the conventional pulse in all 144 paired held-out evaluations.Its close agreement across roll-offs indicates a transferable transmitter operating point rather than compensation for one nominal waveform or impairment.
- Transmitter deformation is largely static: With the fixed receiver, Tx-DMF reduced mean EVM from 29.68% to 24.98%, a 15.8% relative reduction across all 20 HIL rounds.After receiver deformation was active, transmitter deformation added no material reduction under these moderate conditions.
- Transmitter deformation is largely static: Incremental transmitter benefit under combined distortion increased from 0.26% to 2.01% and then 2.76% across mild, intermediate, and most severe conditions.The results indicate increasing value from transmitter shaping as distortion becomes more severe, while controller architecture determines the preferred allocation.
Compensation has no universal allocation
Transmitter and receiver deformation do not have a universally optimal allocation: the preferred balance varies with controller architecture and distortion severity. The mapped optima are controller-specific and boundary-limited for KAN and MLP.
- Allocation results: 4.23% and 3.10% lower mean EVM than the common reference were achieved by KAN and MLP at (Tx,Rx) = (0.75,1.25).These are best mapped points because both receiver scales lie at the sampled boundary.
- Allocation results: 10.17% lower mean EVM than the common reference was achieved by linear control at (Tx,Rx) = (1.00,0.50).Linear control therefore preferred substantially less receiver deformation than KAN and MLP.
- Controller dependence: KAN and MLP favoured greater receiver deformation, whereas linear control favoured substantially less within the tested range.The allocation cannot be reduced to maximising adaptation at both ends or assigning all compensation to the receiver.
- Reference choice: The (1.00,1.00) operating point provides a common architecture-independent reference but is not necessarily the best joint allocation.Subsequent persistent-link experiments returned to this point to preserve direct system comparability.
Stationary dwells separate convergence from deformable-filter advantage
Stationary-dwell experiments allowed the conventional FSE to converge materially, yet deformable receivers generally retained lower late-dwell EVM. Transmitter deformation added its clearest value under severe combined distortion, while controller architecture still determined exceptions.
- Convergence: 13.3%, 8.3%, 20.2%, 18.9%, and 5.7% FSE EVM reductions occurred within single stationary dwells.These first-to-final-quarter reductions show that the conventional equaliser continued converging when conditions were held stationary.
- Late-dwell comparison: 28.4%, 18.5%, 18.7%, 22.8%, and 13.6% lower EVM than the converged FSE was achieved by two-sided KAN across five conditions.The corresponding MLP reductions were 29.0%, 20.6%, 21.4%, 27.6%, and 11.7%.
- Late-dwell comparison: 38.4%, 39.7%, 21.5%, and 24.9% lower EVM than the FSE was achieved by linear control in four conditions.Under the most severe combined condition, linear control finished approximately 2.6% worse than the FSE.
- Transmitter complementarity: 17.8%, 17.1%, and 11.5% late-half EVM reductions followed from adding fixed transmitter deformation under the most severe combined condition.These values correspond to KAN, MLP, and linear control, respectively; gains were small or slightly negative in several milder conditions.
- Interpretation: The deformable-filter advantage was not explained solely by faster response because the FSE converged materially during stationary operation yet remained inferior to KAN and MLP.The linear controller’s hardest-condition exception shows that controller structure still matters.
A full-day switching run tests endurance
A 24.024-hour, six-epoch hardware session tested whether deformable receivers remain bounded and recover after severe intervening distortion. They generally outperformed persistent FSE adaptation while preserving inspectable, controller-dependent behavior and revealing limits and scope boundaries.
- A full-day switching run tests endurance: 542 rounds across six approximately four-hour epochs tested persistent receivers without scheduled resets or condition labels.The session included a return to bandwidth fraction 0.70 after combined distortion.
- A full-day switching run tests endurance: 47.75%, 47.94%, and 48.42% late-quarter EVM were achieved by KAN, MLP, and linear Rx-DMFs under the hardest combined condition.The fixed matched filter reached 58.10%, while the persistent FSE reached 74.84%; frozen transmitter deformation further reduced the corresponding values to 42.12%, 42.35%, and 46.18%.
- A full-day switching run tests endurance: 98.44% EVM marked the persistent FSE after returning to bandwidth fraction 0.70, versus 11.90%, 12.06%, and 11.81% for receiver-only KAN, MLP, and linear DMFs.The fixed matched filter remained essentially unchanged at 30.35% initially and 30.49% on return.
- A full-day switching run tests endurance: All three controllers reshaped the matched-filter response while retaining a recognisable main passband, with evolving band-edge lift and attenuation notches.These changes made the learned physical compensation directly inspectable in frequency.
- A full-day switching run tests endurance: KAN, MLP, and linear control shared one bounded filter family but occupied different condition-dependent operating niches.The platform treats controller choice as matching capacity to the physical regime rather than selecting a universal winner.
- A full-day switching run tests endurance: Receiver deformation produced broad gains, while transmitter deformation was largely captured by a transferable static pulse and added value mainly under severe combined distortion.The preferred transmitter-receiver allocation depended on controller architecture.
- A full-day switching run tests endurance: Deformable filtering remained better than the persistent FSE in most late-dwell comparisons, but the linear receiver fell behind under the hardest combined condition.The experiments therefore do not establish universal dominance over conventional equalisation.
- A full-day switching run tests endurance: The study establishes generalisation within one optical wireless platform, not universal transfer across media or naturally drifting field channels.The HIL loop demonstrates causal physical adaptation but is not yet a real-time embedded implementation.
MATERIALS AND METHODS Study objectives and design
The study evaluates continuously adaptive matched filtering as a prespecified, causal hardware-in-the-loop platform. It keeps the conventional filter as a bounded structural reference while controllers use blind receiver descriptors to update subsequent processing.
- Study objectives and design: The experiments test whether an analytically defined communication filter can remain explicit while learning continuously from blind received-signal observations.The staged design covered development, qualification, composition, allocation, and persistence experiments.
- Study objectives and design: Each DMF starts from the conventional matched-filter solution and constrains adaptation to a bounded deformation family.Blind receiver-state descriptors determine deformation parameters for subsequent blocks; payload references and pilot sequences remain outside the controller path.
- Study objectives and design: Block-level causality lets observations from one HIL round influence receiver processing only from the next round onward.A round includes waveform generation, physical acquisition, receiver processing, scoring, and the subsequent causal state update.
- Study objectives and design: A prespecified receiver fixes architecture, blind features, hyperparameters, deformation constraints, adaptation policy, and stability controls before evaluation.Online controller weights and filter state remain free to evolve causally during transmission.
- Study objectives and design: KAN, MLP, and linear mappings receive identical blind descriptors and control the same allowed filter family.This isolates deformable-filter structure from controller expressive capacity.
- Study objectives and design: The nominal matched filter and span-matched FSE provide same-stage conventional receiver references.The FSE operates on the oversampled waveform before symbol decisions, whereas an in-line neural equaliser was not the principal benchmark.
- Study objectives and design: The experimental sequence progresses from receiver-only qualification through transmitter deformation, two-sided composition, allocation, switching controls, stationary dwell, and endurance operation.Later experiments reuse prespecified designs without architecture-specific retuning unless deformation budget is the variable.
- Study objectives and design: Repeated physical acquisitions pass through the complete hardware chain, combining imposed bandwidth or nonlinear conditions with residual hardware, timing, noise, and channel effects.The platform uses a Zynq UltraScale+ MPSoC with AD9152 DAC and AD9680 ADC hardware.
Waveform generation
The experiments generate PAM-2 and PR4 waveforms, transmit them through controlled impairment regimes and optical hardware, and evaluate causal matched-filter deformation against a fixed analytical reference.
- Waveform and signalling design: PAM-2 and PR4 waveforms test whether one adaptive receiver structure transfers across different temporal signal relationships.The controller receives no signalling-format information; format identity is used only for offline organisation and analysis.
- Waveform and signalling design: The nominal transmit waveform uses RRC shaping, with receiver qualification including held-out roll-offs of 0.15, 0.25, and 0.35.Persistent-link experiments use the common nominal roll-off of 0.25.
- Impairment regimes: Four channel regimes combine clean operation, bandwidth limitation, nonlinear distortion, and sequential bandwidth-plus-nonlinear distortion through the complete optical hardware chain.Combined labels x/y denote bandwidth fraction x followed by hyperbolic-tangent strength y.
- Impairment regimes: Receiver qualification uses clean, 0.625 bandwidth, 0.62 nonlinear, and 0.625/0.62 combined conditions, while later severity tests extend bandwidth fractions to 0.475.Static transmitter-pulse development uses a deterministic five-condition cycle including 0.70, 0.55, 0.769, and 0.70/0.769.
- Causal evaluation: The adaptive algorithms receive no impairment class or parameter values, and condition transitions do not trigger architecture-specific controller changes.Condition identity is reserved for protocol organisation and subsequent analysis.
- Causal evaluation: Abrupt regime changes stress state carryover, while stationary-dwell experiments separate transient adaptation from longer fixed-condition convergence.The persistence sequence is a stress test rather than a statistical model of naturally drifting field channels.
- Receiver processing: The deformable receiver starts from the conventional matched-filter state and updates only after processing each current block.The fixed conventional filter is also the analytical reference and zero-deformation state.
Receiver-state features
Receiver adaptation compresses each received block into fixed blind descriptors, then maps that state to a bounded deformation while preserving the explicit matched-filter path and causal operation.
- Feature construction: The controller receives a fixed 26-dimensional blind feature vector rather than sampled payload data, transmitted references, signalling labels, or impairment labels.The descriptors summarize signal shape, temporal memory, and sampling-phase structure.
- Feature construction: The 26 descriptors comprise six scalar measures, two moment descriptors, six short-lag autocorrelations, and twelve polyphase-energy descriptors.Feature definitions remain fixed across pulse shapes, signalling families, bandwidth restrictions, nonlinearities, and combined impairments.
- Controller mapping: The receiver estimates how the explicit matched-filter response should change rather than inferring transmitted symbols directly from the waveform.This distinguishes the architecture from direct learned equalisation.
- Controller mapping: KAN, MLP, and linear controllers map the same receiver-state vector to deformation parameters within one shared signal-processing path.They differ in their state-to-deformation mapping, not in the physical filter class.
- Controller mapping: The 301-tap receive filter operates at 12 samples per symbol and uses 64 smooth tapered discrete-cosine deformation modes with bounded magnitude.The same deformation family and stability budget support controller comparisons.
- Persistent causal operation: Online adaptation persists across HIL rounds, with controller weights and filter state retained unless blind safety logic requires reset or reheat.The receiver begins each logical link from the conventional matched-filter state and updates only later blocks.
Receiver stability controls
The system constrains online deformation with blind stability mechanisms, while transmitter shaping remains a bounded low-dimensional perturbation whose static operating point supports later experiments.
- Receiver controls: Blind stability mechanisms are incorporated before qualification to limit gradual drift away from the useful matched-filter neighbourhood.They remain unchanged in later held-out and persistent experiments.
- Receiver controls: After initialization or reset, 24 unconstrained acquisition blocks precede a trust radius calibrated from the first eight parameter updates.Exceeding the radius projects the parameter state onto the allowed boundary; reheating reopens a 24-block grace interval.
- Receiver controls: A blind health path can trigger reheating or return toward the matched-filter reference without transmitted symbols, and scheduled condition changes do not directly trigger it.Actions arise from internal receiver criteria.
- Transmitter shaping: Transmitter adaptation is constrained to a bounded low-dimensional deformation of the nominal RRC pulse rather than arbitrary waveform synthesis.The transmit filter uses eight smooth tapered basis functions and 193 taps.
- Transmitter shaping: A single static coefficient vector is optimized across PAM-2, PR4, and multiple impairment regimes, then fixed for subsequent transmitter qualification and two-sided experiments.The frozen pulse is tested at held-out RRC roll-offs without coefficient reoptimization.
- Transmitter shaping: Delayed blind feedback controls only a small residual transmitter deformation beyond the static pulse, using six matched-filter memory descriptors.The residual controllers are causal and do not use transmitted-symbol references or channel labels.
- Transmitter shaping: Because residual adaptation adds only a small gain beyond static deformation, later two-sided experiments use the static transmitter pulse.This isolates receiver adaptation and transmitter-receiver complementarity from a second rapidly varying loop.
Deformation allocation
The allocation study varies transmitter and receiver deformation budgets while holding the underlying machinery fixed, using paired combined-condition measurements and a span-matched conventional equaliser benchmark.
- Budget allocation: Transmitter and receiver deformation budgets are varied while the underlying transmitter direction and receiver adaptation machinery remain unchanged.Receiver budget changes scale both maximum filter deformation and late-trust displacement radius.
- Budget allocation: λTx = 0 restores the conventional RRC pulse, whereas λTx = 1 reproduces the independently determined static transmitter deformation.The transmitter scale therefore spans the nominal and static-deformation operating points.
- Budget allocation: The allocation experiment samples combined conditions 0.625/0.62, 0.55/0.769, and 0.475/0.90 across four receiver budgets using the same acquired waveform per transmitter setting.This creates a paired two-dimensional response surface while reducing physical-channel variation along the receiver-budget axis.
- Interpretation boundary: (λTx, λRx) = (1, 1) is the common reference, while reported best mapped allocations are not global optima because the tested grid is finite.For some controllers, the lowest EVM occurs at a receiver-range boundary.
- Conventional comparison: The conventional benchmark is a decision-directed normalized least-meansquares FSE with two samples per symbol and 51 taps.Its 25-symbol temporal span approximately matches the 301-tap receive-filter path at 12 samples per symbol.
- Conventional comparison: The FSE follows the same canonical matched-filter front end, adapts from an identity-centred state using blind decisions, and uses references only for offline EVM.Current-block observations affect only subsequent updates, matching the deformable receiver’s causal timing.
- Conventional comparison: The FSE provides a conventional adaptive-filter comparison at the same receiver stage, unlike a neural waveform equaliser that would replace the explicit filtering path.The benchmark therefore tests the architectural hypothesis of adapting around an analytical filter.
Persistent changing-condition experiment
Persistence experiments carried adaptive receiver and equaliser states across changing conditions without resets or condition labels. Stationary dwells and a 24-hour run separated convergence behavior from stress-tested state carry-over.
- Rapid switching: 180 logical rounds repeatedly changed channel conditions while adaptive receiver and FSE states persisted continuously.The rapid-switching protocol was designed as a stress test rather than a model of ordinary channel evolution.
- System comparisons: Five complete configurations compared fixed matched filtering, transmitter-only deformation, receiver deformation, two-sided deformation, and a span-matched FSE.Paired physical acquisitions reused the same conventional-transmitter waveform for receiver comparisons.
- Stationary dwells: Five conditions were presented in randomized 12-, 24-, or 48-round stationary dwells to assess convergence without resetting adaptive states.Early and late trajectory portions quantified transient response and longer-term behavior.
- 24-hour persistence: The 24-hour run contained 542 complete rounds across six physical channel epochs, including a repeated bandwidth-0.70 condition.States persisted across all condition boundaries without scheduled resets, pilots, training sequences, condition identity, or transition timing.
- Evaluation: EVM was the primary continuous waveform-error metric, scored offline against known transmitted references unavailable to online adaptation.References supported evaluation and comparison but were excluded from receiver, equaliser, and transmitter-feedback controller inputs.
Relative performance metrics
The study reports EVM improvements relative to explicit reference systems and isolates the incremental contribution of transmitter deformation after receiver adaptation. Settled comparisons use prespecified late windows, with descriptive treatment of repeated observations.
- Relative improvement: Positive G values indicate lower EVM for the tested system relative to the reference receiver.Reported percentage improvements are 100G.
- Transmitter increment: G_Tx|Rx isolates transmitter deformation's additional EVM contribution beyond the receiver-only system.This separates pulse-shaping benefit from the larger receiver-adaptation benefit that may already be present.
- Windowing: Final-quarter dwell comparisons represent settled behavior, while late-half statistics assess transmitter-receiver complementarity with reduced transition sensitivity.First- and final-quarter EVM quantify within-dwell FSE convergence.
- Statistical structure: Receiver comparisons were paired on shared physical acquisitions, while seeds, waveform blocks, and persistent rounds remained repeated observations rather than independent hardware replicates.Controller seed variation and descriptive statistics were matched to the experimental hierarchy.
- Statistical limitation: The shortest 12-round dwells contribute only three rounds to final-quarter estimates and are interpreted by effect direction and consistency.Narrow inferential intervals are not emphasized for these short windows.
- Long-horizon reporting: All 542 rounds are displayed for the 24-hour experiment, but temporal dependence means recorded round counts are not independent sample sizes.Late-window values are arithmetic means over prespecified contiguous windows.
Reproducibility and experimental control
The experiments used prespecified architectures, blind controller inputs, persistent checkpointed state, and documented hardware procedures. Repeats and controls support reproducibility, while the study reports descriptive rather than independent-replicate inference.
- Pre-registration: Architectures, features, deformation spaces, adaptation rules, safety policies, coefficients, and schedules were version-controlled before evaluation.Prespecified components were reused unless an experiment explicitly varied one.
- Blind control: Online adaptation excluded modulation identity, channel identity, impairment severity, payload symbols, and pilot information.These quantities were retained only for configuration, bookkeeping, and offline analysis.
- State persistence: Persistent adaptive state was checkpointed between HIL rounds, with interruptions recorded rather than used to modify receiver behavior.Resumed runs documented continuity and active operating time.
- Stress control: The adversarial 180-round sequence tested bounded state carry-over under repeated impairment changes, not convergence or realistic channel dynamics.The protocol intentionally changed conditions nearly every HIL round.
- Matched comparisons: Five complete system configurations and paired physical acquisitions enabled matched comparisons among fixed, transmitter-deformed, receiver-deformed, two-sided, and FSE systems.Receiver structures processed the same physical waveform realization.
- Repeatability: A partial repeat used newly acquired physical waveforms while retaining the prespecified architecture and schedule.Seeds and waveform blocks were algorithmic observations, not independent hardware replicates.
- Transmitter control: The frozen transmitter deformation benefited all 144 held-out evaluations, while its residual online gain was much smaller and moderate-condition benefit was largely redundant after receiver adaptation.These observations motivated a severity sweep for transmitter-receiver complementarity.
Persistence controls
Supplementary controls characterize deformation geometry, blind-state inputs, controller-specific behavior, rapid switching, stationary dwells, adaptation excursions, and long-horizon trajectories. Together they test transfer, bounded persistence, and reproducibility without changing the receiver structure.
- Deformation geometry: The zero-deformation state is exactly a canonical 301-tap RRC matched filter, while controllers select bounded amplitudes in a 64-mode smooth deformation family.The modes provide structured alternatives rather than an unconstrained waveform filter.
- Blind state: KAN, MLP, and linear controllers receive the same 26 blind descriptors, spanning scalar, moment, autocorrelation, and polyphase-energy groups.No payload samples, transmitted symbols, modulation labels, or impairment labels enter the state.
- Repeatability: Independent clean-condition repeats compare controller trajectories and EVM differences against the matched persistent FSE.The repeat evaluates reproducibility across newly acquired physical data.
- Controller dependence: Condition-resolved relative EVM reductions vary across KAN, MLP, and linear control within one receiver structure.The changing controller pattern, not repeated observations as independent physical replicates, is the relevant result.
- Transmitter qualification: Held-out transmitter qualification covers three roll-offs and 144 paired evaluations, with every Tx-DMF EVM point below conventional RRC equality.Continued online transmitter adaptation provides a separate residual-gain measurement.
- Factorial interaction: Under moderate distortion, a fixed Tx-DMF pulse benefits fixed matched filtering but adds little after nonlinear receiver adaptation and can be mildly negative with linear control.The factorial comparison motivates testing complementarity under more severe combined distortion.
- Persistence controls: Rapid switching carries adaptive state through nearly every-round impairment changes, whereas stationary dwells provide the convergence control.These experiments answer different questions and should not be interpreted interchangeably.
- Transient diagnostics: Small clean-condition excursions coincide with first activation of the late-trust constraint after its 24-block grace period, followed by readaptation.The diagnostic does not claim a matched-capture no-update counterfactual.