Source-linked AI summary
Acoustic Echo Control Based on Sound Object Identification for Suppressing Howling Caused by Complicated Acoustic Paths
Osamu Hoshuyama
TL;DR
Multiple terminals can create unintended echo paths that conventional path-estimation AECs cannot realistically handle, motivating a different control strategy. The paper identifies recurring sound objects and mutes channels by default, allowing playback only for judged-different signals; simulation suppresses howling but reveals a speech-quality trade-off from over-muting.
Problem
Multiple hands-free terminals can create unintended inter-terminal echo paths that conventional AECs cannot realistically handle, causing echo and howling and motivating disruptive mute discipline.
Method
The proposed method identifies whether sound objects reappear and uses default mute with pass/playback only when a signal is judged different from recently observed objects.
Results
Simulation suppressed sustained howling, but identification errors caused over-muting that fragmented desired speech and reduced intelligibility.
Takeaways & Limitations
Sound-object identification shifts acoustic echo control from path estimation toward conditional half-duplex gating that can break unintended echo loops without path information.
Abstract
from arXiv · showhide
This paper proposes acoustic echo control based on sound object identification for suppressing acoustic echo and howling in conferencing environments with complicated acoustic paths, where multiple hands-free terminals coexist in the same room. Conventional acoustic echo cancellers target fixed intra-device echo paths; however, unintended paths, for example, those formed via inter-terminal communication, are difficult to control and can lead to howling. Instead of estimating echo paths, the proposed approach identifies sound objects and keeps channels muted by default, allowing pass/playback only when the signal is judged not dentical to recently observed objects. This breaks echo loops caused by repeated reproduction of the same sound object and can be viewed as an extension of classical voice switching toward conditional half-duplex operation. Technical challenges and connections to related techniques are discussed, and a simulation shows howling suppression together with a trade-off against speech quality.
I. INTRODUCTION
Multiple hands-free terminals can create inter-terminal echo paths that conventional AECs cannot realistically suppress, producing howling and forcing disruptive mute discipline. The paper therefore proposes controlling playback through sound-object identification rather than echo-path estimation.
- Limitations of conventional AEC: Conventional AECs estimate and cancel fixed acoustic feedback paths, but cannot realistically handle inter-terminal paths involving networks, nonlinear processing, delay variation, and mute actions.The paper states that solving these paths by estimation remains extremely difficult even with deep learning.
- Proposed direction: The paper proposes acoustic echo control that identifies sound objects and controls their playback instead of estimating echo paths.The proposal targets acoustic echo and howling in conferencing environments with multiple hands-free terminals.
- Motivation: Inter-terminal communication can route microphone A2 audio through loudspeaker A1 and back into microphone A2, forming a howling loop.The setup includes nonlinear processing, delay variation, and loudspeaker saturation, while direct intra-device feedback is removed by A1’s AEC.
- Practical impact: Users commonly must keep microphone A2 muted or mute loudspeaker A1 before unmuting A2, disrupting conversational flow.Unexpected echo paths can also arise when hearable users join ongoing calls while moving.
III. PROPOSED APPROACH
Sound-object-based echo control detects whether signals reappear and gates playback accordingly, using default muting to break repeated-object echo loops without requiring acoustic-path information. This extends voice-switched half-duplex operation into conditional gating.
- III. PROPOSED APPROACH: The method identifies recurring sound objects, such as utterances or room sounds, rather than estimating an acoustic path and subtracting echo.Sound objects are treated as speech segments lasting tens to hundreds of milliseconds.
- A. Basic Policy: Channels remain muted by default and permit playback only when the current signal is judged different from recently observed objects at the same terminal.If identity is detected, local gating closes to prevent repeated reproduction.
- A. Basic Policy: The policy can break echo loops containing network delay and nonlinear in-device processing without using path information.This is presented as an in-principle consequence of identifying sound objects rather than acoustic paths.
- Implementation: Transmit and receive playback are each gated by comparisons against stored objects on the opposite signal side, blocking duplicate reproduction paths in principle.Transmit signals are sent only when unlikely to stem from the same utterance, while receive playback requires a judged-different object.
B. Processing Blocks
The processing design separates sound-object buffering, identity estimation, and gain control for discussion, challenge analysis, and verification. Playback is muted when identity is likely or comparison evidence is insufficient, and otherwise passed.
- Processing Blocks: The paper separates the processing elements to support discussion, challenge analysis, and verification, while leaving end-to-end joint processing for later.The three elements are extraction and buffering, sound-object identification, and playback control.
- Extraction and buffering: Sound objects are extracted from microphone or receive signals and buffered for roughly the network-delay and room-reverberation duration.The buffer retains comparison material for subsequent identification.
- Sound-object identification: Identification computes a similarity or identity probability between the current object and the buffered set.The comparison is applied to current transmit or receive objects.
- Playback control: Playback gain is muted when identity is likely and passed otherwise; insufficient comparison candidates trigger safe-side muting.A non-persistent identification error tends to produce only a brief error rather than a sustained loop.
IV. EXPECTED CHALLENGES
The main challenges lie beyond extracting and buffering sound objects: identification and playback control dominate performance, while integration, coexistence, and training or evaluation design remain additional barriers.
- IV. EXPECTED CHALLENGES: Sound-object extraction and buffering are relatively easy, but identification and playback control dominate performance.This makes the quality of similarity decisions and gating behavior central to system effectiveness.
- IV. EXPECTED CHALLENGES: Integration with the existing call pipeline, coexistence with current processing, and training and evaluation design create additional implementation barriers.These issues accompany the core identification and playback-control challenge.
A. Identification
Identification must balance robustness and latency: short-delay matching faces signal deformation and ambiguity, while imperfect decisions trade howling prevention against intelligibility.
- A. Identification: Short-fragment matching limits delay but increases false-pass and over-muting errors because fragments provide fewer identifying features.False passes can seed howling, whereas over-muting harms intelligibility.
- A. Identification: Noise, reverberation, codecs, nonlinear processing, clock mismatch, packet effects, and competing audio deform spectra and complicate object boundaries.Large near/far talker differences and continuous television or background music further complicate identity decisions.
- A. Identification: Candidate features include MFCCs and audio fingerprints, while deformation-robust representations from source separation and deep learning are promising but constrained by delay and computation.Longer buffers improve coverage but increase memory, computation, and accidental identification errors.
- A. Identification: Under simultaneous speech, hard mute suppresses howling but can make identification errors audible as dropouts; soft or frequency-selective control risks residual leakage.Context and processing order relative to AEC, noise suppression, and codecs also affect decisions.
C. Placement and System Implementation
System placement trades local low latency and privacy against limited observation, while server aggregation can improve identification at the cost of delay, load, and privacy; deployment conditions require tuning.
- C. Placement and System Implementation: Terminal-side processing avoids extra uplink bandwidth and favors low delay, but each terminal observes only limited signals.With at most one co-located terminal lacking the control, howling can be prevented in principle, but loops elsewhere remain unhandled.
- C. Placement and System Implementation: Server-side aggregation can improve identification but raises uplink delay, privacy, and load.This is the principal trade-off between centralized observation and deployment cost.
- C. Placement and System Implementation: Thresholds and buffer lengths depend on room, device, and codec conditions, making stable operation without site-specific tuning difficult.Under partial deployment, safe-side muting may over-mute and make calls impractical.
- C. Placement and System Implementation: Public data jointly covering network deformation, nonlinear in-device processing, and playback control are scarce, complicating evaluation and supervised identity labeling.Privacy considerations favor storing fingerprints or embeddings rather than raw waveforms.
V. RELATION TO OTHER TECHNOLOGIES
The approach extends voice-switched half-duplex operation through sound-object identification and relates to adaptive cancellation, source separation, and learning-based processing without speaker enrollment.
- V. RELATION TO OTHER TECHNOLOGIES: Sound-object identification extends classical voice-switched half-duplex toward conditional gating that can support echo control and howling suppression.The longer-term comparison is that half-duplex evolved toward full-duplex through AEC and related adaptive processing.
- V. RELATION TO OTHER TECHNOLOGIES: Using reference signals resembles adaptive noise cancellation and crosstalk cancellation, while extending multi-channel AEC framed as source separation to mute and communication paths.The same approach may also serve as a howling canceller for public-address systems.
- V. RELATION TO OTHER TECHNOLOGIES: Unlike personalized speech enhancement, the approach requires no speaker enrollment.End-to-end learning unifying identification and playback control is described as conceivable for combining accurate delayed systems with low-latency models.
VI. VERIFICATION SIMULATION
The verification simulation uses an empirically tuned initial sound-object gate in a two-room, three-terminal setup; it suppresses howling, while identification errors and over-muting remain visible under reverberation and overlapping speech.
- VI. VERIFICATION SIMULATION: The simulation uses a two-room, three-terminal setup with 16 kHz weighted overlap-add analysis and empirically selected control parameters.The implementation is explicitly an initial cosine-similarity gate rather than a final sound-object identifier.
- VI. VERIFICATION SIMULATION: Similarity uses magnitude-spectrum sequences with integer lags of ±4 hops, while short overlaps are discarded before comparison.The lag range corresponds to ±32 ms, and pairs overlapping by fewer than three frames are discarded.
- VI. VERIFICATION SIMULATION: Receive and transmit gains use asymmetric similarity thresholds, exponential smoothing, and terminal-specific mute configurations.A1 uses both receive and transmit muting, A2 uses transmit muting only, and automatic muting on B1 is disabled.
B. Call Chain and Talker Scenario
The simulation models multiterminal conferencing with crosstalk, inter-room feedback, and moderate reverberation, using overlapping talker scenarios to test the control method.
- Call Chain and Talker Scenario: The simulation includes same-room crosstalk, inter-room loudspeaker-to-microphone feedback, and moderate reverberation with T60 = 500 ms.It also uses approximately −50 dBFS background noise and a 200 ms inter-room delay.
- Call Chain and Talker Scenario: Two scheduled male talkers in Room A and one female talker in Room B produce single-talk, double-talk, and triple-talker overlap cases.The sources are shown in Fig. 3(a)–(c).
C. Results
The proposed control suppresses sustained howling in the simulation, but ambiguous overlap and identification errors can over-mute desired speech and reduce intelligibility.
- Results: After AEC convergence at about 13 s, single-talk identification errors are relatively few and howling suppression is stable.The controlled signals are compared with ideal loudspeaker signals in Fig. 3(e)–(h).
- Results: Sustained howling remains suppressed during double-talk and triple-talk despite ambiguous similarity decisions.Identification errors can cause momentary echo, which is quickly muted.
- Results: Over-muting fragments desired speech and reduces intelligibility, leaving a trade-off between howling suppression and speech quality.The paper identifies this trade-off as a key topic for improvement.