Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

361 to 420 of 1,641

A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement
Jean-Marc Valin
cs.SDeess.ASarXiv:1709.08243v32017
Joint Audio and Speech Understanding
Yuan Gong, Alexander H. Liu, Hongyin Luo +2
cs.SDcs.AIeess.ASarXiv:2309.14405v32023
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis
Ho Kei Cheng, Masato Ishii, Akio Hayakawa +3
cs.CVcs.LGcs.SDarXiv:2412.15322v22024
A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognition
Saurabh Kumar, Diptiman Mohanta, Prasanta Kumar Ghosh
cs.CLeess.ASarXiv:2609.30160v12026
Unsupervised Speech Recognition
Alexei Baevski, Wei-Ning Hsu, Alexis Conneau +1
cs.CLcs.SDeess.ASarXiv:2105.11084v32021
AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs
Peize He, Zichen Wen, Yubo Wang +13
cs.SDcs.AIcs.CLarXiv:2510.07293v12025
E-chat: Emotion-sensitive Spoken Dialogue System with Large Language Models
Hongfei Xue, Yuhao Liang, Bingshen Mu +4
cs.SDeess.ASarXiv:2401.00475v32023
An unscented Kalman filter method for real time input-parameter-state estimation
Marios Impraimakis, Andrew W. Smyth
eess.SPcs.AIcs.CVarXiv:2511.02717v12025
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features
Andrew Owens, Alexei A. Efros
cs.CVcs.SDeess.ASarXiv:1804.03641v22018
Lead Vocal Separation from Vocal Ensemble Mixtures Using Phoneme Alignment
Yuma Narahata, Tomohiko Nakamura, Yuki Saito +1
cs.SDeess.ASarXiv:2609.06488v12026
Qwen3-ASR Technical Report
Xian Shi, Xiong Wang, Zhifang Guo +10
cs.CLcs.SDeess.ASarXiv:2601.21337v22026
Summaries:한국어
FireRedASR2S: A State-of-the-Art Industrial-Grade All-in-One Automatic Speech Recognition System
Kaituo Xu, Yan Jia, Kai Huang +6
eess.AScs.SDarXiv:2603.10420v12026
Deep Learning for Depression Recognition with Audiovisual Cues: A Review
Lang He, Mingyue Niu, Prayag Tiwari +9
eess.SPcs.SDeess.ASarXiv:2106.00610v12021
MMDenseLSTM: An efficient combination of convolutional and recurrent neural networks for audio source separation
Naoya Takahashi, Nabarun Goswami, Yuki Mitsufuji
cs.SDeess.ASarXiv:1805.02410v22018
CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition
Xuejing Yuan, Yuxuan Chen, Yue Zhao +7
cs.CRcs.LGcs.SDarXiv:1801.08535v32018
V2A-Mapper: A Lightweight Solution for Vision-to-Audio Generation by Connecting Foundation Models
Heng Wang, Jianbo Ma, Santiago Pascual +2
cs.CVcs.AIcs.MMarXiv:2308.09300v42023
Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challenge
Dan Stowell, Yannis Stylianou, Mike Wood +2
cs.SDeess.ASarXiv:1807.05812v12018
A Frequency-Domain Artificial Reverberator Plug-In
Jonas Roth, Nishanth Kumar, Silvan Krebs +2
eess.ASarXiv:2608.27695v12026
Self-supervised Learning with Random-projection Quantizer for Speech Recognition
Chung-Cheng Chiu, James Qin, Yu Zhang +2
cs.CLcs.SDeess.ASarXiv:2202.01855v22022
End-to-end speaker segmentation for overlap-aware resegmentation
Hervé Bredin, Antoine Laurent
eess.AScs.SDarXiv:2104.04045v22021
Not all generalisation failures can be bought back: four boundaries in affective audio modelling
Jingyi Zhang, Xiaotong Yao
eess.ASarXiv:2608.27674v12026
The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling
Tu Anh Nguyen, Maureen de Seyssel, Patricia Rozé +5
cs.CLcs.SDeess.ASarXiv:2011.11588v22020
AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?
Kaixiong Gong, Kaituo Feng, Bohao Li +8
cs.CVcs.AIcs.CLarXiv:2412.02611v12024
Highway Long Short-Term Memory RNNs for Distant Speech Recognition
Yu Zhang, Guoguo Chen, Dong Yu +3
cs.NEcs.AIcs.CLarXiv:1510.08983v22015
HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection
Ke Chen, Xingjian Du, Bilei Zhu +3
cs.SDcs.AIcs.IRarXiv:2202.00874v12022
Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition
Liam Schoneveld, Alice Othmani, Hazem Abdelkawy
cs.CVcs.LGcs.SDarXiv:2103.09154v22021
Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks
Morten Kolbæk, Dong Yu, Zheng-Hua Tan +1
cs.SDcs.LGeess.ASarXiv:1703.06284v22017
Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models
Zirui Song, Qian Jiang, Mingxuan Cui +9
cs.SDcs.AIeess.ASarXiv:2505.15406v12025
End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction
Zhong-Qiu Wang, Jonathan Le Roux, DeLiang Wang +1
cs.SDcs.CLcs.LGarXiv:1804.10204v12018
Data augmentation approaches for improving animal audio classification
Loris Nanni, Gianluca Maguolo, Michelangelo Paci
cs.LGcs.SDeess.ASarXiv:1912.07756v22019
End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors
Shota Horiguchi, Yusuke Fujita, Shinji Watanabe +2
eess.AScs.CLcs.SDarXiv:2005.09921v32020
BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation
Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi +2
eess.AScs.LGcs.SDarXiv:2103.06695v22021
Knowledge Transfer from Weakly Labeled Audio using Convolutional Neural Network for Sound Events and Scenes
Anurag Kumar, Maksim Khadkevich, Christian Fugen
cs.SDcs.MMeess.ASarXiv:1711.01369v42017
TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Models
Shengpeng Ji, Jialong Zuo, Minghui Fang +5
eess.AScs.LGcs.SDarXiv:2308.14430v12023
Direction-Preserving Active Noise Control with a Conditional Control-Filter Estimation Network
Ziyi Yang, Zhengding Luo, Boxiang Wang +2
eess.ASarXiv:2609.07173v12026
StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks
Hirokazu Kameoka, Takuhiro Kaneko, Kou Tanaka +1
cs.SDcs.LGeess.ASarXiv:1806.02169v22018
AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss
Kaizhi Qian, Yang Zhang, Shiyu Chang +2
eess.AScs.AIcs.LGarXiv:1905.05879v22019
Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation
Wei-Cheng Tseng, Xuanru Zhou, Mingyue Huo +3
eess.AScs.AIarXiv:2511.16757v22025
Music ControlNet: Multiple Time-varying Controls for Music Generation
Shih-Lun Wu, Chris Donahue, Shinji Watanabe +1
cs.SDeess.ASarXiv:2311.07069v12023
In defence of metric learning for speaker recognition
Joon Son Chung, Jaesung Huh, Seongkyu Mun +7
eess.AScs.SDarXiv:2003.11982v22020
Deep context: end-to-end contextual speech recognition
Golan Pundak, Tara N. Sainath, Rohit Prabhavalkar +2
eess.AScs.LGcs.SDarXiv:1808.02480v12018
Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization
Navonil Majumder, Chia-Yu Hung, Deepanway Ghosal +3
cs.SDcs.AIcs.CLarXiv:2404.09956v42024
End-to-end Audio-visual Speech Recognition with Conformers
Pingchuan Ma, Stavros Petridis, Maja Pantic
cs.CVeess.ASarXiv:2102.06657v12021
Adapting Frechet Audio Distance for Generative Music Evaluation
Azalea Gui, Hannes Gamper, Sebastian Braun +1
eess.ASarXiv:2311.01616v22023
Serialized Output Training for End-to-End Overlapped Speech Recognition
Naoyuki Kanda, Yashesh Gaur, Xiaofei Wang +2
cs.CLcs.SDeess.ASarXiv:2003.12687v22020
STEMMA: Song-to-Stem Multi-Audio Reasoning for Large Audio Language Models
Hoyeol Sohn, Wonil Kim, Keunhyoung Kim +7
cs.SDeess.ASarXiv:2610.11884v12026
Language Orthogonalization of Self-Supervised Speech Representations for Cross-lingual Parkinson's Detection
Minu Kim, Eunjung Yeo, Kwanghee Choi +1
eess.ASarXiv:2609.09499v12026
When Does Predictor-Based RL Align with Human Perception? A Study of Subjective Rewards in Codec-Based Speech Language Models
Joonyong Park, Jerry Li
cs.CLcs.SDeess.ASarXiv:2608.31035v12026
A Parameter-Free Few-Shot Evaluation for Elephant Vocalisation Classification
Christiaan M. Geldenhuys, Thomas R. Niesler
eess.AScs.LGcs.SDarXiv:2608.14824v12026
I Hear Your True Colors: Image Guided Audio Generation
Roy Sheffer, Yossi Adi
cs.SDeess.ASarXiv:2211.03089v22022
Streaming keyword spotting on mobile devices
Oleg Rybakov, Natasha Kononenko, Niranjan Subrahmanya +2
eess.AScs.SDarXiv:2005.06720v22020
Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding
Yifan Peng, Siddharth Dalmia, Ian Lane +1
cs.CLcs.SDeess.ASarXiv:2207.02971v12022
DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors
Chandan K A Reddy, Vishak Gopal, Ross Cutler
cs.SDcs.LGeess.ASarXiv:2010.15258v22020
The Second DIHARD Diarization Challenge: Dataset, task, and baselines
Neville Ryant, Kenneth Church, Christopher Cieri +4
eess.AScs.CLarXiv:1906.07839v12019
EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition
Zaijing Li, Fengxiao Tang, Ming Zhao +1
cs.CLcs.SDeess.ASarXiv:2203.13504v12022
EmoGavel: Grounding-Aware Reinforcement Learning with Verifiable Rewards for Multimodal Emotion Preference Learning
Zilong Huang, Kong Aik Lee, Zezhong Jin +2
cs.MMcs.LGeess.ASarXiv:2610.13353v12026
Speech Quality Assessment through MOS using Non-Matching References
Pranay Manocha, Anurag Kumar
eess.AScs.SDarXiv:2206.12285v12022
Attention based Convolutional Recurrent Neural Network for Environmental Sound Classification
Zhichao Zhang, Shugong Xu, Tianhao Qiao +2
cs.SDcs.LGeess.ASarXiv:1907.02230v12019
Large-Scale Weakly Labeled Semi-Supervised Sound Event Detection in Domestic Environments
Romain Serizel, Nicolas Turpault, Hamid Eghbal-Zadeh +1
cs.SDeess.ASarXiv:1807.10501v12018
Fully Learnable Deep Wavelet Transform for Unsupervised Monitoring of High-Frequency Time Series
Gabriel Michau, Gaetan Frusque, Olga Fink
cs.LGcs.SDeess.ASarXiv:2105.00899v22021

Audio and Speech Processing

A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement

Joint Audio and Speech Understanding

MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis

A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognition

Unsupervised Speech Recognition

AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs

E-chat: Emotion-sensitive Spoken Dialogue System with Large Language Models

An unscented Kalman filter method for real time input-parameter-state estimation

Audio-Visual Scene Analysis with Self-Supervised Multisensory Features

Lead Vocal Separation from Vocal Ensemble Mixtures Using Phoneme Alignment

Qwen3-ASR Technical Report

FireRedASR2S: A State-of-the-Art Industrial-Grade All-in-One Automatic Speech Recognition System

Deep Learning for Depression Recognition with Audiovisual Cues: A Review

MMDenseLSTM: An efficient combination of convolutional and recurrent neural networks for audio source separation

CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition

V2A-Mapper: A Lightweight Solution for Vision-to-Audio Generation by Connecting Foundation Models

Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challenge

A Frequency-Domain Artificial Reverberator Plug-In

Self-supervised Learning with Random-projection Quantizer for Speech Recognition

End-to-end speaker segmentation for overlap-aware resegmentation

Not all generalisation failures can be bought back: four boundaries in affective audio modelling

The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling

AV-Odyssey Bench: Can Your Multimodal LLMs Really Understand Audio-Visual Information?

Highway Long Short-Term Memory RNNs for Distant Speech Recognition

HTS-AT: A Hierarchical Token-Semantic Audio Transformer for Sound Classification and Detection

Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition

Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks

Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models

End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Data augmentation approaches for improving animal audio classification

End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors

BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation

Knowledge Transfer from Weakly Labeled Audio using Convolutional Neural Network for Sound Events and Scenes

TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Models

Direction-Preserving Active Noise Control with a Conditional Control-Filter Estimation Network

StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks

AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss

Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation

Music ControlNet: Multiple Time-varying Controls for Music Generation

In defence of metric learning for speaker recognition

Deep context: end-to-end contextual speech recognition

Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization

End-to-end Audio-visual Speech Recognition with Conformers

Adapting Frechet Audio Distance for Generative Music Evaluation

Serialized Output Training for End-to-End Overlapped Speech Recognition

STEMMA: Song-to-Stem Multi-Audio Reasoning for Large Audio Language Models

Language Orthogonalization of Self-Supervised Speech Representations for Cross-lingual Parkinson's Detection

When Does Predictor-Based RL Align with Human Perception? A Study of Subjective Rewards in Codec-Based Speech Language Models

A Parameter-Free Few-Shot Evaluation for Elephant Vocalisation Classification

I Hear Your True Colors: Image Guided Audio Generation

Streaming keyword spotting on mobile devices

Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding

DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors

The Second DIHARD Diarization Challenge: Dataset, task, and baselines

EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition

EmoGavel: Grounding-Aware Reinforcement Learning with Verifiable Rewards for Multimodal Emotion Preference Learning

Speech Quality Assessment through MOS using Non-Matching References

Attention based Convolutional Recurrent Neural Network for Environmental Sound Classification

Large-Scale Weakly Labeled Semi-Supervised Sound Event Detection in Domestic Environments

Fully Learnable Deep Wavelet Transform for Unsupervised Monitoring of High-Frequency Time Series