Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

421 to 480 of 1,641

Disentangling by Partitioning: A Representation Learning Framework for Multimodal Sensory Data
Wei-Ning Hsu, James Glass
stat.MLcs.CLcs.LGarXiv:1805.11264v12018
Lung Sound Classification Using Co-tuning and Stochastic Normalization
Truc Nguyen, Franz Pernkopf
eess.AScs.LGcs.SDarXiv:2108.01991v12021
V2TATC: A Joint Voice-Trajectory Embedding Framework and Dataset for Air Traffic Controller Situational Awareness
Louis Brusset, Mathurin Petit, Jordan Kam +1
cs.LGeess.ASarXiv:2608.28981v12026
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
Hang Zhou, Yasheng Sun, Wayne Wu +3
cs.CVcs.LGcs.MMarXiv:2104.11116v12021
Summary of the ChinaVoices Challenge 2026: Data, Tasks, Baseline, and Methods
Yujie Liao, Bingshen Mu, Shuiyuan Wang +5
eess.ASarXiv:2609.03471v12026
Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
Yuxuan Wang, Daisy Stanton, Yu Zhang +7
cs.CLcs.LGcs.SDarXiv:1803.09017v12018
Step-Audio-EditX Technical Report
Chao Yan, Boyong Wu, Peng Yang +12
cs.CLcs.AIcs.HCarXiv:2511.03601v22025
End-to-End Adversarial Text-to-Speech
Jeff Donahue, Sander Dieleman, Mikołaj Bińkowski +2
cs.SDcs.LGeess.ASarXiv:2006.03575v32020
Improving Generalization of Transformer for Speech Recognition with Parallel Schedule Sampling and Relative Positional Embedding
Pan Zhou, Ruchao Fan, Wei Chen +1
cs.CLeess.ASarXiv:1911.00203v22019
Replay and Synthetic Speech Detection with Res2net Architecture
Xu Li, Na Li, Chao Weng +4
eess.AScs.AIarXiv:2010.15006v32020
A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection
Archontis Politis, Sharath Adavanne, Tuomas Virtanen
eess.AScs.SDarXiv:2006.01919v22020
Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models
Lok-Lam Ieong, Chia-Chien Chen, Chih-Kai Yang +3
cs.SDcs.AIcs.CLarXiv:2603.14636v12026
EEG-based Auditory Attention Decoding: Towards Neuro-Steered Hearing Devices
Simon Geirnaert, Servaas Vandecappelle, Emina Alickovic +6
eess.SPeess.ASarXiv:2008.04569v32020
SALMONN: Towards Generic Hearing Abilities for Large Language Models
Changli Tang, Wenyi Yu, Guangzhi Sun +6
cs.SDcs.CLeess.ASarXiv:2310.13289v22023
Recent Progresses in Deep Learning based Acoustic Models (Updated)
Dong Yu, Jinyu Li
eess.AScs.CLcs.SDarXiv:1804.09298v22018
FedNST: Federated Noisy Student Training for Automatic Speech Recognition
Haaris Mehmood, Agnieszka Dobrowolska, Karthikeyan Saravanan +1
eess.AScs.AIcs.CLarXiv:2206.02797v22022
Targeted Adversarial Examples for Black Box Audio Systems
Rohan Taori, Amog Kamsetty, Brenton Chu +1
cs.LGcs.CRcs.SDarXiv:1805.07820v22018
On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition
Jinyu Li, Yu Wu, Yashesh Gaur +3
eess.AScs.CLarXiv:2005.14327v22020
fairseq S2T: Fast Speech-to-Text Modeling with fairseq
Changhan Wang, Yun Tang, Xutai Ma +4
cs.CLeess.ASarXiv:2010.05171v22020
Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset
Xie Chen, Yu Wu, Zhenghao Wang +2
cs.CLeess.ASarXiv:2010.11395v32020
Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar +15
cs.SDcs.AIcs.CLarXiv:2604.10905v12026
Convolutional-Recurrent Neural Networks for Speech Enhancement
Han Zhao, Shuayb Zarar, Ivan Tashev +1
cs.SDcs.CLcs.LGarXiv:1805.00579v12018
Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention
Yuma Koizumi, Kohei Yatabe, Marc Delcroix +2
eess.AScs.LGcs.SDarXiv:2002.05873v12020
Co-Separating Sounds of Visual Objects
Ruohan Gao, Kristen Grauman
cs.CVcs.MMcs.SDarXiv:1904.07750v22019
Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model
Deepanway Ghosal, Navonil Majumder, Ambuj Mehrish +1
eess.AScs.AIcs.CLarXiv:2304.13731v22023
BigVGAN: A Universal Neural Vocoder with Large-Scale Training
Sang-gil Lee, Wei Ping, Boris Ginsburg +2
cs.SDcs.CLcs.LGarXiv:2206.04658v22022
x-vectors meet emotions: A study on dependencies between emotion and speaker recognition
Raghavendra Pappagari, Tianzi Wang, Jesus Villalba +2
eess.AScs.LGcs.SDarXiv:2002.05039v12020
R-Transformer: Recurrent Neural Network Enhanced Transformer
Zhiwei Wang, Yao Ma, Zitao Liu +1
cs.LGcs.CLcs.CVarXiv:1907.05572v12019
A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet
Jean-Marc Valin, Jan Skoglund
eess.AScs.LGcs.SDarXiv:1903.12087v22019
Universal Speech Enhancement with Score-based Diffusion
Joan Serrà, Santiago Pascual, Jordi Pons +2
cs.SDcs.LGeess.ASarXiv:2206.03065v22022
Weakly Supervised Tabla Stroke Transcription via an Adaptive Dynamic Rhythm Language Model (ADRM)
Rahul Bapusaheb Kodag, Vipul Arora
eess.ASarXiv:2608.30314v12026
Your Microphone Array Retains Your Identity: A Robust Voice Liveness Detection System for Smart Speakers
Yan Meng, Jiachun Li, Matthew Pillari +5
cs.CRcs.SDeess.ASarXiv:2510.24393v12025
Voice Separation with an Unknown Number of Multiple Speakers
Eliya Nachmani, Yossi Adi, Lior Wolf
eess.AScs.LGcs.SDarXiv:2003.01531v42020
NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers
Kai Shen, Zeqian Ju, Xu Tan +6
eess.AScs.AIcs.CLarXiv:2304.09116v32023
Music Gesture for Visual Sound Separation
Chuang Gan, Deng Huang, Hang Zhao +2
cs.CVcs.LGcs.MMarXiv:2004.09476v12020
M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversation
Vishal Chudasama, Purbayan Kar, Ashish Gudmalwar +3
cs.CVcs.SDeess.ASarXiv:2206.02187v12022
Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation
Elizabeth Salesky, Matthias Sperber, Alan W Black
cs.CLcs.SDeess.ASarXiv:1906.01199v12019
Efficient Neural Audio Synthesis
Nal Kalchbrenner, Erich Elsen, Karen Simonyan +7
cs.SDcs.LGeess.ASarXiv:1802.08435v22018
Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese
Shiyu Zhou, Linhao Dong, Shuang Xu +1
eess.AScs.CLcs.SDarXiv:1804.10752v22018
The NES Music Database: A multi-instrumental dataset with expressive performance attributes
Chris Donahue, Huanru Henry Mao, Julian McAuley
cs.SDcs.LGcs.NEarXiv:1806.04278v12018
Mitigating Attention Sinks and Massive Activations in Audio-Visual Speech Recognition with LLMs
Anand, Umberto Cappellazzo, Stavros Petridis +1
eess.AScs.CVcs.SDarXiv:2510.22603v32025
The 2018 Signal Separation Evaluation Campaign
Fabian-Robert Stöter, Antoine Liutkus, Nobutaka Ito
eess.AScs.SDarXiv:1804.06267v32018
Audio-Visual Speech Enhancement Using Multimodal Deep Convolutional Neural Networks
Jen-Cheng Hou, Syu-Siang Wang, Ying-Hui Lai +3
cs.SDcs.MMeess.ASarXiv:1709.00944v52017
Intermediate Loss Regularization for CTC-based Speech Recognition
Jaesong Lee, Shinji Watanabe
eess.AScs.CLcs.SDarXiv:2102.03216v12021
Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations
Ju-chieh Chou, Cheng-chieh Yeh, Hung-yi Lee +1
eess.AScs.CLcs.SDarXiv:1804.02812v22018
HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
Jungil Kong, Jaehyeon Kim, Jaekyoung Bae
cs.SDcs.LGeess.ASarXiv:2010.05646v22020
Inaudible Adversarial Perturbation: Manipulating the Recognition of User Speech in Real Time
Xinfeng Li, Chen Yan, Xuancun Lu +3
cs.CRcs.SDeess.ASarXiv:2308.01040v32023
PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Network
Dacheng Yin, Chong Luo, Zhiwei Xiong +1
cs.SDeess.ASarXiv:1911.04697v12019
TurnFSM for Full-Duplex Dialogue System: Internalizing State-Machine Logic for Streaming Semantic Voice Activity Detection and Utterance-Level Rejection
Zhiwei Lin, Tianjiao Du, Qiaochu Huang +6
eess.AScs.SDarXiv:2609.04240v12026
MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement
Szu-Wei Fu, Chien-Feng Liao, Yu Tsao +1
cs.SDcs.LGeess.ASarXiv:1905.04874v12019
ESPnet-ST: All-in-One Speech Translation Toolkit
Hirofumi Inaguma, Shun Kiyono, Kevin Duh +4
cs.CLcs.SDeess.ASarXiv:2004.10234v22020
An analysis of incorporating an external language model into a sequence-to-sequence model
Anjuli Kannan, Yonghui Wu, Patrick Nguyen +3
eess.AScs.AIcs.CLarXiv:1712.01996v12017
MelNet: A Generative Model for Audio in the Frequency Domain
Sean Vasquez, Mike Lewis
eess.AScs.LGcs.SDarXiv:1906.01083v12019
Listen to Look: Action Recognition by Previewing Audio
Ruohan Gao, Tae-Hyun Oh, Kristen Grauman +1
cs.CVcs.LGcs.SDarXiv:1912.04487v32019
DeepMUSIC: Multiple Signal Classification via Deep Learning
Ahmet M. Elbir
eess.SPcs.LGeess.ASarXiv:1912.04357v32019
SpeechEditBench: A Bilingual Multi-Attribute Benchmark for Instruction-Guided Speech Editing
Hanlin Zhang, Daxin Tan, Dehua Tao +3
eess.AScs.SDarXiv:2606.01804v22026
MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
Szu-Wei Fu, Cheng Yu, Tsun-An Hsieh +4
cs.SDcs.AIeess.ASarXiv:2104.03538v22021
UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022
Takaaki Saeki, Detai Xin, Wataru Nakata +3
cs.SDeess.ASarXiv:2204.02152v22022
Steering Autoregressive Music Generation with Recursive Feature Machines
Daniel Zhao, Daniel Beaglehole, Taylor Berg-Kirkpatrick +2
cs.LGcs.AIcs.SDarXiv:2510.19127v22025
RIR-Mega: a large-scale simulated room impulse response dataset for machine learning and room acoustics modeling
Mandip Goswami
eess.ASeess.SParXiv:2510.18917v22025

Audio and Speech Processing

Disentangling by Partitioning: A Representation Learning Framework for Multimodal Sensory Data

Lung Sound Classification Using Co-tuning and Stochastic Normalization

V2TATC: A Joint Voice-Trajectory Embedding Framework and Dataset for Air Traffic Controller Situational Awareness

Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation

Summary of the ChinaVoices Challenge 2026: Data, Tasks, Baseline, and Methods

Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis

Step-Audio-EditX Technical Report

End-to-End Adversarial Text-to-Speech

Improving Generalization of Transformer for Speech Recognition with Parallel Schedule Sampling and Relative Positional Embedding

Replay and Synthetic Speech Detection with Res2net Architecture

A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection

Nudging Hidden States: Training-Free Model Steering for Chain-of-Thought Reasoning in Large Audio-Language Models

EEG-based Auditory Attention Decoding: Towards Neuro-Steered Hearing Devices

SALMONN: Towards Generic Hearing Abilities for Large Language Models

Recent Progresses in Deep Learning based Acoustic Models (Updated)

FedNST: Federated Noisy Student Training for Automatic Speech Recognition

Targeted Adversarial Examples for Black Box Audio Systems

On the Comparison of Popular End-to-End Models for Large Scale Speech Recognition

fairseq S2T: Fast Speech-to-Text Modeling with fairseq

Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Dataset

Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music

Convolutional-Recurrent Neural Networks for Speech Enhancement

Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention

Co-Separating Sounds of Visual Objects

Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model

BigVGAN: A Universal Neural Vocoder with Large-Scale Training

x-vectors meet emotions: A study on dependencies between emotion and speaker recognition

R-Transformer: Recurrent Neural Network Enhanced Transformer

A Real-Time Wideband Neural Vocoder at 1.6 kb/s Using LPCNet

Universal Speech Enhancement with Score-based Diffusion

Weakly Supervised Tabla Stroke Transcription via an Adaptive Dynamic Rhythm Language Model (ADRM)

Your Microphone Array Retains Your Identity: A Robust Voice Liveness Detection System for Smart Speakers

Voice Separation with an Unknown Number of Multiple Speakers

NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers

Music Gesture for Visual Sound Separation

M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversation

Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation

Efficient Neural Audio Synthesis

Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese

The NES Music Database: A multi-instrumental dataset with expressive performance attributes

Mitigating Attention Sinks and Massive Activations in Audio-Visual Speech Recognition with LLMs

The 2018 Signal Separation Evaluation Campaign

Audio-Visual Speech Enhancement Using Multimodal Deep Convolutional Neural Networks

Intermediate Loss Regularization for CTC-based Speech Recognition

Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Inaudible Adversarial Perturbation: Manipulating the Recognition of User Speech in Real Time

PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Network

TurnFSM for Full-Duplex Dialogue System: Internalizing State-Machine Logic for Streaming Semantic Voice Activity Detection and Utterance-Level Rejection

MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement

ESPnet-ST: All-in-One Speech Translation Toolkit

An analysis of incorporating an external language model into a sequence-to-sequence model

MelNet: A Generative Model for Audio in the Frequency Domain

Listen to Look: Action Recognition by Previewing Audio

DeepMUSIC: Multiple Signal Classification via Deep Learning

SpeechEditBench: A Bilingual Multi-Attribute Benchmark for Instruction-Guided Speech Editing

MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement

UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022

Steering Autoregressive Music Generation with Recursive Feature Machines

RIR-Mega: a large-scale simulated room impulse response dataset for machine learning and room acoustics modeling