Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

1,441 to 1,500 of 1,641

ToyADMOS: A Dataset of Miniature-Machine Operating Sounds for Anomalous Sound Detection
Yuma Koizumi, Shoichiro Saito, Hisashi Uematsu +2
eess.AScs.LGcs.SDarXiv:1908.03299v12019
SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models
Yuxun Tang, Yuning Wu, Jiatong Shi +1
cs.SDeess.ASarXiv:2406.08905v22024
CLAP: Learning Audio Concepts From Natural Language Supervision
Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail +1
cs.SDeess.ASarXiv:2206.04769v12022
Augmenting Generative Adversarial Networks for Speech Emotion Recognition
Siddique Latif, Muhammad Asim, Rajib Rana +3
cs.SDeess.ASarXiv:2005.08447v32020
BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec
Detai Xin, Xu Tan, Shinnosuke Takamichi +1
eess.AScs.SDarXiv:2409.05377v12024
AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIAN
Zewang Zhang, Qiao Tian, Heng Lu +2
cs.SDcs.CLeess.ASarXiv:2005.05642v12020
Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS
Mutian He, Yan Deng, Lei He
cs.CLcs.LGcs.SDarXiv:1906.00672v32019
Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation
Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi +2
eess.AScs.SDarXiv:2204.12260v12022
Seeing wake words: Audio-visual Keyword Spotting
Liliane Momeni, Triantafyllos Afouras, Themos Stafylakis +2
cs.CVeess.ASarXiv:2009.01225v12020
Taming Data and Transformers for Audio Generation
Moayed Haji-Ali, Willi Menapace, Aliaksandr Siarohin +2
cs.SDcs.CLcs.CVarXiv:2406.19388v42024
Masked Spectrogram Prediction For Self-Supervised Audio Pre-Training
Dading Chong, Helin Wang, Peilin Zhou +1
cs.SDeess.ASarXiv:2204.12768v12022
Sketch2Sound: Controllable Audio Generation via Time-Varying Signals and Sonic Imitations
Hugo Flores García, Oriol Nieto, Justin Salamon +2
cs.SDeess.ASarXiv:2412.08550v22024
MAE-AST: Masked Autoencoding Audio Spectrogram Transformer
Alan Baade, Puyuan Peng, David Harwath
eess.AScs.AIcs.CLarXiv:2203.16691v12022
Generating Visually Aligned Sound from Videos
Peihao Chen, Yang Zhang, Mingkui Tan +3
cs.CVcs.SDeess.ASarXiv:2008.00820v12020
Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation
Jinlong Xue, Yayue Deng, Yingming Gao +1
cs.SDcs.AIcs.CLarXiv:2401.01044v12024
SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generation
Youliang Zhang, Zhaoyang Li, Duomin Wang +6
cs.CVeess.ASarXiv:2507.09862v12025
HunyuanVideo-Foley: Multimodal Diffusion with Representation Alignment for High-Fidelity Foley Audio Generation
Sizhe Shan, Qiulin Li, Yutao Cui +5
eess.AScs.CVcs.SDarXiv:2508.16930v12025
Summary on The Multilingual Conversational Speech Language Model Challenge: Datasets, Tasks, Baselines, and Methods
Bingshen Mu, Pengcheng Guo, Zhaokai Sun +8
eess.AScs.SDarXiv:2509.13785v12025
When Creators Meet the Metaverse: A Survey on Computational Arts
Lik-Hang Lee, Zijun Lin, Rui Hu +5
cs.CYcs.AIcs.LGarXiv:2111.13486v12021
Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition
Ye Bai, Jingping Chen, Jitong Chen +52
eess.AScs.SDarXiv:2407.04675v22024
JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Models
Zifan Peng, Yule Liu, Zhen Sun +9
cs.CRcs.AIcs.SDarXiv:2505.17568v32025
Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation
Qiuqiang Kong, Yin Cao, Haohe Liu +2
cs.SDeess.ASarXiv:2109.05418v12021
Training Neural Speech Recognition Systems with Synthetic Speech Augmentation
Jason Li, Ravi Gadde, Boris Ginsburg +1
cs.CLcs.LGcs.SDarXiv:1811.00707v12018
Glottal Closure and Opening Instant Detection from Speech Signals
Thomas Drugman, Thierry Dutoit
cs.SDcs.CLeess.ASarXiv:2001.00841v12019
Data Augmenting Contrastive Learning of Speech Representations in the Time Domain
Eugene Kharitonov, Morgane Rivière, Gabriel Synnaeve +4
eess.AScs.CLcs.SDarXiv:2007.00991v12020
StepAudio 3 Realtime Technical Report
Bin Lin, Bo Zhao, Boyang Zhang +87
cs.SDeess.ASarXiv:2609.14005v12026
OV-InstructTTS: Towards Open-Vocabulary Instruct Text-to-Speech
Yong Ren, Jiangyan Yi, Jianhua Tao +5
cs.SDeess.ASarXiv:2601.01459v12026
SoulX-Duplug: Plug-and-Play Streaming State Prediction Module for Realtime Full-Duplex Speech Conversation
Ruiqi Yan, Wenxi Chen, Zhanxun Liu +17
eess.ASarXiv:2603.14877v12026
Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models
Guan-Ting Lin, Shih-Yun Shan Kuan, Qirui Wang +4
eess.ASarXiv:2507.23159v42025
CN-CELEB: a challenging Chinese speaker recognition dataset
Yue Fan, Jiawen Kang, Lantian Li +7
eess.AScs.CLcs.SDarXiv:1911.01799v12019
Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM
Szu-Wei Fu, Yu Tsao, Hsin-Te Hwang +1
cs.SDcs.AIeess.ASarXiv:1808.05344v22018
Detecting cognitive decline using speech only: The ADReSSo Challenge
Saturnino Luz, Fasih Haider, Sofia de la Fuente +2
eess.AScs.CLcs.LGarXiv:2104.09356v12021
Learning Alignment for Multimodal Emotion Recognition from Speech
Haiyang Xu, Hui Zhang, Kun Han +3
cs.CLcs.SDeess.ASarXiv:1909.05645v22019
Recurrent Neural Network Transducer for Audio-Visual Speech Recognition
Takaki Makino, Hank Liao, Yannis Assael +4
eess.AScs.CLcs.CVarXiv:1911.04890v12019
Speech Recognition with Augmented Synthesized Speech
Andrew Rosenberg, Yu Zhang, Bhuvana Ramabhadran +4
cs.CLcs.SDeess.ASarXiv:1909.11699v12019
Audio Deepfake Detection: A Survey
Jiangyan Yi, Chenglong Wang, Jianhua Tao +3
cs.SDeess.ASarXiv:2308.14970v12023
Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam
Marc Delcroix, Tsubasa Ochiai, Katerina Zmolikova +4
eess.AScs.CLcs.SDarXiv:2001.08378v12020
Xiaomi-CocktailASR-1 Technical Report
Yiru Zhang, Hang Su, Lichun Fan +10
cs.SDcs.CLeess.ASarXiv:2609.11274v12026
ASR is all you need: cross-modal distillation for lip reading
Triantafyllos Afouras, Joon Son Chung, Andrew Zisserman
cs.CVcs.SDeess.ASarXiv:1911.12747v22019
DeepFilterNet: A Low Complexity Speech Enhancement Framework for Full-Band Audio based on Deep Filtering
Hendrik Schröter, Alberto N. Escalante-B., Tobias Rosenkranz +1
eess.AScs.LGeess.SParXiv:2110.05588v22021
X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs
Di Cao, Dongjie Fu, Hai Yu +3
eess.AScs.AIcs.CLarXiv:2603.24596v32026
Understanding Optical Music Recognition
Jorge Calvo-Zaragoza, Jan Hajič, Alexander Pacha
cs.CVcs.AIcs.IRarXiv:1908.03608v32019
AVSRBench: A Multi-Condition AVSR Benchmark
Rishabh Jain, Naomi Harte
eess.AScs.CVcs.MMarXiv:2609.10366v12026
EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation
Hsiao-Tzu Hung, Joann Ching, Seungheon Doh +3
cs.SDcs.MMeess.ASarXiv:2108.01374v12021
Continuous Speech Separation with Conformer
Sanyuan Chen, Yu Wu, Zhuo Chen +6
eess.AScs.CLarXiv:2008.05773v22020
Orukeet: Multilingual ASR with Frozen Gabor Kernels
Nathan Roll, Irene Yi, Büşra Marşan +6
cs.SDcs.LGeess.ASarXiv:2609.10054v12026
Wavenet based low rate speech coding
W. Bastiaan Kleijn, Felicia S. C. Lim, Alejandro Luebs +4
eess.AScs.SDeess.SParXiv:1712.01120v12017
Transfer Learning from Adult to Children for Speech Recognition: Evaluation, Analysis and Recommendations
Prashanth Gurunath Shivakumar, Panayiotis Georgiou
eess.AScs.CLcs.SDarXiv:1805.03322v12018
Iterative Pseudo-Labeling for Speech Recognition
Qiantong Xu, Tatiana Likhomanenko, Jacob Kahn +3
cs.CLcs.SDeess.ASarXiv:2005.09267v22020
From Audio to Semantics: Approaches to end-to-end spoken language understanding
Parisa Haghani, Arun Narayanan, Michiel Bacchiani +6
eess.AScs.CLcs.SDarXiv:1809.09190v12018
ICASSP 2021 Deep Noise Suppression Challenge
Chandan K A Reddy, Harishchandra Dubey, Vishak Gopal +5
eess.ASarXiv:2009.06122v22020
CN-Celeb: multi-genre speaker recognition
Lantian Li, Ruiqi Liu, Jiawen Kang +6
cs.SDeess.ASarXiv:2012.12468v22020
ConversationalVoice: Full-Duplex Speech Data from Real Conversations through Source-Faithful Reconstruction and Conversation-Grounded Expansion
Richard Yucheng He, Baodong Cao, Chen Xu +2
cs.CLcs.SDeess.ASarXiv:2609.08147v12026
Foley Music: Learning to Generate Music from Videos
Chuang Gan, Deng Huang, Peihao Chen +2
cs.CVcs.LGcs.SDarXiv:2007.10984v12020
SpecAugment on Large Scale Datasets
Daniel S. Park, Yu Zhang, Chung-Cheng Chiu +5
eess.AScs.CLcs.LGarXiv:1912.05533v12019
Multi-level Attention network using text, audio and video for Depression Prediction
Anupama Ray, Siddharth Kumar, Rutvik Reddy +2
cs.CVeess.ASarXiv:1909.01417v12019
Transformer-Transducer: End-to-End Speech Recognition with Self-Attention
Ching-Feng Yeh, Jay Mahadeokar, Kaustubh Kalgaonkar +6
eess.AScs.CLcs.SDarXiv:1910.12977v12019
Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens
Rafael Valle, Jason Li, Ryan Prenger +1
cs.SDcs.LGeess.ASarXiv:1910.11997v12019
MLAAD: The Multi-Language Audio Anti-Spoofing Dataset
Nicolas M. Müller, Piotr Kawa, Wei Herng Choong +6
cs.SDeess.ASarXiv:2401.09512v112024
The Third DIHARD Diarization Challenge
Neville Ryant, Prachi Singh, Venkat Krishnamohan +6
eess.AScs.SDarXiv:2012.01477v32020

Audio and Speech Processing

ToyADMOS: A Dataset of Miniature-Machine Operating Sounds for Anomalous Sound Detection

SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models

CLAP: Learning Audio Concepts From Natural Language Supervision

Augmenting Generative Adversarial Networks for Speech Emotion Recognition

BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec

AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIAN

Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTS

Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation

Seeing wake words: Audio-visual Keyword Spotting

Taming Data and Transformers for Audio Generation

Masked Spectrogram Prediction For Self-Supervised Audio Pre-Training

Sketch2Sound: Controllable Audio Generation via Time-Varying Signals and Sonic Imitations

MAE-AST: Masked Autoencoding Audio Spectrogram Transformer

Generating Visually Aligned Sound from Videos

Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation

SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generation

HunyuanVideo-Foley: Multimodal Diffusion with Representation Alignment for High-Fidelity Foley Audio Generation

Summary on The Multilingual Conversational Speech Language Model Challenge: Datasets, Tasks, Baselines, and Methods

When Creators Meet the Metaverse: A Survey on Computational Arts

Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition

JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Models

Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation

Training Neural Speech Recognition Systems with Synthetic Speech Augmentation

Glottal Closure and Opening Instant Detection from Speech Signals

Data Augmenting Contrastive Learning of Speech Representations in the Time Domain

StepAudio 3 Realtime Technical Report

OV-InstructTTS: Towards Open-Vocabulary Instruct Text-to-Speech

SoulX-Duplug: Plug-and-Play Streaming State Prediction Module for Realtime Full-Duplex Speech Conversation

Full-Duplex-Bench v1.5: Evaluating Overlap Handling for Full-Duplex Speech Models

CN-CELEB: a challenging Chinese speaker recognition dataset

Quality-Net: An End-to-End Non-intrusive Speech Quality Assessment Model based on BLSTM

Detecting cognitive decline using speech only: The ADReSSo Challenge

Learning Alignment for Multimodal Emotion Recognition from Speech

Recurrent Neural Network Transducer for Audio-Visual Speech Recognition

Speech Recognition with Augmented Synthesized Speech

Audio Deepfake Detection: A Survey

Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam

Xiaomi-CocktailASR-1 Technical Report

ASR is all you need: cross-modal distillation for lip reading

DeepFilterNet: A Low Complexity Speech Enhancement Framework for Full-Band Audio based on Deep Filtering

X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs

Understanding Optical Music Recognition

AVSRBench: A Multi-Condition AVSR Benchmark

EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation

Continuous Speech Separation with Conformer

Orukeet: Multilingual ASR with Frozen Gabor Kernels

Wavenet based low rate speech coding

Transfer Learning from Adult to Children for Speech Recognition: Evaluation, Analysis and Recommendations

Iterative Pseudo-Labeling for Speech Recognition

From Audio to Semantics: Approaches to end-to-end spoken language understanding

ICASSP 2021 Deep Noise Suppression Challenge

CN-Celeb: multi-genre speaker recognition

ConversationalVoice: Full-Duplex Speech Data from Real Conversations through Source-Faithful Reconstruction and Conversation-Grounded Expansion

Foley Music: Learning to Generate Music from Videos

SpecAugment on Large Scale Datasets

Multi-level Attention network using text, audio and video for Depression Prediction

Transformer-Transducer: End-to-End Speech Recognition with Self-Attention

Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens

MLAAD: The Multi-Language Audio Anti-Spoofing Dataset

The Third DIHARD Diarization Challenge