Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

1,021 to 1,080 of 1,648

The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech
Kaito Baba, Wataru Nakata, Yuki Saito +1
cs.SDcs.AIcs.LGarXiv:2409.09305v12024
Unsupervised Learning of Semantic Audio Representations
Aren Jansen, Manoj Plakal, Ratheet Pandya +5
cs.SDeess.ASstat.MLarXiv:1711.02209v12017
Masked Audio Generation using a Single Non-Autoregressive Transformer
Alon Ziv, Itai Gat, Gael Le Lan +6
cs.SDcs.AIcs.LGarXiv:2401.04577v22024
Less can be More: What Aspects of Speech Drive End-of-Turn Detection
Rini Sharon, Manickavela A, Kadri Hacioglu +1
eess.AScs.AIcs.SDarXiv:2609.11066v12026
Diarization Error Decomposition Under Pause Annotation Ambiguity
Shota Horiguchi, Marc Delcroix, Naohiro Tawara +1
eess.AScs.SDarXiv:2609.11007v22026
MuLan: A Joint Embedding of Music Audio and Natural Language
Qingqing Huang, Aren Jansen, Joonseok Lee +3
eess.AScs.CLcs.SDarXiv:2208.12415v12022
Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction
Lujia Bao, Qian Chen, Luyao Cheng +15
eess.AScs.AIcs.CLarXiv:2609.25176v22026
Audiobox: Unified Audio Generation with Natural Language Prompts
Apoorv Vyas, Bowen Shi, Matthew Le +21
cs.SDcs.LGeess.ASarXiv:2312.15821v12023
Pushing the Boundaries of Streaming Multi-Speaker ASR: A Systematic Study of Architectural Trade-offs
Taejin Park, Ivan Medennikov, Kunal Dhawan +3
eess.AScs.SDarXiv:2609.10265v22026
DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment
Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu +25
eess.AScs.CLcs.SDarXiv:2507.02768v22025
Audio Interaction Model
Zhifei Xie, Zihang Liu, Ze An +8
cs.SDcs.AIcs.CLarXiv:2606.05121v12026
FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion
Jingyi li, Weiping tu, Li xiao
cs.SDcs.LGeess.ASarXiv:2210.15418v12022
Mamba in Speech: Towards an Alternative to Self-Attention
Xiangyu Zhang, Qiquan Zhang, Hexin Liu +6
eess.AScs.SDarXiv:2405.12609v62024
From 2.1M Hours of Raw Broadcast Audio to Finnish Speech SSL: Confidence Filtering and Architectural Analysis at Scale
Yaroslav Getman, Tamás Grósz, Tommi Lehtonen +1
cs.CLcs.SDeess.ASarXiv:2610.13102v12026
Semantic-preserved Communication System for Highly Efficient Speech Transmission
Tianxiao Han, Qianqian Yang, Zhiguo Shi +2
eess.AScs.SDarXiv:2205.12727v12022
Self-training and Pre-training are Complementary for Speech Recognition
Qiantong Xu, Alexei Baevski, Tatiana Likhomanenko +5
cs.LGcs.SDeess.ASarXiv:2010.11430v12020
MRMAD: A Multi-Round Multi-Audio Benchmark for Evaluating Acoustic Degradation Perception in Large Audio-Language Models
Yize Li, Ningyuan Yang, Sile Yin +6
cs.SDcs.LGeess.ASarXiv:2608.22236v12026
Diffsound: Discrete Diffusion Model for Text-to-sound Generation
Dongchao Yang, Jianwei Yu, Helin Wang +4
cs.SDcs.AIeess.ASarXiv:2207.09983v22022
Beyond Localisation Accuracy: Sensorimotor Effects of HRTF Individualisation
Fulvio Missoni, Katarina C. Poole, Tim Murray-Browne +2
eess.ASarXiv:2609.08422v12026
Spirit LM: Interleaved Spoken and Written Language Model
Tu Anh Nguyen, Benjamin Muller, Bokai Yu +13
cs.CLcs.SDeess.ASarXiv:2402.05755v22024
Toward Part-Aware Choral Transcription with singing voice assignment
Hanyu Meng, Zhanhong He, Zixun Guo +1
eess.AScs.SDarXiv:2610.09861v12026
Neural source-filter-based waveform model for statistical parametric speech synthesis
Xin Wang, Shinji Takaki, Junichi Yamagishi
eess.AScs.SDstat.MLarXiv:1810.11946v42018
Noise2Music: Text-conditioned Music Generation with Diffusion Models
Qingqing Huang, Daniel S. Park, Tao Wang +12
cs.SDcs.LGeess.ASarXiv:2302.03917v22023
Music Transformer
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit +7
cs.LGcs.SDeess.ASarXiv:1809.04281v32018
Over-Tightening-Aware Pseudo-Labeling for Tight-Boundary Speaker Diarization
Shota Horiguchi, Takanori Ashihara, Marc Delcroix +2
eess.AScs.SDarXiv:2609.09965v12026
EmphTTS: an emphasis-control TTS with reinforcement learning
Zirui Li, Rech Silas, Lauri Juvela +2
eess.ASarXiv:2609.27599v12026
Modeling, Scaling, and Decoding: Optimizing Controllable Speech Generation with Nonverbal Vocalizations
Ziyu Zhang, Yun Chen, Taihui Wang +5
eess.AScs.AIarXiv:2609.14231v12026
Fast Timing-Conditioned Latent Audio Diffusion
Zach Evans, CJ Carr, Josiah Taylor +2
cs.SDcs.LGeess.ASarXiv:2402.04825v32024
GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities
Sreyan Ghosh, Sonal Kumar, Ashish Seth +6
cs.SDcs.AIcs.CLarXiv:2406.11768v12024
One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization
Ju-chieh Chou, Cheng-chieh Yeh, Hung-yi Lee
cs.LGcs.SDeess.ASarXiv:1904.05742v42019
LP-MusicCaps: LLM-Based Pseudo Music Captioning
SeungHeon Doh, Keunwoo Choi, Jongpil Lee +1
cs.SDcs.IRcs.MMarXiv:2307.16372v12023
Open-Set Vessel Re-Identification from Underwater Ship-Radiated Noise with a Raw-Waveform Selective-Kernel Acoustic Neural Network (SKANN) and a Cross-Passage Evaluation Protocol
Sunil Tyagi
eess.AScs.SDarXiv:2609.07399v12026
FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement
Jun Chen, Zilin Wang, Deyi Tuo +3
cs.SDcs.AIeess.ASarXiv:2203.12188v22022
SteerablePlex: Can We Steer Full-Duplex Models?
Haolong Zheng, Maike Züfle, Dominik Macháček +6
eess.AScs.CLarXiv:2610.12201v12026
Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech
Chien-yu Huang, Ke-Han Lu, Shih-Heng Wang +12
eess.AScs.LGcs.SDarXiv:2309.09510v22023
AudioPaLM: A Large Language Model That Can Speak and Listen
Paul K. Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen +27
cs.CLcs.AIcs.SDarXiv:2306.12925v12023
Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering
Gang Li, Jizhong Liu, Heinrich Dinkel +3
cs.SDcs.AIcs.CLarXiv:2503.11197v42025
Direct speech-to-speech translation with a sequence-to-sequence model
Ye Jia, Ron J. Weiss, Fadi Biadsy +4
cs.CLcs.LGcs.SDarXiv:1904.06037v22019
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities
Zhifeng Kong, Arushi Goel, Rohan Badlani +3
cs.SDcs.LGeess.ASarXiv:2402.01831v32024
WavJourney: Compositional Audio Creation with Large Language Models
Xubo Liu, Zhongkai Zhu, Haohe Liu +8
cs.SDcs.AIcs.MMarXiv:2307.14335v22023
DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
Jinglin Liu, Chengxi Li, Yi Ren +2
eess.AScs.LGcs.SDarXiv:2105.02446v62021
UniStream: Multi-Expert Residual Vector Quantization for 48 kHz Causal Streaming Audio Coding
Mingyu Zhao, Zhiyong Wu
eess.ASarXiv:2609.09866v12026
Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens
Xinsheng Wang, Mingqi Jiang, Ziyang Ma +22
cs.SDcs.AIeess.ASarXiv:2503.01710v12025
Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesis
Daisy Stanton, Yuxuan Wang, RJ Skerry-Ryan
cs.CLcs.LGcs.SDarXiv:1808.01410v12018
CASM: Context-Aware Semi-Markov Post-Processor for Beat Tracking
Zhanhong He, Hanyu Meng, Yaolong Ju
eess.AScs.SDarXiv:2610.09318v12026
A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection
Xin Wang, Junich Yamagishi
eess.ASarXiv:2103.11326v22021
Beyond Decodability: Do Acoustic Factors Drive Predictions in Speech-Based Alzheimer's Assessment?
Serli Kopar, Alkis Koudounas, Roshan P. Rane +3
cs.SDcs.CLcs.LGarXiv:2610.01846v12026
Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation
Zhifei Xie, Kaiyu Pang, Haobin Zhang +4
cs.SDcs.AIcs.CLarXiv:2605.19833v12026
YuE: Scaling Open Foundation Models for Long-Form Music Generation
Ruibin Yuan, Hanfeng Lin, Shuyue Guo +55
eess.AScs.AIcs.MMarXiv:2503.08638v22025
Self-Supervised Generation of Spatial Audio for 360 Video
Pedro Morgado, Nuno Vasconcelos, Timothy Langlois +1
cs.SDcs.CVcs.LGarXiv:1809.02587v12018
Jasper: An End-to-End Convolutional Neural Acoustic Model
Jason Li, Vitaly Lavrukhin, Boris Ginsburg +5
eess.AScs.CLcs.LGarXiv:1904.03288v32019
CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction
Yinghao Ma, Haiwen Xia, Hewei Gao +9
cs.SDcs.AIcs.LGarXiv:2603.00610v32026
Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
Ke Lei, Yu Zhang, Changhao Pan +4
eess.AScs.MMcs.SDarXiv:2605.30940v12026
ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild
Xuechen Liu, Xin Wang, Md Sahidullah +8
cs.SDcs.CRcs.MMarXiv:2210.02437v32022
Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing
Mengqi Wang, Zhan Liu, Zengrui Jin +3
eess.AScs.AIcs.SDarXiv:2509.16622v32025
ASVspoof 5: Design, Collection and Validation of Resources for Spoofing, Deepfake, and Adversarial Attack Detection Using Crowdsourced Speech
Xin Wang, Héctor Delgado, Hemlata Tak +26
eess.ASarXiv:2502.08857v42025
Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network
Sharath Adavanne, Archontis Politis, Tuomas Virtanen
cs.SDcs.LGeess.ASarXiv:1710.10059v22017
VoxReason: Auditing Source-Grounded Speech Plans Before Synthesis
Mengzhe Geng
cs.SDcs.CLcs.LGarXiv:2609.03203v32026
Phoneme-Guided Initialization for LLM-based Speech Recognition
Ryo Magoshi, Shinsuke Sakai, Tatsuya Kawahara
eess.AScs.CLcs.SDarXiv:2610.08994v12026
Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities
Guan-Ting Lin, Jiachen Lian, Tingle Li +4
cs.CLeess.ASarXiv:2503.04721v32025

Audio and Speech Processing

The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech

Unsupervised Learning of Semantic Audio Representations

Masked Audio Generation using a Single Non-Autoregressive Transformer

Less can be More: What Aspects of Speech Drive End-of-Turn Detection

Diarization Error Decomposition Under Pause Annotation Ambiguity

MuLan: A Joint Embedding of Music Audio and Natural Language

Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction

Audiobox: Unified Audio Generation with Natural Language Prompts

Pushing the Boundaries of Streaming Multi-Speaker ASR: A Systematic Study of Architectural Trade-offs

DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment

Audio Interaction Model

FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion

Mamba in Speech: Towards an Alternative to Self-Attention

From 2.1M Hours of Raw Broadcast Audio to Finnish Speech SSL: Confidence Filtering and Architectural Analysis at Scale

Semantic-preserved Communication System for Highly Efficient Speech Transmission

Self-training and Pre-training are Complementary for Speech Recognition

MRMAD: A Multi-Round Multi-Audio Benchmark for Evaluating Acoustic Degradation Perception in Large Audio-Language Models

Diffsound: Discrete Diffusion Model for Text-to-sound Generation

Beyond Localisation Accuracy: Sensorimotor Effects of HRTF Individualisation

Spirit LM: Interleaved Spoken and Written Language Model

Toward Part-Aware Choral Transcription with singing voice assignment

Neural source-filter-based waveform model for statistical parametric speech synthesis

Noise2Music: Text-conditioned Music Generation with Diffusion Models

Music Transformer

Over-Tightening-Aware Pseudo-Labeling for Tight-Boundary Speaker Diarization

EmphTTS: an emphasis-control TTS with reinforcement learning

Modeling, Scaling, and Decoding: Optimizing Controllable Speech Generation with Nonverbal Vocalizations

Fast Timing-Conditioned Latent Audio Diffusion

GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities

One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization

LP-MusicCaps: LLM-Based Pseudo Music Captioning

Open-Set Vessel Re-Identification from Underwater Ship-Radiated Noise with a Raw-Waveform Selective-Kernel Acoustic Neural Network (SKANN) and a Cross-Passage Evaluation Protocol

FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement

SteerablePlex: Can We Steer Full-Duplex Models?

Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech

AudioPaLM: A Large Language Model That Can Speak and Listen

Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering

Direct speech-to-speech translation with a sequence-to-sequence model

Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities

WavJourney: Compositional Audio Creation with Large Language Models

DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism

UniStream: Multi-Expert Residual Vector Quantization for 48 kHz Causal Streaming Audio Coding

Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens

Predicting Expressive Speaking Style From Text In End-To-End Speech Synthesis

CASM: Context-Aware Semi-Markov Post-Processor for Beat Tracking

A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection

Beyond Decodability: Do Acoustic Factors Drive Predictions in Speech-Based Alzheimer's Assessment?

Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulation

YuE: Scaling Open Foundation Models for Long-Form Music Generation

Self-Supervised Generation of Spatial Audio for 360 Video

Jasper: An End-to-End Convolutional Neural Acoustic Model

CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction

Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer

ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild

Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing

ASVspoof 5: Design, Collection and Validation of Resources for Spoofing, Deepfake, and Adversarial Attack Detection Using Crowdsourced Speech

Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network

VoxReason: Auditing Source-Grounded Speech Plans Before Synthesis

Phoneme-Guided Initialization for LLM-based Speech Recognition

Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities