Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

241 to 300 of 1,641

Grounded Decoding for Autoregressive Speech Enhancement via Adaptive Code-Space Grounding and Local LLM Refinement
Hao Shi, Yuan Gao, Zhaoheng Ni +4
cs.SDeess.ASarXiv:2609.04245v12026
FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integration
Kai-Tuo Xu, Feng-Long Xie, Xu Tang +1
eess.AScs.SDarXiv:2501.14350v12025
PADP: Perceptual Audio Data Perturbation for Probing Perception Awareness in Audio Quality Models
Guanxin Jiang, Andreas Brendel, Pablo M. Delgado +1
eess.ASarXiv:2610.01405v12026
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models
Zeqian Ju, Yuancheng Wang, Kai Shen +16
eess.AScs.AIcs.CLarXiv:2403.03100v32024
Source-Adaptive Data Curation for Bilingual NVV-Aware ASR
Yuang Cao, Qirui Zhan, Jingbin Hu +7
eess.AScs.SDarXiv:2609.09929v22026
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities
Sreyan Ghosh, Zhifeng Kong, Sonal Kumar +6
cs.SDcs.CLcs.LGarXiv:2503.03983v12025
Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners
Umberto Cappellazzo, Xubo Liu, Stavros Petridis +1
eess.AScs.AIcs.SDarXiv:2608.19863v12026
EAT: Self-Supervised Pre-Training with Efficient Audio Transformer
Wenxi Chen, Yuzhe Liang, Ziyang Ma +2
eess.AScs.AIcs.CLarXiv:2401.03497v12024
Exact Factorisation and Fast Computation of Invertible Constant-Q Transforms
Facundo Franchino, Eloi Moliner, Vesa Välimäki
eess.SPcs.SDeess.ASarXiv:2609.29119v12026
Frequency-Weighted Soft-Constrained Spatially Selective Active Noise Control for Open-Fitting Hearables
Tong Xiao, Reinhild Roden, Matthias Blau +1
eess.ASeess.SParXiv:2610.00721v12026
Low-Latency State Space Voice Activity Detection with Robust Onset Time Evaluation
Elad Cohen, Arnon Netzer, Hai Victor Habi
eess.ASarXiv:2609.11110v12026
[b]=[d]-[t]+[p]: Self-supervised Speech Models Discover Phonological Vector Arithmetic
Kwanghee Choi, Eunjung Yeo, Cheol Jun Cho +2
eess.AScs.CLcs.LGarXiv:2602.18899v32026
End-to-End Historical Music Restoration in Latent Space
Steven Cho, Junghyun Koo, Raphael Lafargue +3
eess.AScs.LGcs.SDarXiv:2610.00607v12026
VoxTubeS: Distributable Speaker-Anonymized Synthetic Speech Corpora and Their Analysis
Zhe Zhang, Yexin Lu, Junichi Yamagishi
eess.ASarXiv:2609.12432v12026
Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming
Zhifei Xie, Changqiao Wu
cs.AIcs.CLcs.HCarXiv:2408.16725v32024
EmoSteer-TTS: Fine-Grained and Training-Free Emotion-Controllable Text-to-Speech via Activation Steering
Tianxin Xie, Shan Yang, Chenxing Li +2
cs.SDcs.AIeess.ASarXiv:2508.03543v32025
VocalCoachBench: Benchmarking Audio-Language Models on Expert Feedback for Singing
Hayeon Bang, Hounsu Kim, Wonil Kim +1
cs.SDeess.ASarXiv:2609.04241v12026
Drax: Speech Recognition with Discrete Flow Matching
Aviv Navon, Aviv Shamsian, Neta Glazer +4
eess.AScs.LGcs.SDarXiv:2510.04162v12025
Mustango: Toward Controllable Text-to-Music Generation
Jan Melechovsky, Zixun Guo, Deepanway Ghosal +3
eess.ASarXiv:2311.08355v32023
Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency
Guan-Ting Lin, Chen Chen, Zhehuai Chen +1
eess.AScs.CLarXiv:2604.04847v12026
VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild
Puyuan Peng, Po-Yao Huang, Shang-Wen Li +2
eess.AScs.AIcs.CLarXiv:2403.16973v32024
StepAudio 3 Music Technical Report
Chengli Feng, Zhiyue Wu, Jiahao Song +10
eess.AScs.SDarXiv:2609.16034v12026
Likelihood-Constrained Acoustic Reranking for Training-Free Hallucination Mitigation in LLM-Based ASR
Jiasheng Kuang, Linru Zheng, Hongjin Song +2
eess.ASarXiv:2608.30776v12026
Speech Denoising with Deep Feature Losses
Francois G. Germain, Qifeng Chen, Vladlen Koltun
eess.AScs.SDarXiv:1806.10522v22018
CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens
Zhihao Du, Qian Chen, Shiliang Zhang +9
cs.SDcs.AIeess.ASarXiv:2407.05407v22024
Tacit-TTS: From Autoregressive Decoding to Masked Prediction for Efficient Transcript-Free Voice Cloning
Jian Chen, You Zhang, Mark Vinton
eess.AScs.CLcs.SDarXiv:2609.38658v12026
EMODY Flow: Emotion-Aware Audio-Driven Full-Body Motion Generation
Harsh Kumar Agarwal, Xavier Alameda-Pineda, Olivier Perrotin
cs.GRcs.CVcs.LGarXiv:2609.16011v12026
Enhancing Neural Speech Coding with Semantic and Visual Cues
Yao Guo, Yang Ai, Hui-Peng Du +3
eess.ASarXiv:2609.05076v12026
RAVE: A variational autoencoder for fast and high-quality neural audio synthesis
Antoine Caillon, Philippe Esling
cs.LGcs.SDeess.ASarXiv:2111.05011v22021
Disentangling Representation using Attributes-based Gaussian Estimation for Medical Sound Diagnosis
Ke Zhao
cs.AIcs.CVeess.ASarXiv:2608.29026v12026
A Review of Deep Learning Techniques for Speech Processing
Ambuj Mehrish, Navonil Majumder, Rishabh Bhardwaj +2
eess.ASarXiv:2305.00359v32023
DuplexGen: Decoupling Content, Timing, and Acoustics for Synthetic Dialogue Speech
Pengcheng Wang, Sheng Li, Jiyi Li +1
cs.CLeess.ASarXiv:2608.16053v12026
DAMSEP: Distance-Aware Monaural Source Separation using Multi-RIR Estimation
Wen Wen, Qiang Zhou, Yu Xi +3
eess.ASarXiv:2609.29749v12026
dots.tts Technical Report
Shi Lian, Changtao Li, Bohan Li +6
cs.SDcs.AIeess.ASarXiv:2606.07080v22026
FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs
Keyu An, Qian Chen, Chong Deng +30
cs.SDcs.AIeess.ASarXiv:2407.04051v32024
LLaMA-Omni: Seamless Speech Interaction with Large Language Models
Qingkai Fang, Shoutao Guo, Yan Zhou +3
cs.CLcs.AIcs.SDarXiv:2409.06666v22024
Enhancing Speech Recognition through Text-to-Speech and Voice Conversion Augmentation
Yunus Emre Ozkose, Ali Haznedaroglu
cs.SDeess.ASarXiv:2610.13662v12026
Emo-BPO: Emotion Bidirectional Preference Optimization for Diffusion-based Emotional TTS
Jiacheng Shi, Hongfei Du, Xinyuan Song +3
cs.SDeess.ASarXiv:2610.12958v12026
LiveSynth: A Streaming Neural Synthesizer for Instrument Cloning and Text-to-Instrument
Kyungsu Kim, Yejin Kim, Kyogu Lee
cs.SDeess.ASarXiv:2610.13054v12026
Music Flamingo: Scaling Music Understanding in Audio Language Models
Sreyan Ghosh, Arushi Goel, Lasha Koroshinadze +8
eess.AScs.CLarXiv:2511.10289v12025
FedCFM: Federated Continual Domain Generalization for Fake Speech Detection via Conditional Flow Matching
Yingjian Yu, Haiyan Guo, Tianshun Wang +4
eess.ASarXiv:2610.01242v12026
Investigating catastrophic forgetting in sound event classification
Riccardo Casciotti, Annamaria Mesaros
eess.AScs.AIcs.SDarXiv:2609.11447v12026
Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Models
Zhifei Xie, Ziyang Ma, Zihang Liu +7
cs.CLcs.AIcs.LGarXiv:2508.15827v22025
ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers
Kaizhi Qian, Yang Zhang, Heting Gao +5
cs.SDcs.AIeess.ASarXiv:2204.09224v22022
Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucination
Mohammad Khalooei, Mohammad Sabokrou
cs.AIcs.LGcs.SDarXiv:2610.13112v12026
Preference Optimization with LALM Feedback for Continuous Autoregressive Non-Verbal Vocalization Generation
Jingbin Hu, Qirui Zhan, Yuang Cao +7
eess.AScs.SDarXiv:2609.11260v22026
SonicVisionLM: Playing Sound with Vision Language Models
Zhifeng Xie, Shengye Yu, Qile He +1
cs.MMcs.SDeess.ASarXiv:2401.04394v32024
It's Raw! Audio Generation with State-Space Models
Karan Goel, Albert Gu, Chris Donahue +1
cs.SDcs.AIcs.LGarXiv:2202.09729v12022
U-PAST: A Phase-Aware Audio Spectrogram Transformer-U-Net for Single-Channel Speech Enhancement
Cao Duong Ly, Jörn Anemüller
eess.AScs.SDarXiv:2609.00431v12026
NVV-Locator: From Transcript Tags to Acoustic Boundaries for Fine-Grained Nonverbal Vocalization Grounding
Yuang Cao, Bingshen Mu, Zhennan Lin +7
eess.AScs.SDarXiv:2609.09940v32026
Simple and Effective Unsupervised Speech Synthesis
Alexander H. Liu, Cheng-I Jeff Lai, Wei-Ning Hsu +3
cs.SDcs.CLeess.ASarXiv:2204.02524v32022
UniAudio: An Audio Foundation Model Toward Universal Audio Generation
Dongchao Yang, Jinchuan Tian, Xu Tan +9
cs.SDeess.ASarXiv:2310.00704v62023
SURE-EVAL: A Systematic and Unified Agentic Framework for Reproducible Evaluation
Jing Peng, Junhao Du, Yixuan Wang +21
eess.ASarXiv:2609.39030v12026
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models
Simon Alexanderson, Rajmund Nagy, Jonas Beskow +1
cs.LGcs.CVcs.GRarXiv:2211.09707v22022
Neural Voice Cloning with a Few Samples
Sercan O. Arik, Jitong Chen, Kainan Peng +2
cs.CLcs.LGcs.SDarXiv:1802.06006v32018
Automatic Detection of Depression in Speech Using Ensemble Convolutional Neural Networks
Adrián Vázquez-Romero, Ascensión Gallardo-Antolín
eess.ASarXiv:2402.02830v12024
Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models
Xuanru Zhou, Yiwen Shao, Jiahong Li +1
cs.CLcs.SDeess.ASarXiv:2608.18132v12026
LEMAS: Large A 150K-Hour Large-scale Extensible Multilingual Audio Suite with Generative Speech Models
Zhiyuan Zhao, Lijian Lin, Ye Zhu +3
cs.SDeess.ASarXiv:2601.04233v12026
SonicCaps: Large-Scale Diverse and Fine-Grained Captioning for Improved Audio-Retrieval
Zineb Lahrichi, Marc Ferras, Gaël Richard +1
cs.SDcs.CLcs.MMarXiv:2609.02343v12026
S-JEPA : Soft Clustering Anchors for Self-Supervised Speech Representation Learning
Georgios Ioannides, Adrian Kieback, Judah Goldfeder +5
cs.SDeess.ASeess.SParXiv:2606.19398v12026

Audio and Speech Processing

Grounded Decoding for Autoregressive Speech Enhancement via Adaptive Code-Space Grounding and Local LLM Refinement

FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integration

PADP: Perceptual Audio Data Perturbation for Probing Perception Awareness in Audio Quality Models

NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models

Source-Adaptive Data Curation for Bilingual NVV-Aware ASR

Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners

EAT: Self-Supervised Pre-Training with Efficient Audio Transformer

Exact Factorisation and Fast Computation of Invertible Constant-Q Transforms

Frequency-Weighted Soft-Constrained Spatially Selective Active Noise Control for Open-Fitting Hearables

Low-Latency State Space Voice Activity Detection with Robust Onset Time Evaluation

[b]=[d]-[t]+[p]: Self-supervised Speech Models Discover Phonological Vector Arithmetic

End-to-End Historical Music Restoration in Latent Space

VoxTubeS: Distributable Speaker-Anonymized Synthetic Speech Corpora and Their Analysis

Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming

EmoSteer-TTS: Fine-Grained and Training-Free Emotion-Controllable Text-to-Speech via Activation Steering

VocalCoachBench: Benchmarking Audio-Language Models on Expert Feedback for Singing

Drax: Speech Recognition with Discrete Flow Matching

Mustango: Toward Controllable Text-to-Music Generation

Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency

VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild

StepAudio 3 Music Technical Report

Likelihood-Constrained Acoustic Reranking for Training-Free Hallucination Mitigation in LLM-Based ASR

Speech Denoising with Deep Feature Losses

CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens

Tacit-TTS: From Autoregressive Decoding to Masked Prediction for Efficient Transcript-Free Voice Cloning

EMODY Flow: Emotion-Aware Audio-Driven Full-Body Motion Generation

Enhancing Neural Speech Coding with Semantic and Visual Cues

RAVE: A variational autoencoder for fast and high-quality neural audio synthesis

Disentangling Representation using Attributes-based Gaussian Estimation for Medical Sound Diagnosis

A Review of Deep Learning Techniques for Speech Processing

DuplexGen: Decoupling Content, Timing, and Acoustics for Synthetic Dialogue Speech

DAMSEP: Distance-Aware Monaural Source Separation using Multi-RIR Estimation

dots.tts Technical Report

FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs

LLaMA-Omni: Seamless Speech Interaction with Large Language Models

Enhancing Speech Recognition through Text-to-Speech and Voice Conversion Augmentation

Emo-BPO: Emotion Bidirectional Preference Optimization for Diffusion-based Emotional TTS

LiveSynth: A Streaming Neural Synthesizer for Instrument Cloning and Text-to-Instrument

Music Flamingo: Scaling Music Understanding in Audio Language Models

FedCFM: Federated Continual Domain Generalization for Fake Speech Detection via Conditional Flow Matching

Investigating catastrophic forgetting in sound event classification

Mini-Omni-Reasoner: Token-Level Thinking-in-Speaking in Large Speech Models

ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers

Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucination

Preference Optimization with LALM Feedback for Continuous Autoregressive Non-Verbal Vocalization Generation

SonicVisionLM: Playing Sound with Vision Language Models

It's Raw! Audio Generation with State-Space Models

U-PAST: A Phase-Aware Audio Spectrogram Transformer-U-Net for Single-Channel Speech Enhancement

NVV-Locator: From Transcript Tags to Acoustic Boundaries for Fine-Grained Nonverbal Vocalization Grounding

Simple and Effective Unsupervised Speech Synthesis

UniAudio: An Audio Foundation Model Toward Universal Audio Generation

SURE-EVAL: A Systematic and Unified Agentic Framework for Reproducible Evaluation

Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models

Neural Voice Cloning with a Few Samples

Automatic Detection of Depression in Speech Using Ensemble Convolutional Neural Networks

Alignment Is All You Need: Instruction-Free Training for General Audio-Language Models

LEMAS: Large A 150K-Hour Large-scale Extensible Multilingual Audio Suite with Generative Speech Models

SonicCaps: Large-Scale Diverse and Fine-Grained Captioning for Improved Audio-Retrieval

S-JEPA : Soft Clustering Anchors for Self-Supervised Speech Representation Learning