Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

1,081 to 1,140 of 1,641

OpenPA: Forced-alignment-free Multi-granular Pronunciation Assessment with Mixed-transcript Adaptation
Fu-An Chao, Bi-Cheng Yan, Berlin Chen
eess.AScs.CLcs.SDarXiv:2610.13826v12026
Lossless Inference Acceleration for ASR via EAGLE-Style Speculative Decoding
Sen Zhang, Xianghu Yue, Yiming Chen +3
cs.SDeess.ASarXiv:2610.13506v12026
Translatotron 2: High-quality direct speech-to-speech translation with voice preservation
Ye Jia, Michelle Tadmor Ramanovich, Tal Remez +1
cs.CLcs.LGcs.SDarXiv:2107.08661v52021
Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module
Yihe Liu, Ziqi Yuan, Huisheng Mao +7
cs.MMcs.AIcs.CVarXiv:2209.02604v12022
AURA: Unified Multimodal Framework for Conversational Music Editing
Quoc-Huy Trinh, Minh-Van Nguyen, Debesh Jha
cs.SDcs.AIeess.ASarXiv:2609.14344v12026
Vector-Quantized Autoregressive Predictive Coding
Yu-An Chung, Hao Tang, James Glass
eess.AScs.CLcs.LGarXiv:2005.08392v12020
VoxAnon: A balanced dataset for a fair evaluation of speech anonymization
Orane Dufour, Paul Magron, Mickael Rouvier
cs.SDeess.ASarXiv:2610.13313v12026
Does Financial Speech Need Continual Learning Regularization? A Study of EWC and mini-KFAC on Earnings Call ASR
Thai Quoc On, Homayoon Beigi
eess.AScs.CLcs.LGarXiv:2610.13432v12026
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation
Qinglin Zhang, Luyao Cheng, Chong Deng +8
cs.CLcs.AIcs.SDarXiv:2410.17799v22024
OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academia
Xuelong Geng, Kun Wei, Qijie Shao +18
cs.SDcs.CLeess.ASarXiv:2501.13306v22025
Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis
Guangzhi Sun, Yu Zhang, Ron J. Weiss +3
eess.AScs.LGcs.SDarXiv:2002.03785v12020
RDSE: Real-Time Directional Speech Enhancement Framework for Dual-Mic Commodity Smartphones
Chinmay Deo, Danish Mahajan, Soudeep Ghoshal +1
cs.SDeess.ASarXiv:2610.12476v12026
CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation
Zineng Tang, Ziyi Yang, Mahmoud Khademi +3
cs.CVcs.AIcs.CLarXiv:2311.18775v12023
How Should We Evaluate Naturalness for Synthetic Speech with a Foreign Accent?
Kentaro Onda, Satoru Fukayama, Daisuke Saito +1
cs.SDeess.ASarXiv:2610.13175v12026
RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition
Peiyan Dong, Siyue Wang, Wei Niu +8
cs.SDcs.LGeess.ASarXiv:2002.11474v12020
Improving Speech-to-Speech Translation Through Unlabeled Text
Xuan-Phi Nguyen, Sravya Popuri, Changhan Wang +3
cs.CLcs.SDeess.ASarXiv:2210.14514v12022
NELS -- Never-Ending Learner of Sounds
Benjamin Elizalde, Rohan Badlani, Ankit Shah +2
cs.SDeess.ASarXiv:1801.05544v22018
DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors
Keon Lee, Dong Won Kim, Jaehyeon Kim +2
eess.AScs.AIcs.CLarXiv:2406.11427v22024
FireRedTTS: A Foundation Text-To-Speech Framework for Industry-Level Generative Speech Applications
Hao-Han Guo, Yao Hu, Kun Liu +6
cs.SDeess.ASarXiv:2409.03283v22024
Qwen-Audio-3.0-Gen-Preview Technical Report
Junyu Dai, Xiaoyue Duan, Xinyue Fan +14
eess.ASarXiv:2607.27011v22026
FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks
Luca Della Libera, Francesco Paissan, Cem Subakan +1
cs.LGcs.AIcs.SDarXiv:2502.04465v22025
Privacy against Real-Time Speech Emotion Detection via Acoustic Adversarial Evasion of Machine Learning
Brian Testa, Yi Xiao, Harshit Sharma +2
cs.LGcs.CRcs.SDarXiv:2211.09273v42022
LLM can Read Spectrogram: Encoder-free Speech-Language Modeling
Ruchao Fan, Yiming Wang, Yuxuan Hu +6
eess.AScs.SDarXiv:2606.10231v42026
Serialized Multi-Layer Multi-Head Attention for Neural Speaker Embedding
Hongning Zhu, Kong Aik Lee, Haizhou Li
cs.SDcs.CLeess.ASarXiv:2107.06493v12021
NAS-VAD: Neural Architecture Search for Voice Activity Detection
Daniel Rho, Jinhyeok Park, Jong Hwan Ko
cs.SDcs.LGeess.ASarXiv:2201.09032v22022
Nonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs
Reo Yoneyama, Ryuichi Yamamoto, Kentaro Tachibana
eess.AScs.LGcs.SDarXiv:2210.15887v22022
AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarios
Yihan Wu, Xu Tan, Bohan Li +5
eess.AScs.AIcs.CLarXiv:2204.00436v12022
A Pyramid Recurrent Network for Predicting Crowdsourced Speech-Quality Ratings of Real-World Signals
Xuan Dong, Donald S. Williamson
eess.AScs.CLcs.LGarXiv:2007.15797v12020
Cold Diffusion for Speech Enhancement
Hao Yen, François G. Germain, Gordon Wichern +1
eess.AScs.SDarXiv:2211.02527v32022
Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understanding
Yingting Li, Ambuj Mehrish, Shuai Zhao +5
cs.CLcs.SDeess.ASarXiv:2303.03267v12023
Improved Language Identification Through Cross-Lingual Self-Supervised Learning
Andros Tjandra, Diptanu Gon Choudhury, Frank Zhang +6
cs.CLcs.SDeess.ASarXiv:2107.04082v42021
PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptions
Reo Shimizu, Ryuichi Yamamoto, Masaya Kawamura +4
eess.AScs.CLcs.LGarXiv:2309.08140v22023
Efficient Streaming LLM for Speech Recognition
Junteng Jia, Gil Keren, Wei Zhou +6
cs.SDcs.AIcs.CLarXiv:2410.03752v12024
SoulX-Podcast: Towards Realistic Long-form Podcasts with Dialectal and Paralinguistic Diversity
Hanke Xie, Haopeng Lin, Wenxiao Cao +17
eess.AScs.SDarXiv:2510.23541v22025
AzeroS: Extending LLM to Speech with Self-Generated Instruction-Free Tuning
Yiwen Shao, Wei Liu, Jiahong Li +4
cs.CLcs.SDeess.ASarXiv:2601.06086v12025
DiarizationLM: Speaker Diarization Post-Processing with Large Language Models
Quan Wang, Yiling Huang, Guanlong Zhao +3
eess.AScs.LGcs.SDarXiv:2401.03506v122024
NONOTO: A Model-agnostic Web Interface for Interactive Music Composition by Inpainting
Théis Bazin, Gaëtan Hadjeres
cs.HCcs.LGcs.SDarXiv:1907.10380v12019
NVMOS: Non-Verbal Vocalization Quality Assessment in Speech
Jialong Mai, Jinxin Ji, Xiaofen Xing +2
cs.SDcs.AIeess.ASarXiv:2606.15888v12026
Explainable Attribute-Based Speaker Verification
Xiaoliang Wu, Chau Luu, Peter Bell +1
cs.SDcs.AIeess.ASarXiv:2405.19796v12024
A Transformer-based Audio Captioning Model with Keyword Estimation
Yuma Koizumi, Ryo Masumura, Kyosuke Nishida +2
eess.AScs.LGcs.SDarXiv:2007.00222v22020
DiffRhythm+: Controllable and Flexible Full-Length Song Generation with Preference Optimization
Huakang Chen, Yuepeng Jiang, Guobin Ma +7
eess.AScs.SDarXiv:2507.12890v22025
Phonemes vs. Projectors: An Investigation of Speech-Language Interfaces for LLM-based ASR
Ziwei Li, Lukuang Dong, Saierdaer Yusuyin +2
eess.ASarXiv:2604.09332v22026
Phoneme-First Prediction for LLM-Based Speech Recognition
Jakob Poncelet, Hugo Van hamme
eess.ASarXiv:2606.10864v12026
In-Context Learning Boosts Speech Recognition via Human-like Adaptation to Speakers and Language Varieties
Nathan Roll, Calbert Graham, Yuka Tatsumi +3
cs.CLcs.SDeess.ASarXiv:2505.14887v12025
Multilingual and Multi-Accent Jailbreaking of Audio LLMs
Jaechul Roh, Virat Shejwalkar, Amir Houmansadr
cs.SDcs.AIcs.CLarXiv:2504.01094v12025
Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models
Atsumoto Ohashi, Neil Zeghidour, Alexandre Défossez +1
cs.CLeess.ASarXiv:2606.11167v22026
BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data
Mateusz Łajszczak, Guillermo Cámbara, Yang Li +16
cs.LGcs.CLeess.ASarXiv:2402.08093v22024
A New Transformer-Based Approach for Audio-Based Kinship Verification and a New Uncontrolled Mandarin Kinship Speech Dataset
Qiyang Sun, Langqing Zhang, Yupei Li +1
cs.SDcs.AIcs.LGarXiv:2609.14145v22026
DiTAR+: Dual Optimization for Robust Autoregressive Diffusion Speech Synthesis
Ziyu Zhang, Tianlun Zuo, Hanzhao Li +2
cs.SDcs.AIeess.ASarXiv:2609.13909v12026
CRAF: Cross-View Residual-Aware Fusion for Deepfake Speech Detection
Minh-Xuan Phan, Khalid Zaman, Candy Olivia Mawalim +1
cs.SDcs.AIeess.ASarXiv:2609.13842v12026
Long-form music generation with latent diffusion
Zach Evans, Julian D. Parker, CJ Carr +3
cs.SDcs.LGeess.ASarXiv:2404.10301v22024
Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioning
Shansong Liu, Atin Sakkeer Hussain, Chenshuo Sun +1
cs.SDcs.AIcs.CLarXiv:2308.11276v12023
AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalization
Yen-Hao Chen, Da-Yi Wu, Tsung-Han Wu +1
eess.AScs.SDarXiv:2011.00316v12020
A Device to Control and Manipulate Occlusion Effects for Own Voice Perception Studies
Rouben Rehman, Simon Kersten, Aron Schliep +1
eess.ASarXiv:2609.12845v12026
Location-based Training with Complementary Folded Linear Orderings for Multichannel Speech Separation
Kaixuan Yang, Stijn Kindt, Nilesh Madhu
eess.ASarXiv:2609.12629v12026
Objective Intelligibility Prediction Using Distance Metrics on Speech Foundation Model Representations
Lyonel Behringer, Andreas Brendel
eess.ASarXiv:2609.13046v12026
Randomized Scores and Diverse Timbres: Augmenting Automatic Music Transcription with Online-Generated Data
Haiwen Xia, Chao Zhang, Qiuqiang Kong
eess.ASarXiv:2610.11197v12026
MIMII DG: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspection for Domain Generalization Task
Kota Dohi, Tomoya Nishida, Harsh Purohit +5
cs.SDcs.AIcs.LGarXiv:2205.13879v22022
Towards AI-Generated Music Plagiarism Detection as a Version Identification Problem
Fotis Koutsikos, Ioannis Prokopiou, Spyridon Kantarelis +6
cs.SDcs.LGeess.ASarXiv:2610.09075v12026
Contextual RNN-T For Open Domain ASR
Mahaveer Jain, Gil Keren, Jay Mahadeokar +3
eess.AScs.CLcs.LGarXiv:2006.03411v22020

Audio and Speech Processing

OpenPA: Forced-alignment-free Multi-granular Pronunciation Assessment with Mixed-transcript Adaptation

Lossless Inference Acceleration for ASR via EAGLE-Style Speculative Decoding

Translatotron 2: High-quality direct speech-to-speech translation with voice preservation

Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module

AURA: Unified Multimodal Framework for Conversational Music Editing

Vector-Quantized Autoregressive Predictive Coding

VoxAnon: A balanced dataset for a fair evaluation of speech anonymization

Does Financial Speech Need Continual Learning Regularization? A Study of EWC and mini-KFAC on Earnings Call ASR

OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation

OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academia

Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis

RDSE: Real-Time Directional Speech Enhancement Framework for Dual-Mic Commodity Smartphones

CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation

How Should We Evaluate Naturalness for Synthetic Speech with a Foreign Accent?

RTMobile: Beyond Real-Time Mobile Acceleration of RNNs for Speech Recognition

Improving Speech-to-Speech Translation Through Unlabeled Text

NELS -- Never-Ending Learner of Sounds

DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors

FireRedTTS: A Foundation Text-To-Speech Framework for Industry-Level Generative Speech Applications

Qwen-Audio-3.0-Gen-Preview Technical Report

FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networks

Privacy against Real-Time Speech Emotion Detection via Acoustic Adversarial Evasion of Machine Learning

LLM can Read Spectrogram: Encoder-free Speech-Language Modeling

Serialized Multi-Layer Multi-Head Attention for Neural Speaker Embedding

NAS-VAD: Neural Architecture Search for Voice Activity Detection

Nonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs

AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarios

A Pyramid Recurrent Network for Predicting Crowdsourced Speech-Quality Ratings of Real-World Signals

Cold Diffusion for Speech Enhancement

Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understanding

Improved Language Identification Through Cross-Lingual Self-Supervised Learning

PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptions

Efficient Streaming LLM for Speech Recognition

SoulX-Podcast: Towards Realistic Long-form Podcasts with Dialectal and Paralinguistic Diversity

AzeroS: Extending LLM to Speech with Self-Generated Instruction-Free Tuning

DiarizationLM: Speaker Diarization Post-Processing with Large Language Models

NONOTO: A Model-agnostic Web Interface for Interactive Music Composition by Inpainting

NVMOS: Non-Verbal Vocalization Quality Assessment in Speech

Explainable Attribute-Based Speaker Verification

A Transformer-based Audio Captioning Model with Keyword Estimation

DiffRhythm+: Controllable and Flexible Full-Length Song Generation with Preference Optimization

Phonemes vs. Projectors: An Investigation of Speech-Language Interfaces for LLM-based ASR

Phoneme-First Prediction for LLM-Based Speech Recognition

In-Context Learning Boosts Speech Recognition via Human-like Adaptation to Speakers and Language Varieties

Multilingual and Multi-Accent Jailbreaking of Audio LLMs

Multi-Faceted Interactivity Alignment in Full-Duplex Speech Models

BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data

A New Transformer-Based Approach for Audio-Based Kinship Verification and a New Uncontrolled Mandarin Kinship Speech Dataset

DiTAR+: Dual Optimization for Robust Autoregressive Diffusion Speech Synthesis

CRAF: Cross-View Residual-Aware Fusion for Deepfake Speech Detection

Long-form music generation with latent diffusion

Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioning

AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalization

A Device to Control and Manipulate Occlusion Effects for Own Voice Perception Studies

Location-based Training with Complementary Folded Linear Orderings for Multichannel Speech Separation

Objective Intelligibility Prediction Using Distance Metrics on Speech Foundation Model Representations

Randomized Scores and Diverse Timbres: Augmenting Automatic Music Transcription with Online-Generated Data

MIMII DG: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspection for Domain Generalization Task

Towards AI-Generated Music Plagiarism Detection as a Version Identification Problem

Contextual RNN-T For Open Domain ASR