Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

1,201 to 1,260 of 1,641

SottoVoce: An Ultrasound Imaging-Based Silent Speech Interaction Using Deep Neural Networks
Naoki Kimura, Michinari Kono, Jun Rekimoto
cs.HCcs.LGcs.SDarXiv:2303.01758v12023
Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model
Bo Li, Tara N. Sainath, Khe Chai Sim +6
eess.AScs.SDarXiv:1712.01541v12017
Teaching LLMs to Hear Who Spoke What: Metadata-Supervised Pretraining for Encoder-Free Speech-LLMs
Mohan Shi, Ruchao Fan, Sunit Sivasankaran +2
eess.ASarXiv:2610.01695v12026
Emotional End-to-End Neural Speech Synthesizer
Younggun Lee, Azam Rabiee, Soo-Young Lee
cs.SDcs.CLeess.ASarXiv:1711.05447v22017
Measuring Depression Symptom Severity from Spoken Language and 3D Facial Expressions
Albert Haque, Michelle Guo, Adam S Miner +1
cs.CVcs.SDeess.ASarXiv:1811.08592v22018
WavCraft: Audio Editing and Generation with Large Language Models
Jinhua Liang, Huan Zhang, Haohe Liu +7
eess.ASarXiv:2403.09527v32024
Connecting Speech Encoder and Large Language Model for ASR
Wenyi Yu, Changli Tang, Guangzhi Sun +6
eess.AScs.CLcs.SDarXiv:2309.13963v22023
Natural Language Supervision for General-Purpose Audio Representations
Benjamin Elizalde, Soham Deshmukh, Huaming Wang
cs.SDeess.ASarXiv:2309.05767v22023
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks
Suwon Shon, Siddhant Arora, Chyi-Jiunn Lin +7
cs.CLeess.ASarXiv:2212.10525v22022
Voice Conversion With Just Nearest Neighbors
Matthew Baas, Benjamin van Niekerk, Herman Kamper
eess.AScs.CLcs.SDarXiv:2305.18975v12023
Controlling Emotion in Text-to-Speech with Natural Language Prompts
Thomas Bott, Florian Lux, Ngoc Thang Vu
cs.CLcs.SDeess.ASarXiv:2406.06406v22024
PromptStyle: Controllable Style Transfer for Text-to-Speech with Natural Language Descriptions
Guanghou Liu, Yongmao Zhang, Yi Lei +4
cs.SDeess.ASarXiv:2305.19522v22023
EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speech
Daria Diatlova, Vitaly Shutov
eess.AScs.LGcs.SDarXiv:2307.00024v12023
Get Large Language Models Ready to Speak: A Late-fusion Approach for Speech Generation
Maohao Shen, Shun Zhang, Jilong Wu +5
cs.CLcs.AIcs.SDarXiv:2410.20336v12024
EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance
Yiwei Guo, Chenpeng Du, Xie Chen +1
eess.AScs.AIcs.HCarXiv:2211.09496v22022
Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training
Wei-Ning Hsu, Anuroop Sriram, Alexei Baevski +8
cs.SDcs.CLcs.LGarXiv:2104.01027v22021
LakhNES: Improving multi-instrumental music generation with cross-domain pre-training
Chris Donahue, Huanru Henry Mao, Yiting Ethan Li +2
cs.SDcs.LGcs.MMarXiv:1907.04868v12019
Zephyr: An Efficient Audio Denoising System Using Spiking Neural Networks Enabled With A Sparsity-Aware Flexible FPGA PE Array
Cheng-En Chang, Chi-Wei Kao, Chung-Lun Yang +4
cs.AReess.ASarXiv:2609.37711v12026
STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence
Zihan Liu, Zhikang Niu, Qiuyang Xiao +10
cs.SDcs.CLeess.ASarXiv:2510.24693v22025
Text-only adaptation in LLM-based ASR through text denoising
Andrés Carofilis, Sergio Burdisso, Esaú Villatoro-Tello +8
cs.SDcs.CLcs.LGarXiv:2601.20900v32026
WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research
Xinhao Mei, Chutong Meng, Haohe Liu +6
eess.AScs.CLcs.MMarXiv:2303.17395v22023
Tracing and Relearning Detection Evidence in Text-to-Speech Systems
Eunji Shin, Kyudan Jung, Jihwan Kim +2
cs.SDeess.ASarXiv:2609.30983v12026
SpEx+: A Complete Time Domain Speaker Extraction Network
Meng Ge, Chenglin Xu, Longbiao Wang +3
eess.AScs.SDarXiv:2005.04686v22020
RetroThinker: Enabling Retrospective Thinking in Speech LLMs
Yi-Jen Shih, Puyuan Peng, Abdelrahman Mohamed +1
eess.AScs.AIcs.CLarXiv:2609.11864v12026
VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation
Jiarui Hai, Karan Thakkar, Ke Chen +5
eess.AScs.LGarXiv:2608.13613v12026
WHAMR!: Noisy and Reverberant Single-Channel Speech Separation
Matthew Maciejewski, Gordon Wichern, Emmett McQuinn +1
cs.SDeess.ASarXiv:1910.10279v22019
VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech
Yi-Cheng Lin, Yusuke Hirota, Sung-Feng Huang +1
eess.AScs.CLcs.SDarXiv:2604.17248v22026
Louder, Longer, Livelier: Acoustic Shortcuts and Underspecified Rationales in Speech LLM Judges
Mingyue Huo, Shivam Mehta, Bhavin Jawade +2
eess.AScs.SDarXiv:2609.36979v12026
Deep Convolutional Neural Network with Mixup for Environmental Sound Classification
Zhichao Zhang, Shugong Xu, Shan Cao +1
cs.SDeess.ASarXiv:1808.08405v12018
LLM-ForcedAligner: A Non-Autoregressive and Accurate LLM-Based Forced Aligner for Multilingual and Long-Form Speech
Bingshen Mu, Xian Shi, Xiong Wang +3
cs.SDeess.ASarXiv:2601.18220v22026
Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy
Abhinav Sharma, Sai Karthik Navuluru, Wang Wei +21
cs.CVcs.MMcs.SDarXiv:2609.34381v12026
Acoustic-to-Text KV Compression for Full-Duplex Speech Models
Yejin Lee, Seungbeom Kim, Yongha Lee +1
cs.SDcs.AIeess.ASarXiv:2609.31224v12026
Beyond Textual Chain-of-Thought: JEPA-Conditioned Latent Reasoning for Large Audio Language Models
Donghang Wu, Haoyang Zhang, Yizhou Peng +6
eess.ASarXiv:2609.34407v12026
Training-Free Contextual ASR via SpeechLLM-Based Error-Aware Selective Retrieval
Natsuo Yamashita, Ai Nemoto, Ryosuke Koichi +1
cs.SDeess.ASarXiv:2609.30694v12026
GenSE: Generative Speech Enhancement via Language Models using Hierarchical Modeling
Jixun Yao, Hexin Liu, Chen Chen +3
eess.AScs.SDarXiv:2502.02942v12025
High-quality nonparallel voice conversion based on cycle-consistent adversarial network
Fuming Fang, Junichi Yamagishi, Isao Echizen +1
eess.AScs.CLcs.SDarXiv:1804.00425v12018
Subject-Invariant Cross-Modal Decoding of Perceived Speech from Brain Recordings
Aoke Zhang, Jing Chen
cs.SDcs.AIeess.ASarXiv:2609.30832v12026
CLEAR: Online Speech Content Leakage Estimation through Cross-ASR Disagreement
Bhawana Chhaglani, Tanvi Kandepuneni, Jeremy Gummeson +1
cs.SDeess.ASarXiv:2609.30415v12026
Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?
Andrew Rouditchenko, Saurabhchand Bhati, Edson Araujo +4
eess.AScs.SDarXiv:2505.09439v32025
SongEval: A Benchmark Dataset for Song Aesthetics Evaluation
Jixun Yao, Guobin Ma, Huixin Xue +9
eess.ASarXiv:2505.10793v12025
RePlay: Retrieval-Based Voice Playback for Multi-Turn spoken dialogue
Sathvik Udupa, Naveen Kumar, Ryan Folmsbee
eess.AScs.SDarXiv:2609.31588v12026
Natural language guidance of high-fidelity text-to-speech with synthetic annotations
Dan Lyth, Simon King
cs.SDcs.CLeess.ASarXiv:2402.01912v12024
Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy
Yin Cao, Qiuqiang Kong, Turab Iqbal +3
cs.SDeess.ASarXiv:1905.00268v42019
MiDashengLM: Efficient Audio Understanding with General Audio Captions
Heinrich Dinkel, Gang Li, Jizhong Liu +7
cs.SDeess.ASarXiv:2508.03983v32025
Geometry-Aware Optimization for Respiratory Sound Classification: Enhancing Sensitivity with SAM-Optimized Audio Spectrogram Transformers
Atakan Işık, Selin Vulga Işık, Ahmet Feridun Işık +1
eess.AScs.AIcs.LGarXiv:2512.22564v12025
Synth-JEPA: Joint Embedding Prediction for Renderer-Free Synthesizer Parameter Search
Ben Hayes, Haokun Tian, Stefan Lattner
cs.SDcs.LGeess.ASarXiv:2609.31024v12026
BAT-CLIP: Trimodal Alignment of Brain, Audio and Text
Suhyun Kim, Jinmo Han, Danny Dongyeop Han +9
cs.SDcs.AIcs.LGarXiv:2609.31180v12026
Assessing a Mathematical Model of Syllable Production via CTW Alignment with EMA Data
Frédéric Berthommier
eess.ASarXiv:2609.31508v12026
BreathGRU: A Novel Semi-Supervised Bidirectional Gated Recurrent Unit Framework for Speech and Breath Segmentation for Respiratory Audio
Sania Fatima Sayed, John W. Holloway, Reyer Zwiggelaar +1
cs.SDcs.LGeess.ASarXiv:2609.31165v12026
Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder
Kei Akuzawa, Yusuke Iwasawa, Yutaka Matsuo
cs.CLcs.SDeess.ASarXiv:1804.02135v32018
Unsupervised pretraining transfers well across languages
Morgane Rivière, Armand Joulin, Pierre-Emmanuel Mazaré +1
eess.AScs.CLcs.LGarXiv:2002.02848v12020
Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks
Sharath Adavanne, Archontis Politis, Joonas Nikunen +1
cs.SDeess.ASarXiv:1807.00129v32018
Does per-frame early exit pay? A compute-matched study of dynamic depth for on-device speech enhancement
Clément Laroche, Riccardo Miccini
eess.AScs.LGcs.SDarXiv:2609.29867v12026
Beyond Model Size: Redesigning LiSenNet for embedded speech enhancement
Clément Laroche, Rasmus Kongsgaard Olsson
eess.AScs.LGcs.SDarXiv:2609.29866v12026
Candor-LR: A Dyadic Conversational Dataset for Audio-Visual Speech Recognition
Rishabh Jain, Aristeidis Papadopoulos, Zhaofeng Lin +1
eess.AScs.CVcs.MMarXiv:2609.10394v12026
Beyond DER: Speaker Counting in Crowded End-to-End Diarization
Lahiru Samarakoon, Hongxu Zhu
eess.ASarXiv:2609.27315v12026
ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection
Taewoo Kim, Young Han Lee, Nam In Park +1
eess.AScs.AIcs.SDarXiv:2609.03620v12026
Exploring a Single Autoregressive LLM for Unified Target Speech Extraction across Synchronous and Asynchronous Cues
Wenxuan Wu, Shuhan Zhang, Shuai Wang +1
cs.SDcs.MMeess.ASarXiv:2609.29238v12026
Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures
Se Un Park, Yutae Kim, Junyoung Park
eess.AScs.LGcs.SDarXiv:2609.28974v12026
Is Prosody Lost in Translation? Fine-Grained Cross-Lingual Prosody Similarity Across Languages
Haopeng Xie, Ismail Rasim Ulgen, Sofia Son +2
cs.SDcs.CLeess.ASarXiv:2608.27848v12026

Audio and Speech Processing

SottoVoce: An Ultrasound Imaging-Based Silent Speech Interaction Using Deep Neural Networks

Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model

Teaching LLMs to Hear Who Spoke What: Metadata-Supervised Pretraining for Encoder-Free Speech-LLMs

Emotional End-to-End Neural Speech Synthesizer

Measuring Depression Symptom Severity from Spoken Language and 3D Facial Expressions

WavCraft: Audio Editing and Generation with Large Language Models

Connecting Speech Encoder and Large Language Model for ASR

Natural Language Supervision for General-Purpose Audio Representations

SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks

Voice Conversion With Just Nearest Neighbors

Controlling Emotion in Text-to-Speech with Natural Language Prompts

PromptStyle: Controllable Style Transfer for Text-to-Speech with Natural Language Descriptions

EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speech

Get Large Language Models Ready to Speak: A Late-fusion Approach for Speech Generation

EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance

Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training

LakhNES: Improving multi-instrumental music generation with cross-domain pre-training

Zephyr: An Efficient Audio Denoising System Using Spiking Neural Networks Enabled With A Sparsity-Aware Flexible FPGA PE Array

STAR-Bench: Probing Deep Spatio-Temporal Reasoning as Audio 4D Intelligence

Text-only adaptation in LLM-based ASR through text denoising

WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research

Tracing and Relearning Detection Evidence in Text-to-Speech Systems

SpEx+: A Complete Time Domain Speaker Extraction Network

RetroThinker: Enabling Retrospective Thinking in Speech LLMs

VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation

WHAMR!: Noisy and Reverberant Single-Channel Speech Separation

VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech

Louder, Longer, Livelier: Acoustic Shortcuts and Underspecified Rationales in Speech LLM Judges

Deep Convolutional Neural Network with Mixup for Environmental Sound Classification

LLM-ForcedAligner: A Non-Autoregressive and Accurate LLM-Based Forced Aligner for Multilingual and Long-Form Speech

Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy

Acoustic-to-Text KV Compression for Full-Duplex Speech Models

Beyond Textual Chain-of-Thought: JEPA-Conditioned Latent Reasoning for Large Audio Language Models

Training-Free Contextual ASR via SpeechLLM-Based Error-Aware Selective Retrieval

GenSE: Generative Speech Enhancement via Language Models using Hierarchical Modeling

High-quality nonparallel voice conversion based on cycle-consistent adversarial network

Subject-Invariant Cross-Modal Decoding of Perceived Speech from Brain Recordings

CLEAR: Online Speech Content Leakage Estimation through Cross-ASR Disagreement

Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?

SongEval: A Benchmark Dataset for Song Aesthetics Evaluation

RePlay: Retrieval-Based Voice Playback for Multi-Turn spoken dialogue

Natural language guidance of high-fidelity text-to-speech with synthetic annotations

Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy

MiDashengLM: Efficient Audio Understanding with General Audio Captions

Geometry-Aware Optimization for Respiratory Sound Classification: Enhancing Sensitivity with SAM-Optimized Audio Spectrogram Transformers

Synth-JEPA: Joint Embedding Prediction for Renderer-Free Synthesizer Parameter Search

BAT-CLIP: Trimodal Alignment of Brain, Audio and Text

Assessing a Mathematical Model of Syllable Production via CTW Alignment with EMA Data

BreathGRU: A Novel Semi-Supervised Bidirectional Gated Recurrent Unit Framework for Speech and Breath Segmentation for Respiratory Audio

Expressive Speech Synthesis via Modeling Expressions with Variational Autoencoder

Unsupervised pretraining transfers well across languages

Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks

Does per-frame early exit pay? A compute-matched study of dynamic depth for on-device speech enhancement

Beyond Model Size: Redesigning LiSenNet for embedded speech enhancement

Candor-LR: A Dyadic Conversational Dataset for Audio-Visual Speech Recognition

Beyond DER: Speaker Counting in Crowded End-to-End Diarization

ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection

Exploring a Single Autoregressive LLM for Unified Target Speech Extraction across Synchronous and Asynchronous Cues

Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures

Is Prosody Lost in Translation? Fine-Grained Cross-Lingual Prosody Similarity Across Languages