Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

121 to 180 of 1,641

RALL-E: Robust Codec Language Modeling with Chain-of-Thought Prompting for Text-to-Speech Synthesis
Detai Xin, Xu Tan, Kai Shen +8
eess.AScs.AIcs.CLarXiv:2404.03204v32024
UtterTune: LoRA-Based Target-Language Pronunciation Edit and Control in Multilingual Text-to-Speech
Shuhei Kato
cs.SDcs.CLeess.ASarXiv:2508.09767v42025
Making Sense of Audio Vibration for Liquid Height Estimation in Robotic Pouring
Hongzhuo Liang, Shuang Li, Xiaojian Ma +4
cs.ROcs.SDeess.ASarXiv:1903.00650v22019
SonoEdit: Null-Space Constrained Knowledge Editing for Pronunciation Correction in LLM-Based TTS
Ayush Pratap Singh, Harshit Singh, Nityanand Mathur +2
cs.SDcs.AIeess.ASarXiv:2601.17086v12026
Attention-Constrained Inference for Robust Decoder-Only Text-to-Speech
Hankun Wang, Chenpeng Du, Yiwei Guo +3
eess.AScs.SDarXiv:2404.19723v22024
Sarashina2.2-TTS: Tackling Kanji Polyphony in Japanese Speech Generation via Data Scaling and Targeted Data Synthesis
Lianbo Liu, Shiao Zhu, Kai Washizaki +10
cs.SDcs.AIcs.CLarXiv:2606.25369v12026
D2: High-Capability Full-Duplex Spoken Language Models from Half-Duplex Backbones
Cheng-Kuang Chang, Alexander H. Liu, Brian Kingsbury +5
cs.CLcs.AIeess.ASarXiv:2610.17124v12026
StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model
Shoutao Guo, Xiang Li, Mengge Liu +2
cs.CLcs.SDeess.ASarXiv:2507.07803v22025
Optimal transport of image sources for interpolation of room impulse responses with moving sources
Jesper Brunnström, Filip Elvander, Isabel Haasler
eess.ASeess.SParXiv:2609.17237v12026
Counting Closures in Spanish Trills: A Multi-Corpus Acoustic Study
Mateo Cámara, Maria F. Alcala-Durand
eess.ASarXiv:2609.17424v12026
Self-Distilled Pronunciation and Accent Control for Neural Text-to-Speech
Shuhei Kato
cs.SDeess.ASarXiv:2609.17234v32026
Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation
Ariel Ephrat, Inbar Mosseri, Oran Lang +5
cs.SDcs.CVeess.ASarXiv:1804.03619v22018
Toward Joint Language Modeling for Speech Units and Text
Ju-Chieh Chou, Chung-Ming Chien, Wei-Ning Hsu +5
cs.CLcs.AIcs.SDarXiv:2310.08715v12023
Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks
Soumi Maiti, Yifan Peng, Shukjae Choi +3
eess.AScs.LGcs.SDarXiv:2309.07937v32023
High-Quality Visually-Guided Sound Separation from Diverse Categories
Chao Huang, Susan Liang, Yapeng Tian +2
cs.CVcs.SDeess.ASarXiv:2308.00122v22023
Speech Driven Video Editing via an Audio-Conditioned Diffusion Model
Dan Bigioi, Shubhajit Basak, Michał Stypułkowski +4
cs.CVcs.LGcs.SDarXiv:2301.04474v32023
Scaling Self-Supervised Speech Models Uncovers Deep Linguistic Relationships: Evidence from the Pacific Cluster
Minu Kim, Hoirin Kim, David R. Mortensen
cs.CLeess.ASarXiv:2603.07238v22026
MSU-Bench: Towards Speaker-Centric Understanding in Conversational Multi-Speaker Scenarios
Zhaokai Sun, Shuai Wang, Zhennan Lin +6
eess.ASarXiv:2606.22868v12026
Stream RAG: Instant and Accurate Spoken Dialogue Systems with Streaming Tool Usage
Siddhant Arora, Haidar Khan, Kai Sun +14
cs.CLcs.SDeess.ASarXiv:2510.02044v12025
SonicSense: Object Perception from In-Hand Acoustic Vibration
Jiaxun Liu, Boyuan Chen
cs.ROcs.MMcs.SDarXiv:2406.17932v22024
GeoMER: A Joint Euclidean-Hyperbolic Geometry Framework for Multimodal Emotion Recognition
Ting Zhu, Huizhi Liang, Boguslaw Obara +2
cs.SDcs.AIcs.LGarXiv:2610.18587v12026
Clotho-AQA: A Crowdsourced Dataset for Audio Question Answering
Samuel Lipping, Parthasaarathy Sudarsanam, Konstantinos Drossos +1
cs.SDcs.LGeess.ASarXiv:2204.09634v22022
AI-Synthesized Voice Detection Using Neural Vocoder Artifacts
Chengzhe Sun, Shan Jia, Shuwei Hou +1
cs.SDcs.MMeess.ASarXiv:2304.13085v22023
Unsupervised Instantaneous Phase and Frequency Tracking by Inverse Voice Synthesis
Chin-Yun Yu, György Fazekas
eess.AScs.SDarXiv:2610.03058v12026
Language Orthogonalization for Zero-Shot Cross-Lingual Audio Deepfake Detection
Minu Kim, Ji Sub Um, Hoirin Kim
eess.AScs.CLarXiv:2609.16458v12026
Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents
Bandhav Veluri, Benjamin N Peloquin, Bokai Yu +2
cs.CLcs.LGcs.SDarXiv:2409.15594v12024
SEAL: Mixture-Closed Additive Reconstruction and Refinement-Aware Expert Routing for Efficient Speech Separation
Shao-Chun Hu, Zi-Xiang Lin, Jeih-Weih Hung +1
eess.AScs.CLcs.SDarXiv:2610.07047v12026
Segmental Posterior Decoding for Audio Moment Retrieval
Seungdeok Choi, Seongmin Choi, Inhan Choi +3
eess.ASarXiv:2609.16495v12026
Logbook: Extremely Long-form Audio Event Understanding
Kwanghee Choi, Suwon Shon, Dmitriy Serdyuk +7
eess.AScs.AIcs.CLarXiv:2610.07338v12026
TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
Liang-Hsuan Tseng, Yi-Chang Chen, Kuan-Yi Lee +2
cs.CLcs.SDeess.ASarXiv:2504.07053v32025
FAM-gate: field-aware metadata attention with prior-constrained vector gating for audio classification
Xiangyu Jing, Hongqing Liu, Liming Shi
eess.AScs.SDarXiv:2610.14673v12026
PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
Cheng-I Jeff Lai, Yang Zhang, Alexander H. Liu +7
cs.CLcs.LGcs.SDarXiv:2106.05933v22021
SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation
Wenyi Yu, Siyin Wang, Xiaoyu Yang +7
eess.AScs.AIcs.CLarXiv:2411.18138v12024
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer
Xiaofei Wang, Manthan Thakker, Zhuo Chen +7
eess.AScs.CLcs.LGarXiv:2308.06873v22023
Open-Qwen-Music: An Auditable Framework for LLM-Based Music Composition and Diffusion Rendering
Yangbin Yu, Mingyu Yang
cs.SDcs.AIeess.ASarXiv:2609.31652v12026
Structural and Representational Agency in LLM-Driven Timbre Generation and Interpolation
Joseph Cameron, Alan Blackwell
cs.SDcs.HCcs.LGarXiv:2610.12532v12026
Accent Analogy Guidance: More Speaker Similarity at Equal Accent in Cross-Lingual Voice Cloning
Yoomee Cho, Jisun Lee
cs.SDcs.CLeess.ASarXiv:2609.29123v12026
MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners
Fang-Duo Tsai, Shih-Lun Wu, Weijaw Lee +4
cs.SDcs.AIeess.ASarXiv:2506.18729v22025
SongCraft: Unified Song Generation and Editing with Reconstructive Learning
Haohe Liu, Varun Nagaraja, Gael Le Lan +5
cs.SDeess.ASeess.SParXiv:2609.16315v12026
Automatic Analysis and Influence of Hierarchical Structure on Melody, Rhythm and Harmony in Popular Music
Shuqi Dai, Huan Zhang, Roger B. Dannenberg
cs.SDeess.ASarXiv:2010.07518v12020
QC-MI-OVSEL: Benchmarking Query-Conditioned Multi-Instance Open-Vocabulary Sound Event Localization
Yongqiang Chen, Peiwei Chang, Yuelan Cheng +2
cs.SDeess.ASarXiv:2610.13198v12026
Contrastive Predictive Coding Based Feature for Automatic Speaker Verification
Cheng-I Lai
cs.CLcs.LGcs.SDarXiv:1904.01575v12019
t-DCF: a Detection Cost Function for the Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification
Tomi Kinnunen, Kong Aik Lee, Hector Delgado +5
eess.AScs.CRcs.SDarXiv:1804.09618v22018
Demucs: Deep Extractor for Music Sources with extra unlabeled data remixed
Alexandre Défossez, Nicolas Usunier, Léon Bottou +1
cs.SDcs.LGeess.ASarXiv:1909.01174v12019
A multi-room reverberant dataset for sound event localization and detection
Sharath Adavanne, Archontis Politis, Tuomas Virtanen
cs.SDeess.ASarXiv:1905.08546v22019
T5Gemma-TTS Technical Report
Chihiro Arata, Kiyoshi Kurihara
eess.ASarXiv:2604.01760v12026
Omni Interaction Agent Technical Report
Orantqing, Shengpeng Ji, Junlong Tong +20
eess.AScs.AIcs.LGarXiv:2609.08977v12026
DEFINE: Exemplar-Guided Accent Control for Zero-Shot TTS
Ambuj Mehrish, Abhinaba Roy, Alex Ivanov +2
cs.SDeess.ASarXiv:2609.32777v22026
A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition
R. Gnana Praveen, Wheidima Carneiro de Melo, Nasib Ullah +8
cs.CVcs.HCcs.SDarXiv:2203.14779v42022
Talk2Agent: Benchmarking Voice Interfaces for Text Agents
Terumi Chiba, Guangzhi Sun, Zheqi Yuan +1
cs.AIeess.ASarXiv:2609.38867v12026
Low-Latency Spell Correction for Japanese Music Search Queries
Anshul Garg, Pavni Tandon, Karan Bhukar +2
cs.SDcs.LGeess.ASarXiv:2609.04262v12026
Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models
Zhifei Xie, Mingbao Lin, Zihang Liu +3
cs.SDcs.AIcs.CLarXiv:2503.02318v22025
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing
Junyi Ao, Rui Wang, Long Zhou +11
eess.AScs.CLcs.LGarXiv:2110.07205v32021
Benchmarking Adult Addressee Classification Across Child- and Adult-Directed Speech Datasets
Sofoklis Kakouros, Daniil Kocharov, Okko Räsänen
eess.ASarXiv:2610.10240v12026
Voice Memory for Agentic Speech Recognition
Chao-Han Huck Yang, Zih-Ching Chen, Piotr Zelasko +3
cs.CLcs.AIcs.SDarXiv:2607.26410v12026
CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
Zhihao Du, Yuxuan Wang, Qian Chen +16
cs.SDcs.AIcs.LGarXiv:2412.10117v32024
Audiovisual joint learning for end-to-end hearing aids
You-Jin Li, Yu Tsao, Borching Su +2
eess.ASarXiv:2610.08579v12026
Realtime-Venus: A full-duplex interaction system with asynchronous delegation
Ant Group
cs.CVeess.ASarXiv:2609.13814v32026
Source-Directed Trajectory Perturbation at First-Order Cost for Domain Generalization in Speech Deepfake Detection
Siqing Qin, Kong Aik Lee, Youzhi Tu
eess.ASarXiv:2610.10094v12026
Articulatory Source-Filter TTS: Physically Grounded Control through Vocal Tract Kinematics
Jesuraj Bandekar, Shinji Watanabe, Prasanta Kumar Ghosh
eess.ASarXiv:2610.00735v12026

Audio and Speech Processing

RALL-E: Robust Codec Language Modeling with Chain-of-Thought Prompting for Text-to-Speech Synthesis

UtterTune: LoRA-Based Target-Language Pronunciation Edit and Control in Multilingual Text-to-Speech

Making Sense of Audio Vibration for Liquid Height Estimation in Robotic Pouring

SonoEdit: Null-Space Constrained Knowledge Editing for Pronunciation Correction in LLM-Based TTS

Attention-Constrained Inference for Robust Decoder-Only Text-to-Speech

Sarashina2.2-TTS: Tackling Kanji Polyphony in Japanese Speech Generation via Data Scaling and Targeted Data Synthesis

D2: High-Capability Full-Duplex Spoken Language Models from Half-Duplex Backbones

StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model

Optimal transport of image sources for interpolation of room impulse responses with moving sources

Counting Closures in Spanish Trills: A Multi-Corpus Acoustic Study

Self-Distilled Pronunciation and Accent Control for Neural Text-to-Speech

Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation

Toward Joint Language Modeling for Speech Units and Text

Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation tasks

High-Quality Visually-Guided Sound Separation from Diverse Categories

Speech Driven Video Editing via an Audio-Conditioned Diffusion Model

Scaling Self-Supervised Speech Models Uncovers Deep Linguistic Relationships: Evidence from the Pacific Cluster

MSU-Bench: Towards Speaker-Centric Understanding in Conversational Multi-Speaker Scenarios

Stream RAG: Instant and Accurate Spoken Dialogue Systems with Streaming Tool Usage

SonicSense: Object Perception from In-Hand Acoustic Vibration

GeoMER: A Joint Euclidean-Hyperbolic Geometry Framework for Multimodal Emotion Recognition

Clotho-AQA: A Crowdsourced Dataset for Audio Question Answering

AI-Synthesized Voice Detection Using Neural Vocoder Artifacts

Unsupervised Instantaneous Phase and Frequency Tracking by Inverse Voice Synthesis

Language Orthogonalization for Zero-Shot Cross-Lingual Audio Deepfake Detection

Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents

SEAL: Mixture-Closed Additive Reconstruction and Refinement-Aware Expert Routing for Efficient Speech Separation

Segmental Posterior Decoding for Audio Moment Retrieval

Logbook: Extremely Long-form Audio Event Understanding

TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling

FAM-gate: field-aware metadata attention with prior-constrained vector gating for audio classification

PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition

SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation

SpeechX: Neural Codec Language Model as a Versatile Speech Transformer

Open-Qwen-Music: An Auditable Framework for LLM-Based Music Composition and Diffusion Rendering

Structural and Representational Agency in LLM-Driven Timbre Generation and Interpolation

Accent Analogy Guidance: More Speaker Similarity at Equal Accent in Cross-Lingual Voice Cloning

MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners

SongCraft: Unified Song Generation and Editing with Reconstructive Learning

Automatic Analysis and Influence of Hierarchical Structure on Melody, Rhythm and Harmony in Popular Music

QC-MI-OVSEL: Benchmarking Query-Conditioned Multi-Instance Open-Vocabulary Sound Event Localization

Contrastive Predictive Coding Based Feature for Automatic Speaker Verification

t-DCF: a Detection Cost Function for the Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification

Demucs: Deep Extractor for Music Sources with extra unlabeled data remixed

A multi-room reverberant dataset for sound event localization and detection

T5Gemma-TTS Technical Report

Omni Interaction Agent Technical Report

DEFINE: Exemplar-Guided Accent Control for Zero-Shot TTS

A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition

Talk2Agent: Benchmarking Voice Interfaces for Text Agents

Low-Latency Spell Correction for Japanese Music Search Queries

Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models

SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing

Benchmarking Adult Addressee Classification Across Child- and Adult-Directed Speech Datasets

Voice Memory for Agentic Speech Recognition

CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models

Audiovisual joint learning for end-to-end hearing aids

Realtime-Venus: A full-duplex interaction system with asynchronous delegation

Source-Directed Trajectory Perturbation at First-Order Cost for Domain Generalization in Speech Deepfake Detection

Articulatory Source-Filter TTS: Physically Grounded Control through Vocal Tract Kinematics