Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

481 to 540 of 1,641

Self-Supervised Speech Representation Learning: A Review
Abdelrahman Mohamed, Hung-yi Lee, Lasse Borgholt +9
cs.CLcs.SDeess.ASarXiv:2205.10643v32022
Diffusion Buffer for Online Generative Speech Enhancement
Bunlong Lay, Rostislav Makarov, Simon Welker +2
eess.AScs.LGcs.SDarXiv:2510.18744v12025
Sensing Bone-Conducted Speech with Earbuds
Christoph Weyer, Peter Jax
eess.ASarXiv:2609.02165v12026
Improved Regularization Techniques for End-to-End Speech Recognition
Yingbo Zhou, Caiming Xiong, Richard Socher
cs.CLcs.SDeess.ASarXiv:1712.07108v12017
MoME: Mixture of Matryoshka Experts for Audio-Visual Speech Recognition
Umberto Cappellazzo, Minsu Kim, Pingchuan Ma +4
eess.AScs.CVcs.SDarXiv:2510.04136v12025
Rethinking MUSHRA: Addressing Modern Challenges in Text-to-Speech Evaluation
Praveen Srinivasa Varadhan, Amogh Gulati, Ashwin Sankar +8
cs.CLcs.LGcs.SDarXiv:2411.12719v32024
EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis
Tianchi Liu, Zeyang Song, Tianrui Wang +3
eess.AScs.AIarXiv:2608.23791v12026
SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models
Chih-Kai Yang, Yen-Ting Piao, Tzu-Wen Hsu +8
cs.SDcs.AIcs.CLarXiv:2510.16917v22025
SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
Xin Zhang, Dong Zhang, Shimin Li +2
cs.CLcs.SDeess.ASarXiv:2308.16692v22023
Positive Sample Propagation along the Audio-Visual Event Line
Jinxing Zhou, Liang Zheng, Yiran Zhong +2
cs.CVcs.MMcs.SDarXiv:2104.00239v32021
Time Domain Audio Visual Speech Separation
Jian Wu, Yong Xu, Shi-Xiong Zhang +4
eess.AScs.SDarXiv:1904.03760v22019
CMGAN: Conformer-based Metric GAN for Speech Enhancement
Ruizhe Cao, Sherif Abdulatif, Bin Yang
cs.SDcs.AIcs.LGarXiv:2203.15149v42022
Cooperative Multi-Agent Reinforcement Learning for Adaptive Aggregation in Semi-Supervised Federated Learning with non-IID Data
Rene Glitza, Luca Becker, Rainer Martin
cs.LGcs.DCcs.SDarXiv:2608.25794v12026
Text-Free Prosody-Aware Generative Spoken Language Modeling
Eugene Kharitonov, Ann Lee, Adam Polyak +8
cs.CLcs.LGcs.SDarXiv:2109.03264v22021
Deep Learning Enabled Semantic Communications with Speech Recognition and Synthesis
Zhenzi Weng, Zhijin Qin, Xiaoming Tao +3
eess.AScs.SDarXiv:2205.04603v22022
UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data
Chengyi Wang, Yu Wu, Yao Qian +5
cs.CLcs.LGcs.SDarXiv:2101.07597v22021
Joint Robust Voicing Detection and Pitch Estimation Based on Residual Harmonics
Thomas Drugman, Abeer Alwan
cs.SDcs.CLeess.ASarXiv:2001.00459v12019
TernaryBERT: Distillation-aware Ultra-low Bit BERT
Wei Zhang, Lu Hou, Yichun Yin +4
cs.CLcs.LGcs.SDarXiv:2009.12812v32020
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation
Jingwen Hu, Yuchen Liu, Jinming Zhao +1
cs.CLcs.SDeess.ASarXiv:2107.06779v12021
Broadband Acoustic Intensity Direction Estimation with Tight-Frame Cardioid Arrays
Akira Omoto
eess.ASarXiv:2609.03490v12026
CoJEPA: Combining Contrastive Learning and JEPA for Global-Local Music Representations
Gabriel Meseguer-Brocal, Yuexuan Kong, Romain Hennequin
cs.SDcs.AIcs.LGarXiv:2608.30974v12026
MoshiRAG: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Models
Chung-Ming Chien, Manu Orsini, Eugene Kharitonov +3
cs.CLeess.ASarXiv:2604.12928v32026
BUT System Description to VoxCeleb Speaker Recognition Challenge 2019
Hossein Zeinali, Shuai Wang, Anna Silnova +2
eess.AScs.CLcs.SDarXiv:1910.12592v12019
MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer
Yuancheng Wang, Haoyue Zhan, Liwei Liu +7
cs.SDcs.AIcs.LGarXiv:2409.00750v32024
AdaSpeech: Adaptive Text to Speech for Custom Voice
Mingjian Chen, Xu Tan, Bohan Li +4
eess.AScs.AIcs.CLarXiv:2103.00993v12021
UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation
Huaishao Luo, Lei Ji, Botian Shi +6
cs.CVcs.CLcs.LGarXiv:2002.06353v32020
Spoken DialogSum: An Emotion-Rich Conversational Dataset for Spoken Dialogue Summarization
Yen-Ju Lu, Kunxiao Gao, Mingrui Liang +5
cs.CLcs.AIcs.LGarXiv:2512.14687v22025
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining
Haohe Liu, Yi Yuan, Xubo Liu +7
cs.SDcs.AIcs.MMarXiv:2308.05734v32023
Distilling Conversations: Abstract Compression of Conversational Audio Context for LLM-based ASR
Shashi Kumar, Esaú Villatoro-Tello, Sergio Burdisso +7
cs.CLcs.AIcs.LGarXiv:2603.26246v12026
Auto-Tuning Spectral Clustering for Speaker Diarization Using Normalized Maximum Eigengap
Tae Jin Park, Kyu J. Han, Manoj Kumar +1
eess.AScs.LGarXiv:2003.02405v12020
Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models
Kyudan Jung, Jihwan Kim, Soyoon Kim +3
cs.SDcs.AIeess.ASarXiv:2603.25750v22026
A Framework for the Robust Evaluation of Sound Event Detection
Cagdas Bilen, Giacomo Ferroni, Francesco Tuveri +2
eess.AScs.SDarXiv:1910.08440v22019
Statistical Parametric Speech Synthesis Incorporating Generative Adversarial Networks
Yuki Saito, Shinnosuke Takamichi, Hiroshi Saruwatari
cs.SDcs.LGeess.ASarXiv:1709.08041v12017
Deep Spoken Keyword Spotting: An Overview
Iván López-Espejo, Zheng-Hua Tan, John Hansen +1
cs.SDcs.HCcs.LGarXiv:2111.10592v12021
SE-MSB: End-to-End Unpaired Speech Enhancement using Mamba Schrödinger Bridges
Andreas Bagge, Andreas Nymand, Michael Riis Andersen +1
eess.AScs.AIcs.SDarXiv:2609.26000v12026
LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generation
Zhen Zeng, Jianzong Wang, Ning Cheng +1
eess.ASarXiv:2102.10815v12021
DeviceTTS: A Small-Footprint, Fast, Stable Network for On-Device Text-to-Speech
Zhiying Huang, Hao Li, Ming Lei
eess.AScs.SDarXiv:2010.15311v22020
Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilities
Zhifei Xie, Changqiao Wu
eess.AScs.AIcs.CVarXiv:2410.11190v32024
T-GSA: Transformer with Gaussian-weighted self-attention for speech enhancement
Jaeyoung Kim, Mostafa El-Khamy, Jungwon Lee
eess.AScs.SDarXiv:1910.06762v32019
Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model
Zhen Ye, Peiwen Sun, Jiahe Lei +9
eess.AScs.AIcs.CLarXiv:2408.17175v32024
Contrastive Learning of Musical Representations
Janne Spijkervet, John Ashley Burgoyne
cs.SDcs.LGeess.ASarXiv:2103.09410v22021
What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models
Angelos-Nikolaos Kanatas, Yuexuan Kong, Pablo Alonso-Jiménez +2
cs.SDcs.LGeess.ASarXiv:2608.14819v12026
VibeVoice-ASR-Streaming Technical Report
Yujie Tu, Zhiliang Peng, Jianwei Yu +10
eess.ASarXiv:2609.02812v12026
Efficient Sequence Transduction by Jointly Predicting Tokens and Durations
Hainan Xu, Fei Jia, Somshubra Majumdar +3
eess.AScs.CLcs.LGarXiv:2304.06795v22023
F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching
Yushen Chen, Zhikang Niu, Ziyang Ma +5
eess.AScs.SDarXiv:2410.06885v32024
Deep Learning for Audio Signal Processing
Hendrik Purwins, Bo Li, Tuomas Virtanen +3
cs.SDeess.ASstat.MLarXiv:1905.00078v22019
Gesticulator: A framework for semantically-aware speech-driven gesture generation
Taras Kucherenko, Patrik Jonell, Sanne van Waveren +4
cs.HCcs.LGeess.ASarXiv:2001.09326v52020
Localizing Visual Sounds the Hard Way
Honglie Chen, Weidi Xie, Triantafyllos Afouras +3
cs.CVeess.ASeess.IVarXiv:2104.02691v12021
Learning Neural Acoustic Fields
Andrew Luo, Yilun Du, Michael J. Tarr +3
cs.SDcs.CVcs.LGarXiv:2204.00628v22022
Neural source-filter waveform models for statistical parametric speech synthesis
Xin Wang, Shinji Takaki, Junichi Yamagishi
eess.AScs.SDstat.MLarXiv:1904.12088v22019
Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs
Jihoo Jung, Youngjoon Jang, Joon Son Chung
cs.CVcs.SDeess.ASarXiv:2609.31193v12026
Generative Spoken Dialogue Language Modeling
Tu Anh Nguyen, Eugene Kharitonov, Jade Copet +8
cs.CLcs.LGcs.SDarXiv:2203.16502v22022
Music Source Separation via Stem Discovery
V. Valtteri Kallinen, Eloi Moliner, Lauri Juvela +1
eess.AScs.SDarXiv:2609.30912v12026
Moshi: a speech-text foundation model for real-time dialogue
Alexandre Défossez, Laurent Mazaré, Manu Orsini +5
eess.AScs.AIcs.CLarXiv:2410.00037v22024
MuseTimbre: Zero-Shot Timbre Transfer by Controlling a Frozen Music Generator
Yuan-Chiao Cheng, Zhiyao Duan
cs.SDeess.ASarXiv:2609.30548v12026
Qwen2-Audio Technical Report
Yunfei Chu, Jin Xu, Qian Yang +9
eess.AScs.CLcs.LGarXiv:2407.10759v12024
Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information
Heqing Zou, Yuke Si, Chen Chen +2
cs.SDcs.AIeess.ASarXiv:2203.15326v12022
Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations
Hyeong-Seok Choi, Juheon Lee, Wansoo Kim +3
cs.SDcs.AIeess.ASarXiv:2110.14513v22021
The challenge of realistic music generation: modelling raw audio at scale
Sander Dieleman, Aäron van den Oord, Karen Simonyan
cs.SDcs.LGeess.ASarXiv:1806.10474v12018
Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech
Jaehyeon Kim, Jungil Kong, Juhee Son
cs.SDeess.ASarXiv:2106.06103v12021

Audio and Speech Processing

Self-Supervised Speech Representation Learning: A Review

Diffusion Buffer for Online Generative Speech Enhancement

Sensing Bone-Conducted Speech with Earbuds

Improved Regularization Techniques for End-to-End Speech Recognition

MoME: Mixture of Matryoshka Experts for Audio-Visual Speech Recognition

Rethinking MUSHRA: Addressing Modern Challenges in Text-to-Speech Evaluation

EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis

SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models

Positive Sample Propagation along the Audio-Visual Event Line

Time Domain Audio Visual Speech Separation

CMGAN: Conformer-based Metric GAN for Speech Enhancement

Cooperative Multi-Agent Reinforcement Learning for Adaptive Aggregation in Semi-Supervised Federated Learning with non-IID Data

Text-Free Prosody-Aware Generative Spoken Language Modeling

Deep Learning Enabled Semantic Communications with Speech Recognition and Synthesis

UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data

Joint Robust Voicing Detection and Pitch Estimation Based on Residual Harmonics

TernaryBERT: Distillation-aware Ultra-low Bit BERT

MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation

Broadband Acoustic Intensity Direction Estimation with Tight-Frame Cardioid Arrays

CoJEPA: Combining Contrastive Learning and JEPA for Global-Local Music Representations

MoshiRAG: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Models

BUT System Description to VoxCeleb Speaker Recognition Challenge 2019

MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer

AdaSpeech: Adaptive Text to Speech for Custom Voice

UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation

Spoken DialogSum: An Emotion-Rich Conversational Dataset for Spoken Dialogue Summarization

AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining

Distilling Conversations: Abstract Compression of Conversational Audio Context for LLM-based ASR

Auto-Tuning Spectral Clustering for Speaker Diarization Using Normalized Maximum Eigengap

Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models

A Framework for the Robust Evaluation of Sound Event Detection

Statistical Parametric Speech Synthesis Incorporating Generative Adversarial Networks

Deep Spoken Keyword Spotting: An Overview

SE-MSB: End-to-End Unpaired Speech Enhancement using Mamba Schrödinger Bridges

LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generation

DeviceTTS: A Small-Footprint, Fast, Stable Network for On-Device Text-to-Speech

Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex Capabilities

T-GSA: Transformer with Gaussian-weighted self-attention for speech enhancement

Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model

Contrastive Learning of Musical Representations

What Makes a Good Layer? Assessing the Layer-Wise Intrinsic Properties of Music Foundation Models

VibeVoice-ASR-Streaming Technical Report

Efficient Sequence Transduction by Jointly Predicting Tokens and Durations

F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching

Deep Learning for Audio Signal Processing

Gesticulator: A framework for semantically-aware speech-driven gesture generation

Localizing Visual Sounds the Hard Way

Learning Neural Acoustic Fields

Neural source-filter waveform models for statistical parametric speech synthesis

Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs

Generative Spoken Dialogue Language Modeling

Music Source Separation via Stem Discovery

Moshi: a speech-text foundation model for real-time dialogue

MuseTimbre: Zero-Shot Timbre Transfer by Controlling a Frozen Music Generator

Qwen2-Audio Technical Report

Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information

Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations

The challenge of realistic music generation: modelling raw audio at scale

Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech