Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

301 to 360 of 1,641

From Scores to Evidence: Auditable Decisions Can Improve Speech Deepfake Detection
Mengzhe Geng, Yujia Lu, Patrick Littell +2
cs.SDcs.CLeess.ASarXiv:2609.08899v22026
Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction
Ailin Huang, Boyong Wu, Bruce Wang +142
cs.CLcs.AIcs.HCarXiv:2502.11946v22025
Step-Audio-R1.5 Technical Report
Yuxin Zhang, Xiangyu Tony Zhang, Daijiao Liu +16
eess.ASarXiv:2604.25719v22026
Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation
Chetwin Low, Weimin Wang, Calder Katyal
cs.MMcs.CVcs.SDarXiv:2510.01284v12025
Imputer: Sequence Modelling via Imputation and Dynamic Programming
William Chan, Chitwan Saharia, Geoffrey Hinton +2
eess.AScs.CLcs.LGarXiv:2002.08926v22020
JASPER: Joint Audio and Speech Pre-trained Encoder Representations
Geeth George, Ameenudeen P E, Hrishikesh H Pillai +1
eess.ASarXiv:2609.27260v12026
UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training
Sanyuan Chen, Yu Wu, Chengyi Wang +8
cs.CLcs.SDeess.ASarXiv:2110.05752v12021
Interpreting hierarchical organisation of speaker embeddings
Yanze Xu, Wenwu Wang, Mark D. Plumbley
eess.AScs.AIarXiv:2609.15203v22026
Listen As You Wish: Audio based Event Detection via Text-to-Audio Grounding in Smart Cities
Haoyu Tang, Yunxiao Wang, Jihua Zhu +4
cs.SDcs.MMeess.ASarXiv:2106.14136v32021
HiKE: Hierarchical Evaluation Framework for Korean-English Code-Switching Speech Recognition
Gio Paik, Yongbeom Kim, Soungmin Lee +2
cs.CLcs.SDeess.ASarXiv:2509.24613v42025
Pisets: A Robust Speech Recognition System for Lectures and Interviews
Ivan Bondarenko, Daniil Grebenkin, Oleg Sedukhin +3
cs.CLcs.SDeess.ASarXiv:2601.18415v12026
End-to-end Listen, Look, Speak and Act
Siyin Wang, Wenyi Yu, Xianzhao Chen +4
cs.AIcs.CLcs.CVarXiv:2510.16756v22025
Symbiotic Architecture for Post-Hoc Audio Extension of Frozen Language Models
Yotaro Kubo, Qi Sun, Yujin Tang
cs.SDcs.CLeess.ASarXiv:2609.30784v12026
DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation
Dongya Jia, Zhuo Chen, Jiawei Chen +8
eess.AScs.AIcs.CLarXiv:2502.03930v42025
Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Models
Kunlin Cai, Kaiyuan Zhang, Zihang Xiang +4
cs.CRcs.LGcs.SDarXiv:2609.01723v12026
Domain-Incremental Learning for Generative Speech Enhancement
Manjunath Mulimani, Annamaria Mesaros, Minje Kim +1
eess.AScs.SDarXiv:2609.34901v12026
MMDisCo: Multi-Modal Discriminator-Guided Cooperative Diffusion for Joint Audio and Video Generation
Akio Hayakawa, Masato Ishii, Takashi Shibuya +1
cs.CVcs.LGcs.MMarXiv:2405.17842v22024
Does Your Voice Assistant Remember? Analyzing Conversational Context Recall and Utilization in Voice Interaction Models
Heeseung Kim, Che Hyun Lee, Sangkwon Park +4
cs.SDeess.ASarXiv:2502.19759v22025
Unifying Score and Performance for Fine-Grained Music Understanding in Audio-Language Models
Milan Liessens Dujardin, Song-Ze Yu, Kevin Miao
cs.SDeess.ASarXiv:2609.10351v12026
Glance and Gaze: A Collaborative Learning Framework for Single-channel Speech Enhancement
Andong Li, Chengshi Zheng, Lu Zhang +1
cs.SDeess.ASarXiv:2106.11789v12021
StreamHear: Domain-Adapted Pseudo-Labeling for Semi-Supervised Streaming Speech Recognition
Zefang Liu, Chenyang Zhu, Sangwoo Cho +3
cs.CLeess.ASarXiv:2608.13717v12026
Review of end-to-end speech synthesis technology based on deep learning
Zhaoxi Mu, Xinyu Yang, Yizhuo Dong
cs.SDcs.CLeess.ASarXiv:2104.09995v12021
MusGU+: Toward a Musician-Centered Evaluation Framework and Discovery Tool for Generative Music AI
Laura Ibáñez-Martínez, Roser Batlle-Roca, Xavier Serra +1
cs.SDcs.AIcs.CYarXiv:2608.30940v12026
Jukebox: A Generative Model for Music
Prafulla Dhariwal, Heewoo Jun, Christine Payne +3
eess.AScs.LGcs.SDarXiv:2005.00341v12020
Helping Music Co-Creation Agents 'Listen' Well: Hierarchical Self-Supervised World Models for Understanding and Generation
Scott H. Hawley
cs.SDcs.LGeess.ASarXiv:2608.04378v12026
Baichuan-Audio: A Unified Framework for End-to-End Speech Interaction
Tianpeng Li, Jun Liu, Tao Zhang +11
cs.CLcs.SDeess.ASarXiv:2502.17239v12025
StepAudio 3 Gen Technical Report
Bin Lin, Bo Zhao, Boyang Wang +68
cs.SDeess.ASarXiv:2609.12945v12026
Frontend-Assisted Large Audio-Language Models for SLT2026 SmartGlasses Challenge
Yushi Wang, Zeyan Song, Xiaobin Rong +3
eess.AScs.SDarXiv:2610.13282v12026
Controlled Acquisition and Abstention in Three-Channel Score Conflicts
Mengzhe Geng
cs.LGcs.MMeess.ASarXiv:2610.10808v12026
CAL-MOS: Bridging Layers with Adapters for Robust MOS Prediction Across Speech Foundation Models
Alef Iury Siqueira Ferreira, Pedro Lustosa Rege Botelho, Fernanda Silva +5
cs.SDcs.AIeess.ASarXiv:2609.14956v12026
NVAlign: Direct-Gradient Optimization for Non-Verbal Control in Continuous Autoregressive Flow Matching Text-to-Speech
Qiaolin Wang, Pedro Sandoval-Segura, Anunaya Joshi +2
cs.SDcs.AIcs.CLarXiv:2609.31892v12026
SSAST: Self-Supervised Audio Spectrogram Transformer
Yuan Gong, Cheng-I Jeff Lai, Yu-An Chung +1
cs.SDcs.AIeess.ASarXiv:2110.09784v22021
rVAD: An Unsupervised Segment-Based Robust Voice Activity Detection Method
Zheng-Hua Tan, Achintya kr. Sarkar, Najim Dehak
cs.SDcs.CLcs.LGarXiv:1906.03588v22019
CIViC: Evidence Gates for Audio-Conditioned Visual Prediction on Proxy Tasks
Mengzhe Geng, Max W. Y. Lam, Yuewen Cao +4
cs.LGcs.CVcs.SDarXiv:2610.15327v12026
Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
Wei Ping, Kainan Peng, Andrew Gibiansky +5
cs.SDcs.AIcs.CLarXiv:1710.07654v32017
Language Model Can Listen While Speaking
Ziyang Ma, Yakun Song, Chenpeng Du +5
cs.CLcs.AIcs.HCarXiv:2408.02622v12024
MOV-AAD: A Large-Scale Multimodal Dataset for Auditory Attention Decoding During Moving Conversations
Xiaomin He, Vishal Choudhari, Tristan J. Spratt +3
eess.ASarXiv:2610.04180v12026
Fun-ASR Technical Report
Keyu An, Yanni Chen, Zhigao Chen +35
cs.CLcs.AIcs.SDarXiv:2509.12508v42025
Perception-Inspired Bayesian Causal Fusion for Audiovisual Source Localization
Kyung Yun Lee, Sungnyun Kim, Sebastian J. Schlecht +2
eess.ASarXiv:2609.36441v12026
GAN-based Joint Dereverberation and Directional Filtering
Weilong Huang, Shrishti Saha Shetu, Emanuël A. P. Habets
eess.ASarXiv:2608.26403v12026
SpeechBrain: A General-Purpose Speech Toolkit
Mirco Ravanelli, Titouan Parcollet, Peter Plantinga +18
eess.AScs.AIcs.LGarXiv:2106.04624v12021
FastPitch: Parallel Text-to-speech with Pitch Prediction
Adrian Łańcucki
eess.AScs.CLcs.LGarXiv:2006.06873v22020
TasNet: time-domain audio separation network for real-time, single-channel speech separation
Yi Luo, Nima Mesgarani
cs.SDcs.LGcs.MMarXiv:1711.00541v22017
DuplexSLA: A Full-Duplex Spoken Language Model with Synchronized Speech, Language, and Action
Haoyang Zhang, Jun Chen, Donghang Wu +13
eess.ASarXiv:2605.20755v22026
AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark
Sören Becker, Johanna Vielhaben, Marcel Ackermann +3
cs.SDcs.AIcs.LGarXiv:1807.03418v32018
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations
Adam Polyak, Yossi Adi, Jade Copet +5
cs.SDcs.LGeess.ASarXiv:2104.00355v32021
EmoRES-TTS: Residual-Enhanced Vector Steering for Emotional Speech Generation
Kuan-Po Huang, Haohe Liu, Puyuan Peng +5
cs.SDcs.CLeess.ASarXiv:2609.38157v12026
When Vocal Tone and Literal Meaning Diverge: An Acoustic-Semantic Incongruity Study for Large Audio-Language Models
Yu-Wen Chen, William Ho, Maxim Topaz +2
eess.ASarXiv:2608.28966v12026
Factorized Neural Transducer for Efficient Language Model Adaptation
Xie Chen, Zhong Meng, Sarangarajan Parthasarathy +1
cs.CLeess.ASarXiv:2110.01500v52021
Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation
Yi Luo, Nima Mesgarani
cs.SDcs.LGeess.ASarXiv:1809.07454v32018
Audio-Visual Segmentation
Jinxing Zhou, Jianyuan Wang, Jiayi Zhang +7
cs.CVcs.MMcs.SDarXiv:2207.05042v32022
TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization
Chia-Yu Hung, Navonil Majumder, Zhifeng Kong +6
cs.SDcs.AIcs.CLarXiv:2412.21037v22024
Recent Advances in Discrete Speech Tokens: A Review
Yiwei Guo, Zhihan Li, Hankun Wang +7
eess.AScs.AIcs.MMarXiv:2502.06490v42025
VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
Chaoyou Fu, Haojia Lin, Xiong Wang +13
cs.CVcs.SDeess.ASarXiv:2501.01957v42025
CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement
Sherif Abdulatif, Ruizhe Cao, Bin Yang
cs.SDcs.AIcs.LGarXiv:2209.11112v32022
TF-GridNet: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separation
Zhong-Qiu Wang, Samuele Cornell, Shukjae Choi +3
cs.SDeess.ASarXiv:2209.03952v22022
JEPA as a Neural Tokenizer: Learning Robust Speech Representations with Density Adaptive Attention
Georgios Ioannides, Christos Constantinou, Aman Chadha +4
cs.SDcs.AIcs.LGarXiv:2512.07168v12025
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
Yu Zhang, Daniel S. Park, Wei Han +23
eess.AScs.CLcs.LGarXiv:2109.13226v32021
Formal, Executable and Explainable Runtime Monitoring of Spoken Air Traffic Control Operational Procedures
Roberto Luvini, Giacomo Longo, Alessandro Armando +1
cs.AIcs.CLeess.ASarXiv:2608.25926v12026
Omni-AVSR: Towards Unified Multimodal Speech Recognition with Large Language Models
Umberto Cappellazzo, Xubo Liu, Pingchuan Ma +2
eess.AScs.CVcs.SDarXiv:2511.07253v32025

Audio and Speech Processing

From Scores to Evidence: Auditable Decisions Can Improve Speech Deepfake Detection

Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction

Step-Audio-R1.5 Technical Report

Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation

Imputer: Sequence Modelling via Imputation and Dynamic Programming

JASPER: Joint Audio and Speech Pre-trained Encoder Representations

UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Training

Interpreting hierarchical organisation of speaker embeddings

Listen As You Wish: Audio based Event Detection via Text-to-Audio Grounding in Smart Cities

HiKE: Hierarchical Evaluation Framework for Korean-English Code-Switching Speech Recognition

Pisets: A Robust Speech Recognition System for Lectures and Interviews

End-to-end Listen, Look, Speak and Act

Symbiotic Architecture for Post-Hoc Audio Extension of Frozen Language Models

DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation

Hearing the Whispers: Black-Box Membership Inference Attacks on Finetuned TTS Models

Domain-Incremental Learning for Generative Speech Enhancement

MMDisCo: Multi-Modal Discriminator-Guided Cooperative Diffusion for Joint Audio and Video Generation

Does Your Voice Assistant Remember? Analyzing Conversational Context Recall and Utilization in Voice Interaction Models

Unifying Score and Performance for Fine-Grained Music Understanding in Audio-Language Models

Glance and Gaze: A Collaborative Learning Framework for Single-channel Speech Enhancement

StreamHear: Domain-Adapted Pseudo-Labeling for Semi-Supervised Streaming Speech Recognition

Review of end-to-end speech synthesis technology based on deep learning

MusGU+: Toward a Musician-Centered Evaluation Framework and Discovery Tool for Generative Music AI

Jukebox: A Generative Model for Music

Helping Music Co-Creation Agents 'Listen' Well: Hierarchical Self-Supervised World Models for Understanding and Generation

Baichuan-Audio: A Unified Framework for End-to-End Speech Interaction

StepAudio 3 Gen Technical Report

Frontend-Assisted Large Audio-Language Models for SLT2026 SmartGlasses Challenge

Controlled Acquisition and Abstention in Three-Channel Score Conflicts

CAL-MOS: Bridging Layers with Adapters for Robust MOS Prediction Across Speech Foundation Models

NVAlign: Direct-Gradient Optimization for Non-Verbal Control in Continuous Autoregressive Flow Matching Text-to-Speech

SSAST: Self-Supervised Audio Spectrogram Transformer

rVAD: An Unsupervised Segment-Based Robust Voice Activity Detection Method

CIViC: Evidence Gates for Audio-Conditioned Visual Prediction on Proxy Tasks

Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Language Model Can Listen While Speaking

MOV-AAD: A Large-Scale Multimodal Dataset for Auditory Attention Decoding During Moving Conversations

Fun-ASR Technical Report

Perception-Inspired Bayesian Causal Fusion for Audiovisual Source Localization

GAN-based Joint Dereverberation and Directional Filtering

SpeechBrain: A General-Purpose Speech Toolkit

FastPitch: Parallel Text-to-speech with Pitch Prediction

TasNet: time-domain audio separation network for real-time, single-channel speech separation

DuplexSLA: A Full-Duplex Spoken Language Model with Synchronized Speech, Language, and Action

AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations

EmoRES-TTS: Residual-Enhanced Vector Steering for Emotional Speech Generation

When Vocal Tone and Literal Meaning Diverge: An Acoustic-Semantic Incongruity Study for Large Audio-Language Models

Factorized Neural Transducer for Efficient Language Model Adaptation

Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation

Audio-Visual Segmentation

TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization

Recent Advances in Discrete Speech Tokens: A Review

VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement

TF-GridNet: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separation

JEPA as a Neural Tokenizer: Learning Robust Speech Representations with Density Adaptive Attention

BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition

Formal, Executable and Explainable Runtime Monitoring of Spoken Air Traffic Control Operational Procedures

Omni-AVSR: Towards Unified Multimodal Speech Recognition with Large Language Models