Audio and Speech Processing

Papers filed under eess.AS on arXiv, each one already summarized by Paperlayer. Open any of them to read the summary beside the original PDF, with every point linked to the line, figure, or table it came from.

Search paper metadata (including unsummarized papers)

1 to 60 of 1,648

Bottom-up Broadcast Neural Network For Music Genre Classification
Caifeng Liu, Lin Feng, Guochao Liu +2
cs.SDcs.AIeess.ASarXiv:1901.08928v12019
Joint Speech Recognition and Speaker Diarization via Sequence Transduction
Laurent El Shafey, Hagen Soltau, Izhak Shafran
cs.CLcs.SDeess.ASarXiv:1907.05337v12019
Spectrogram features for audio and speech analysis
Ian McLoughlin, Lam Pham, Yan Song +7
eess.AScs.AIcs.LGarXiv:2603.14917v12026
RubatoCodec: Budgeted Adaptive Speech Tokenization with Streaming Decoding
Chihiro Arata, Shohei Taniguchi, Gouki Minegishi +1
cs.SDeess.ASarXiv:2610.29434v12026
MAESTRO: Matched Speech Text Representations through Modality Matching
Zhehuai Chen, Yu Zhang, Andrew Rosenberg +4
cs.CLcs.SDeess.ASarXiv:2204.03409v22022
ChatMusician: Understanding and Generating Music Intrinsically with LLM
Ruibin Yuan, Hanfeng Lin, Yi Wang +32
cs.SDcs.AIcs.CLarXiv:2402.16153v12024
ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering
Yakun Song, Zhuo Chen, Xiaofei Wang +2
cs.CLcs.AIcs.SDarXiv:2401.07333v12024
DQ-Whisper: Joint Distillation and Quantization for Efficient Multilingual Speech Recognition
Hang Shao, Bei Liu, Wei Wang +2
cs.SDcs.CLeess.ASarXiv:2305.10788v22023
AudioRouter: Data Efficient Audio Understanding via RL based Dual Reasoning
Liyang Chen, Hongkai Chen, Yujun Cai +3
cs.SDcs.AIeess.ASarXiv:2602.10439v12026
Soft Clustering Anchors for Self-Supervised Speech Representation Learning in Joint Embedding Prediction Architectures
Georgios Ioannides, Adrian Kieback, Judah Goldfeder +5
eess.AScs.AIcs.LGarXiv:2602.09040v12026
WavJEPA: Semantic learning unlocks robust audio foundation models for raw waveforms
Goksenin Yuksel, Pierre Guetschel, Michael Tangermann +2
cs.SDeess.ASarXiv:2509.23238v32025
DIFFA: Large Language Diffusion Models Can Listen and Understand
Jiaming Zhou, Hongjie Chen, Shiwan Zhao +9
cs.SDeess.ASarXiv:2507.18452v32025
EnvSDD: Benchmarking Environmental Sound Deepfake Detection
Han Yin, Yang Xiao, Rohan Kumar Das +4
cs.SDcs.AIeess.ASarXiv:2505.19203v22025
Automatic Speech Recognition for African Low-Resource Languages: Challenges and Future Directions
Sukairaj Hafiz Imam, Babangida Sani, Dawit Ketema Gete +6
cs.CLcs.SDeess.ASarXiv:2505.11690v12025
AI-Generated Music Detection and its Challenges
Darius Afchar, Gabriel Meseguer-Brocal, Romain Hennequin
cs.SDeess.ASarXiv:2501.10111v12025
VoxEval: Benchmarking the Knowledge Understanding Capabilities of End-to-End Spoken Language Models
Wenqian Cui, Xiaoqi Jiao, Ziqiao Meng +1
cs.CLcs.SDeess.ASarXiv:2501.04962v42025
Style-Talker: Finetuning Audio Language Model and Style-Based Text-to-Speech Model for Fast Spoken Dialogue Generation
Yinghao Aaron Li, Xilin Jiang, Jordan Darefsky +2
cs.CLcs.AIeess.ASarXiv:2408.11849v12024
WenetSpeech4TTS: A 12,800-hour Mandarin TTS Corpus for Large Speech Generation Model Benchmark
Linhan Ma, Dake Guo, Kun Song +7
eess.ASarXiv:2406.05763v32024
A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech
Li-Wei Chen, Shinji Watanabe, Alexander Rudnicky
eess.AScs.AIcs.LGarXiv:2302.04215v12023
A-JEPA: Joint-Embedding Predictive Architecture Can Listen
Zhengcong Fei, Mingyuan Fan, Junshi Huang
cs.SDcs.CVeess.ASarXiv:2311.15830v32023
Towards human-like spoken dialogue generation between AI agents from written dialogue
Kentaro Mitsui, Yukiya Hono, Kei Sawada
cs.CLcs.LGcs.SDarXiv:2310.01088v12023
A Light Weight Model for Active Speaker Detection
Junhua Liao, Haihan Duan, Kanghui Feng +3
cs.CVcs.SDeess.ASarXiv:2303.04439v12023
TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation
Rongjie Huang, Jinglin Liu, Huadai Liu +4
cs.CLcs.SDeess.ASarXiv:2205.12523v22022
RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning
Nan Jiang, Sheng Jin, Zhiyao Duan +1
cs.LGcs.HCeess.ASarXiv:2002.03082v12020
Audio-Cogito: Towards Deep Audio Reasoning in Large Audio Language Models
Longhao Li, Hongjie Chen, Zehan Li +5
eess.ASarXiv:2604.12527v32026
CompSpoof: A Dataset and Joint Learning Framework for Component-Level Audio Anti-spoofing Countermeasures
Xueping Zhang, Yechen Wang, Linxi Li +2
cs.SDeess.ASarXiv:2509.15804v22025
URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models
Ruiqi Yan, Xiquan Li, Wenxi Chen +5
cs.CLeess.ASarXiv:2502.17810v32025
Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition
Shuai Zhang, Jiangyan Yi, Zhengkun Tian +2
cs.CLcs.LGcs.SDarXiv:2002.08126v12020
Monolingual Data Selection Analysis for English-Mandarin Hybrid Code-switching Speech Recognition
Haobo Zhang, Haihua Xu, Van Tung Pham +2
eess.ASarXiv:2006.07094v22020
Towards End-to-End Code-Switching Speech Recognition
Ne Luo, Dongwei Jiang, Shuaijiang Zhao +3
cs.CLeess.ASarXiv:1810.13091v22018
Multi-Encoder-Decoder Transformer for Code-Switching Speech Recognition
Xinyuan Zhou, Emre Yılmaz, Yanhua Long +2
eess.AScs.SDarXiv:2006.10414v12020
Do Audio-Language Models Understand Linguistic Variations?
Ramaneswaran Selvakumar, Sonal Kumar, Hemant Kumar Giri +4
cs.SDcs.LGeess.ASarXiv:2410.16505v22024
FakeMusicCaps: a Dataset for Detection and Attribution of Synthetic Music Generated via Text-to-Music Models
Luca Comanducci, Paolo Bestagini, Stefano Tubaro
eess.AScs.SDarXiv:2409.10684v22024
Autoregressive Speech Synthesis without Vector Quantization
Lingwei Meng, Long Zhou, Shujie Liu +9
cs.CLcs.SDeess.ASarXiv:2407.08551v22024
Adversarial synthesis based data-augmentation for code-switched spoken language identification
Parth Shastri, Chirag Patil, Poorval Wanere +3
eess.AScs.CVcs.LGarXiv:2205.15747v22022
Representation Learning of Music Using Artist, Album, and Track Information
Jongpil Lee, Jiyoung Park, Juhan Nam
cs.IRcs.MMcs.SDarXiv:1906.11783v12019
Learning Contextual Tag Embeddings for Cross-Modal Alignment of Audio and Tags
Xavier Favory, Konstantinos Drossos, Tuomas Virtanen +1
cs.SDcs.IRcs.LGarXiv:2010.14171v12020
A Framework for Generative and Contrastive Learning of Audio Representations
Prateek Verma, Julius Smith
cs.SDcs.LGeess.ASarXiv:2010.11459v22020
Representation Learning of Music Using Artist Labels
Jiyoung Park, Jongpil Lee, Jangyeon Park +2
cs.SDeess.ASarXiv:1710.06648v22017
Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR
Balázs Tarján, György Szaszák, Tibor Fegyó +1
eess.AScs.CLarXiv:2007.06949v32020
Self-supervised Learning for Speech Enhancement
Yu-Che Wang, Shrikant Venkataramani, Paris Smaragdis
eess.AScs.SDarXiv:2006.10388v12020
VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech
Chenpeng Du, Yiwei Guo, Hankun Wang +6
eess.AScs.SDarXiv:2401.14321v52024
MultiSpeech: Multi-Speaker Text to Speech with Transformer
Mingjian Chen, Xu Tan, Yi Ren +5
eess.AScs.CLcs.LGarXiv:2006.04664v22020
AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanisms
Kou Tanaka, Hirokazu Kameoka, Takuhiro Kaneko +1
eess.AScs.LGcs.SDarXiv:1811.04076v12018
Wav2vec-C: A Self-supervised Model for Speech Representation Learning
Samik Sadhu, Di He, Che-Wei Huang +6
eess.AScs.LGcs.SDarXiv:2103.08393v22021
EasyCom: An Augmented Reality Dataset to Support Algorithms for Easy Communication in Noisy Environments
Jacob Donley, Vladimir Tourbabin, Jung-Suk Lee +6
cs.SDcs.CVcs.LGarXiv:2107.04174v22021
DuET: Note-Frame Dual Encoding for Piano Performance-to-Score Transcription
Wei Zeng, Ye Wang
cs.SDcs.LGeess.ASarXiv:2610.21467v12026
Normalise or condition? Noise-floor front-ends for on-board keyword spotting under UAV rotor ego-noise
Yida Lin, Bing Xue, Mengjie Zhang +2
cs.SDcs.CVeess.ASarXiv:2609.31699v12026
DiffVQE2: An Efficient Low-delay Diffusion Model for Acoustic Echo and Noise Control
Haljan Lugo, Ernst Seidel, Pejman Mowlaee +2
eess.AScs.SDarXiv:2609.31703v12026
AVSegFormer: Audio-Visual Segmentation with Transformer
Shengyi Gao, Zhe Chen, Guo Chen +2
cs.CVcs.LGcs.SDarXiv:2307.01146v42023
Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling
Jing-Xuan Zhang, Genshun Wan, Jianqing Gao +3
cs.CVeess.ASarXiv:2610.19559v12026
Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
Julian Salazar, Katrin Kirchhoff, Zhiheng Huang
eess.AScs.CLcs.LGarXiv:1901.10055v22019
Robust and Complex Approach of Pathological Speech Signal Analysis
Jiri Mekyska, Eva Janousova, Pedro Gomez-Vilda +8
cs.SDeess.ASarXiv:2203.09402v12022
What Does LALM Post-training Learn? Rethinking Audio Understanding and Reasoning Capabilities
Hua-Hsuan Chen, Xuanjun Chen, Wei-Chung Lu +5
cs.SDcs.CLcs.MMarXiv:2610.19476v12026
Rethinking Batch Composition: Promoting Feature Diversity for Speech Enhancement
Mingchi Hou, Ina Kodrasi
eess.ASarXiv:2610.21803v12026
ElasticAST: An Audio Spectrogram Transformer for All Length and Resolutions
Jiu Feng, Mehmet Hamza Erol, Joon Son Chung +1
cs.SDcs.AIeess.ASarXiv:2407.08691v12024
Variable-rate discrete representation learning
Sander Dieleman, Charlie Nash, Jesse Engel +1
cs.LGcs.CLcs.SDarXiv:2103.06089v12021
SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation
Zhehuai Chen, He Huang, Andrei Andrusenko +6
cs.CLcs.HCcs.SDarXiv:2310.09424v12023
A Comparative Study of Glottal Source Estimation Techniques
Thomas Drugman, Baris Bozkurt, Thierry Dutoit
cs.SDcs.CLeess.ASarXiv:2001.00840v12019
Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis
Desh Raj, Pavel Denisov, Zhuo Chen +11
eess.AScs.SDarXiv:2011.02014v12020

Audio and Speech Processing

Bottom-up Broadcast Neural Network For Music Genre Classification

Joint Speech Recognition and Speaker Diarization via Sequence Transduction

Spectrogram features for audio and speech analysis

RubatoCodec: Budgeted Adaptive Speech Tokenization with Streaming Decoding

MAESTRO: Matched Speech Text Representations through Modality Matching

ChatMusician: Understanding and Generating Music Intrinsically with LLM

ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering

DQ-Whisper: Joint Distillation and Quantization for Efficient Multilingual Speech Recognition

AudioRouter: Data Efficient Audio Understanding via RL based Dual Reasoning

Soft Clustering Anchors for Self-Supervised Speech Representation Learning in Joint Embedding Prediction Architectures

WavJEPA: Semantic learning unlocks robust audio foundation models for raw waveforms

DIFFA: Large Language Diffusion Models Can Listen and Understand

EnvSDD: Benchmarking Environmental Sound Deepfake Detection

Automatic Speech Recognition for African Low-Resource Languages: Challenges and Future Directions

AI-Generated Music Detection and its Challenges

VoxEval: Benchmarking the Knowledge Understanding Capabilities of End-to-End Spoken Language Models

Style-Talker: Finetuning Audio Language Model and Style-Based Text-to-Speech Model for Fast Spoken Dialogue Generation

WenetSpeech4TTS: A 12,800-hour Mandarin TTS Corpus for Large Speech Generation Model Benchmark

A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech

A-JEPA: Joint-Embedding Predictive Architecture Can Listen

Towards human-like spoken dialogue generation between AI agents from written dialogue

A Light Weight Model for Active Speaker Detection

TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation

RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning

Audio-Cogito: Towards Deep Audio Reasoning in Large Audio Language Models

CompSpoof: A Dataset and Joint Learning Framework for Component-Level Audio Anti-spoofing Countermeasures

URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Models

Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition

Monolingual Data Selection Analysis for English-Mandarin Hybrid Code-switching Speech Recognition

Towards End-to-End Code-Switching Speech Recognition

Multi-Encoder-Decoder Transformer for Code-Switching Speech Recognition

Do Audio-Language Models Understand Linguistic Variations?

FakeMusicCaps: a Dataset for Detection and Attribution of Synthetic Music Generated via Text-to-Music Models

Autoregressive Speech Synthesis without Vector Quantization

Adversarial synthesis based data-augmentation for code-switched spoken language identification

Representation Learning of Music Using Artist, Album, and Track Information

Learning Contextual Tag Embeddings for Cross-Modal Alignment of Audio and Tags

A Framework for Generative and Contrastive Learning of Audio Representations

Representation Learning of Music Using Artist Labels

Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR

Self-supervised Learning for Speech Enhancement

VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech

MultiSpeech: Multi-Speaker Text to Speech with Transformer

AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanisms

Wav2vec-C: A Self-supervised Model for Speech Representation Learning

EasyCom: An Augmented Reality Dataset to Support Algorithms for Easy Communication in Noisy Environments

DuET: Note-Frame Dual Encoding for Piano Performance-to-Score Transcription

Normalise or condition? Noise-floor front-ends for on-board keyword spotting under UAV rotor ego-noise

DiffVQE2: An Efficient Low-delay Diffusion Model for Acoustic Echo and Noise Control

AVSegFormer: Audio-Visual Segmentation with Transformer

Learning Visual Speech Representations via Cross-Modal Distillation and Joint Face-Lip Modeling

Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition

Robust and Complex Approach of Pathological Speech Signal Analysis

What Does LALM Post-training Learn? Rethinking Audio Understanding and Reasoning Capabilities

Rethinking Batch Composition: Promoting Feature Diversity for Speech Enhancement

ElasticAST: An Audio Spectrogram Transformer for All Length and Resolutions

Variable-rate discrete representation learning

SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation

A Comparative Study of Glottal Source Estimation Techniques

Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis