PaperlayerView exact revision on arXiv

Source-linked AI summary

Speech Recognition by Machine, A Review

M. A. Anusuya, S. K. Katti

arXiv:1001.2267v1cs.CL

TL;DR

Automatic speech recognition still faces accuracy challenges from context, speaker, and environmental variation. This paper surveys six decades of ASR themes, methods, problems, and applications, concluding that robust recognition across environmental and speaker variation remains an important goal.

  • Problem

    Automatic speech recognition accuracy remains challenging under variations in context, speakers, and environment.

  • Method

    The paper provides a chronological survey comparing speech-recognition methods, problems, advances, applications, and system-design stages.

  • Results

    The review describes substantial progress across six decades while reporting that robust recognition under environmental and speaker variation still requires further work.

  • Takeaways & Limitations

    Speech recognition remains a significant area of machine intelligence and human-machine interaction with continuing technological and societal impact.

Abstract

from arXiv · show

This paper presents a brief survey on Automatic Speech Recognition and discusses the major themes and advances made in the past 60 years of research, so as to provide a technological perspective and an appreciation of the fundamental progress that has been accomplished in this important area of speech communication. After years of research and development the accuracy of automatic speech recognition remains one of the important research challenges (e.g., variations of the context, speakers, and environment).The design of Speech Recognition system requires careful attentions to the following issues: Definition of various types of speech classes, speech representation, feature extraction techniques, speech classifiers, database and performance evaluation. The problems that are existing in ASR and the various techniques to solve these problems constructed by various research workers have been presented in a chronological order. Hence authors hope that this work shall be a contribution in the area of speech recognition. The objective of this review paper is to summarize and compare some of the well known methods used in various stages of speech recognition system and identify research topic and applications which are at the forefront of this exciting and challenging field.

I. INTRODUCTION

Speech recognition, also called Automatic Speech Recognition or computer speech recognition, converts a speech signal into a sequence of words using a computer algorithm.

  • Speech recognition converts a speech signal into a sequence of words through a computer-implemented algorithm.

1.2 Basic Model of Speech Recognition:

The paper frames speech processing as an effort to model human verbal communication and describes speech recognition as a system with front-end, model, language-model, and search units.

  • Speech-processing research has been motivated by attempts to build mechanical models of human verbal communication.
  • Speech recognition aims to let computers follow human voice commands and understand human languages.
  • The basic recognition model contains front-end, model, language-model, and search units.

1.3 Types of Speech Recognition

Speech recognition systems are classified by the kinds of utterances they can recognize, ranging from isolated words to natural spontaneous speech.

  • Isolated-word recognizers process one word or utterance at a time and typically require quiet before and after it.
  • Connected-word systems allow separate utterances to run together with minimal pauses.
  • Continuous-speech recognizers allow nearly natural speaking but must determine utterance boundaries.
  • Spontaneous-speech recognition must handle natural features such as run-together words, filled pauses, and slight stutters.

1.4 Applications of Speech Recognition:

The paper presents speech recognition applications and organizes recognition approaches around acoustic-phonetic, pattern-recognition, and artificial-intelligence perspectives, while identifying environmental, channel, speaker, and speech-style factors.

  • Speech recognition applications include automated transcription, telematics, air travel information, multimodal interaction, court reporting, and grocery-related uses.
  • ASR design must consider environment, transducer, channel, speaker characteristics, and speech styles.
  • The paper identifies acoustic-phonetic, pattern-recognition, and artificial-intelligence approaches to speech recognition.

2.1 Acoustic phonetic approach:

The acoustic-phonetic approach identifies finite phonetic units through their acoustic properties, then analyzes, segments, and labels speech accordingly.

  • The acoustic-phonetic approach assumes spoken language contains finite, distinctive phonemes characterized by acoustic properties over time.
  • It begins with spectral analysis and feature detection to describe broad acoustic properties of phonetic units.
  • The speech signal is then segmented into stable acoustic regions and assigned one or more phonetic labels.

2.2 Pattern Recognition approach:

Pattern recognition represents speech using trained templates or statistical models and compares unknown utterances against those representations. Stochastic modeling addresses uncertainty from confusable sounds, speaker variability, contextual effects, and homophones, while HMMs offer a more general probabilistic framework but limited interpretability.

  • Pattern recognition uses pattern training and comparison to create consistent speech representations from labeled training samples.
  • Template-based recognition stores prototypical word patterns and selects the category whose reference template best matches an unknown utterance.
  • Stochastic modeling uses probabilistic models to represent uncertainty from confusable sounds, speaker variability, contextual effects, and homophones.
  • Compared with templates, HMMs have a broader mathematical foundation and integrate knowledge sources more easily, but typically underperforming templates is not reported; templates typically underperform HMMs.
  • HMMs provide limited insight into recognition processes, making system-error analysis difficult, although incorporating knowledge has improved HMM-based systems.

2.3. Dynamic Time Warping(DTW):

Dynamic time warping compares sequences that differ in timing or speed by nonlinearly aligning them in the time dimension. In speech recognition, it supports matching across speaking-rate variation using constrained dynamic-programming optimization.

  • Dynamic time warping measures similarity between sequences whose timing or speed varies.
  • DTW finds an optimal match by warping sequences nonlinearly in time, making similarity less dependent on certain temporal variations.
  • DTW has been applied to automatic speech recognition to cope with different speaking speeds.
  • Its matching restrictions can include monotonicity, and dynamic programming performs the optimization.

2.4. Vector Quantization(VQ):

Vector quantization applies compact codebooks to ASR reference models and codebook search, reducing representation size without explicitly preserving temporal order.

  • Vector quantization is applied to ASR through compact codebooks for reference models and codebook search.
  • Basic VQ codebooks lack explicit temporal information because their entries are unordered and ignore phonetic-segment order and relative durations.
  • Indirect durational cues remain because longer acoustic segments contribute more frames to training and are more likely to determine code words.

2.5. Artificial Intelligence approach (Knowledge Based approach)

The knowledge-based artificial intelligence approach incorporates linguistic, phonetic, and spectrographic expertise into speech recognition, either explicitly through rules or indirectly through model and algorithm design. Its integration of multiple knowledge sources remains unresolved, while connectionist variants require effective learning strategies and substantial training data.

  • The artificial intelligence approach combines acoustic-phonetic and pattern-recognition methods to model speech recognition using expert knowledge.Knowledge may include phonetic, linguistic, and spectrographic information.
  • Knowledge-based systems explicitly encode speech expertise as rules or procedures derived from spectrograms and linguistic or phonetic study.This distinguishes explicit knowledge engineering from approaches that use knowledge to guide other algorithms.
  • Knowledge also guides input representation, speech-unit definition, and recognition-algorithm design without being directly encoded as individual rules.The paper separates knowledge from the algorithms it helps improve.
  • Connectionist models distribute knowledge across many simple computing units and use neural networks to learn relationships among phonetic events.They can integrate phonemic, lexical, syntactic, semantic, and pragmatic knowledge for segmentation and labeling.
  • Connectionist systems depend critically on effective learning strategies, and training may require many iterations over large datasets at prohibitive cost.Their practical realization remained unresolved despite promise as cognitive models.

2.7. Support Vector Machine(SVM):

Support Vector Machines are discriminative classifiers that separate data with linear or nonlinear hyperplanes. Because they operate on fixed-length vectors, variable-length speech data must first be transformed, while maximum-margin regularization controls model complexity.

  • SVMs classify data using linear or nonlinear separating hyperplanes in a discriminative pattern-recognition framework.The method is presented as a powerful tool for pattern recognition.
  • Variable-length speech cannot be directly classified by SVMs and must be transformed into fixed-length vectors beforehand.This is the main task-specific constraint identified in the section.
  • Maximum-margin fitting provides regularization that helps the classifier generalize better.The fitting function is described as a generalized linear classifier.
  • SVMs control model complexity through the VC dimension rather than by simply reducing features or hidden units.The paper contrasts this with conventional statistical and neural-network methods.

2.8. Taxonomy of Speech Recognition:

The survey organizes speech-recognition systems around feature extraction, classification, performance evaluation, and historical development. It traces progress from early hardware and phonetic systems through statistical, template, dynamic-alignment, and connected-speech approaches.

  • Feature extraction: Feature extraction seeks a compact sequence of feature vectors that preserves information useful for discriminating similar-sounding speech sounds.The survey presents feature-extraction methods in Table 4.
  • Pattern classification: Pattern classifiers may use labeled supervision or unsupervised clustering, with vector quantization producing a codebook of cluster centers.The codebook indexes new vectors by their nearest cluster center.
  • Pattern classification: Classifier design includes similarity-based methods such as template matching and minimum-distance classification, with prototype choice and metrics affecting success.Nearest-mean, vector-quantization, and learning-vector-quantization methods are described as prototype-based options.
  • Performance evaluation: Speech-recognition performance is commonly reported using accuracy measures such as WER and speed measures such as the real-time factor.SWER and CSR are additional accuracy measures.
  • Performance evaluation: WER is derived from word-level Levenshtein distance after aligning recognized and reference word sequences, accounting for substitutions, deletions, and insertions.The reference length is N; correctly recognized words are represented by H.
  • Historical development: Historically, research progressed from 1950s acoustic-phonetic and filter-bank systems to 1960s special-purpose hardware, dynamic programming, and continuous phoneme tracking.Early systems recognized vowels, digits, or phonemes under constrained conditions.
  • Historical development: The 1970s established viable isolated-word recognition and finite-state search, while the 1980s focused on robust recognition of connected word sequences.DARPA systems also explored parallel asynchronous knowledge sources and blackboard integration.

6.6. 2000-2009:

Research during 2000–2009 addressed adaptation, discriminative modeling, noisy and spontaneous speech, multimodal recognition, databases, and data-driven methods. Reported advances included improved robustness, recognition accuracy, and modeling of natural speech conditions.

  • Spontaneous speech recognition: Spontaneous-speech recognition accuracy decreases sharply relative to read speech, but explicit spontaneous-effect modeling and spontaneous-speech training data can improve word accuracy.The reviewed analyses also report that fluent spontaneous speech can achieve word accuracies equivalent to read speech.
  • Additional directions: The period also examined multimodal audio–visual recognition, confidence measures, speech databases, and data-driven decomposition and decision-tree methods.The data-driven review reports that PIC outperforms ML and MDL criteria for compact decision-tree construction.
  • Recognition models: Discriminative SVM rescoring was applied to maximum-likelihood HMMs, significantly reducing error rates compared with standard ML training.The approach targeted both discrete and continuous output-probability HMMs.
  • Robust speech recognition: Noisy-speech research explored log-spectral feature enhancement, switching linear dynamic models, environment-dependent HMM parameters, and combined speech-noise modeling.These methods were designed to improve recognition performance under noisy conditions.

7.5. Air Travel Information System(ATIS):

The paper places ATIS and broader speech-recognition progress within unresolved scientific and engineering questions. Despite substantial progress, human speech processing remains poorly understood and robust recognition across environmental and speaker variation remains an open goal.

  • 7.5. Air Travel Information System(ATIS): ATIS is commonly used to evaluate word-error performance and simulates a realistic spontaneous-conversation environment.Its application setting makes it relevant to evaluating recognition beyond read speech.
  • Open research questions: The review identifies unresolved questions about speech contrast, variability, individuality, representation, units, formalism, adaptation, learning, and architecture.These questions span both speech science and speech-processing system design.
  • Open research questions: After more than 10 years, the paper reports that clear answers to these 20 questions were still unavailable.This is presented as a continuing boundary of knowledge rather than a completed theoretical framework.
  • Progress and remaining gaps: The review describes major technological progress over 60 years while emphasizing that human speech processing remains limited in current understanding.It connects the historical survey with the continuing gap between machine and human recognition.
  • Progress and remaining gaps: A robust speech-recognition system is described as needing effectiveness across environmental conditions and speaker variability.The statement defines robustness broadly rather than restricting it to a single dataset or operating condition.

Selection for Speech Recognition , IEEE Transactions

The cited literature covers feature extraction, HMMs, SVM training, fuzzy modeling, noisy-speech recognition, template matching, and signal decomposition. Together, these references represent statistical, discriminative, model-based, and data-driven directions in speech recognition research.

  • Features and representations: Feature and representation methods include empirical mode decomposition, vector quantization, subspace methods, covariance estimation, and projection pursuit.The cited works span signal decomposition and statistical pattern-recognition techniques.
  • Modeling and classification: Other cited approaches include fuzzy HMMs, variable-parameter Gaussian-mixture HMMs, and phoneme recognition using HMMs.The methods address recognition and modeling through probabilistic or fuzzy variants of hidden Markov models.
  • Recognition and enhancement: The bibliography also covers template-based continuous recognition, iterative speech-feature enhancement, and structured speech modeling.These references extend the review beyond a single recognition architecture.
Loading 1001.2267v1…