PaperlayerView exact revision on arXiv

Source-linked AI summary

Continuous online sequence learning with an unsupervised neural network model

Yuwei Cui, Subutai Ahmad, Jeff Hawkins

arXiv:1512.05463v2cs.NEq-bio.NC

TL;DR

Online sequence learning must recognize and predict temporal patterns in continuously changing sensory streams. This paper evaluates an unsupervised HTM sequence-memory model and finds comparable prediction accuracy to ARIMA, ELM, LSTM, and ESN while supporting continuous learning and branching predictions.

  • Problem

    Online sequence learning needs algorithms that learn from continuously changing sensory streams under real-time constraints.

  • Method

    The paper comparatively evaluates HTM sequence memory, using sparse temporal codes and an online unsupervised Hebbian-style learning rule, against four sequence-learning methods.

  • Results

    HTM achieves comparable prediction accuracy to ARIMA, ELM, LSTM, and ESN while learning high-order sequences and handling branching predictions in streaming data.

  • Takeaways & Limitations

    HTM sequence memory is applicable to online sequence prediction, anomaly detection, and sequence classification on noisy, time-varying data streams.

  • Takeaways & Limitations

    On Reber grammar learning, HTM achieves 98.4% accuracy versus 100% for LSTM and may require more samples for recursive grammars.

Abstract

from arXiv · show

The ability to recognize and predict temporal sequences of sensory inputs is vital for survival in natural environments. Based on many known properties of cortical neurons, hierarchical temporal memory (HTM) sequence memory is recently proposed as a theoretical framework for sequence learning in the cortex. In this paper, we analyze properties of HTM sequence memory and apply it to sequence learning and prediction problems with streaming data. We show the model is able to continuously learn a large number of variable-order temporal sequences using an unsupervised Hebbian-like learning rule. The sparse temporal codes formed by the model can robustly handle branching temporal sequences by maintaining multiple predictions until there is sufficient disambiguating evidence. We compare the HTM sequence memory with other sequence learning algorithms, including statistical methods: autoregressive integrated moving average (ARIMA), feedforward neural networks: online sequential extreme learning machine (ELM), and recurrent neural networks: long short-term memory (LSTM) and echo-state networks (ESN), on sequence prediction problems with both artificial and real-world data. The HTM model achieves comparable accuracy to other state-of-the-art algorithms. The model also exhibits properties that are critical for sequence learning, including continuous online learning, the ability to handle multiple predictions and branching sequences with high order statistics, robustness to sensor noise and fault tolerance, and good performance without task-specific hyper- parameters tuning. Therefore the HTM sequence memory not only advances our understanding of how the brain may solve the sequence learning problem, but is also applicable to a wide range of real-world problems such as discrete and continuous sequence prediction, anomaly detection, and sequence classification.

1. Introduction

The paper addresses sequence learning for continuously changing sensory streams, where conventional benchmarks often assume stationary train/test statistics. It presents and evaluates an unsupervised HTM sequence-memory model against statistical and neural methods, showing comparable prediction accuracy alongside biologically motivated online-learning properties.

  • Motivation: Temporal sequence recognition and prediction are critical to brain functions including speech recognition, active tactile perception, and natural vision.The cortex continuously processes sensory streams and builds spatiotemporal models of the world.
  • Problem: Standard sequence-learning benchmarks often separate training and testing data and assume both share similar statistics, limiting evaluation in dynamic, non-stationary scenarios.The paper highlights a mismatch between benchmark assumptions and real-time streaming environments.
  • Approach: HTM sequence memory represents complex sequences with sparse distributed temporal codes and learns online using an unsupervised Hebbian-style rule.Its neuron model incorporates properties of pyramidal cells and active dendrites.
  • Evaluation: HTM sequence memory is compared with ARIMA, ELM, LSTM, and ESN for sequence prediction.The comparison spans statistical forecasting, feedforward online learning, and recurrent neural-network methods.
  • Findings: HTM sequence memory achieves comparable prediction accuracy to the other evaluated techniques while exhibiting additional features relevant to sequence learning.The introduction identifies these features as part of the paper’s comparative contribution, with later sections applying the methods to artificial and real-world data.

2. Criteria for a good sequence learning algorithm

A good real-time sequence-learning algorithm should process changing, noisy data streams under limited resources while adapting continuously and making context-sensitive predictions. The section identifies high-order predictions, simultaneous alternatives, noise robustness, fault tolerance, and minimal hyperparameter tuning as key criteria.

  • Continuous learning: Streaming algorithms must learn continuously from changing data and rapidly adapt to newly emerging patterns.Real-time streams require processing under limited computing and storage capabilities.
  • Multiple simultaneous predictions: Algorithms should maintain multiple plausible future predictions and evaluate their likelihoods online when temporal information is ambiguous.This requires outputting a distribution of possible future outcomes rather than only the single best prediction.
  • High-order predictions: Algorithms should learn the relevant Markov order automatically to capture dependencies spanning multiple time steps.High-order prediction means using the minimum necessary history for accurate predictions.
  • Noise robustness and fault tolerance: Algorithms should remain robust to sensor noise, transmission errors, missing data, and neural-system faults.Fault tolerance and noise robustness support flexibility and broad applicability, including neuromorphic processors.
  • No hyperparameter tuning: An ideal algorithm should minimize reliance on task-specific hyperparameter tuning, which creates challenges for automated data-stream mining.Most machine-learning methods require manually guided searches over hyperparameter spaces using cross-validation performance.

3. HTM sequence memory

HTM sequence memory uses dendritic predictive states and dual sparse representations to encode current inputs alongside temporal context. Its Hebbian-like learning reinforces context-bearing connections, enabling multiple predictions and robustness to noise and component loss.

  • Neuron model: HTM neurons use proximal and distal dendritic zones, with active distal segments depolarizing cells into a predictive state that detects temporal context.Each distal segment contains lateral synapses from neurons within the layer; sufficient coincident activity activates the segment without immediately firing the cell.
  • Sequence representation: The network represents high-order sequences with sparse distributed codes that simultaneously encode current feedforward input and previous sequence context.Inter-columnar inhibition activates the top 2% of columns receiving the strongest feedforward input, while cell-level activity represents temporal context.
  • Sequence representation: Sparse temporal representations let the model maintain multiple predictions when sequence context is ambiguous, resolving branches as disambiguating evidence arrives.For example, presenting “B” without context can produce predictions for both subsequent branches in the learned sequences A-B-C-D and X-B-C-Y.
  • Robustness: Distributed co-activation makes the model robust to input noise and system faults, including loss of neurons and synapses.The robustness follows from information being stored across multiple co-active cells rather than in a single element.
  • Learning rule: Lateral connections learn through a Hebbian-like rule that reinforces the dendritic segment responsible when a depolarized cell subsequently becomes active.If no active-column cell was predicted, the model selects the cell with the most activated segment and reinforces it by increasing active-synapse permanence and decreasing inactive-synapse permanence.

4. High-order sequence prediction with artificial data

On artificial high-order sequence streams, HTM learned single- and multiple-ending sequences online, adapted after contradictory endings, and achieved perfect performance through order 100. Its performance remained sensitive to temporal noise, while the experiments also evaluated robustness to neuron removal.

  • Single prediction: HTM quickly achieved perfect accuracy when high-order sequences had a single possible ending.LSTM also learned the sequences given a sufficiently large learning window.
  • Adaptation to changes: HTM quickly recovered after sequence endings were swapped, whereas batch LSTM and ELM recovered slowly because buffered data contained contradictory information.The modification tested whether models could forget old sequences and learn the changed stream.
  • Multiple predictions: HTM rapidly achieved perfect accuracy when each high-order context had either two or four possible endings.The model can maintain many simultaneous predictions when the dataset requires them.
  • Variable-order sequences: HTM achieved perfect prediction performance for variable-order sequences up to order 100, with the required training sequences increasing linearly with sequence order.The model reached 50% accuracy sooner because first-order knowledge was sufficient for that partial performance.
  • Noise robustness: With temporal noise throughout training, HTM behaved like a first-order model at about 0.5 accuracy, while LSTM performed slightly better; after noise injection, both models dropped rapidly.The results show sensitivity to disrupted high-order context and dependence of LSTM performance on training history.

5. Prediction of taxi passenger demand

The paper evaluates HTM sequence memory for 30-minute New York City taxi-demand prediction against ARIMA, ELM, LSTM, and ESN. HTM performs comparably to LSTM on both error metrics and adapts rapidly when demand patterns change.

  • Task and data: Taxi passenger counts were aggregated into 30-minute intervals, producing sequences with rich daily and weekly temporal patterns.The data came from a public New York City Transportation Authority stream.
  • Methods and metrics: The evaluation used mean absolute percentage error and negative log-likelihood to assess point estimates and predicted future-input distributions.ARIMA and LSTM were converted to online learning through repeated retraining with buffered historical samples.
  • Prediction performance: HTM sequence memory achieved comparable performance to LSTM on both error metrics, while both had much lower error than ELM, ESN, and ARIMA.The comparison covered statistical, feedforward-neural, and recurrent-neural sequence prediction algorithms.
  • Adaptation to distribution changes: After weekday traffic patterns were artificially changed by 20%, prediction error immediately increased for HTM and LSTM.Weekday morning traffic was decreased and weekday night traffic increased beginning April 1st.
  • Adaptation to distribution changes: HTM prediction error returned to its prior level in about two weeks after the manipulated patterns were introduced.The figure passage reports that HTM had better accuracy than LSTM on both MAPE and negative log-likelihood after the manipulation.

6. Discussion and conclusions

The discussion presents HTM sequence memory as a continuously learning, SDR-based model for streaming sequence problems, while identifying limitations involving long-term dependencies, temporal noise, grammar learning, and untested high-dimensional data.

  • Continuous online learning: HTM sequence memory learns continuously from each data point through unsupervised Hebbian-like associative mechanisms, enabling rapid adaptation to changing data statistics.Unlike batch-training approaches, it does not need to store a batch as a training dataset.
  • Sparse representations: Sparse distributed representations give HTM a rich temporal-sequence representation that can support prediction, anomaly detection, and sequence classification.The SDR coding scheme also offers large capacity and flexibility for streaming data whose unique symbols are not known upfront.
  • Parameter tuning: HTM aims to handle varied problems without task-specific parameter tuning, using computational principles derived from cortical learning.The discussion contrasts this with machine-learning algorithms that typically require task-specific parameter searches for novel problems.
  • Limitations: HTM may learn very long-term dependencies more slowly because it is a strict one-pass algorithm with access only to the current input.Maintaining a history buffer and running HTM on it multiple times can speed learning of such sequences.
  • Limitations: HTM is robust to spatial noise but sensitive to temporal noise, performs below LSTM on grammar learning, and has only been tested on low-dimensional categorical or scalar streams.On the Reber grammar task after observing 500 sequences, HTM reached 98.4% accuracy versus 100% for LSTM and 86.7% for ELM; hierarchical models are proposed to improve temporal-noise robustness.

7. Appendix

The appendix describes the data encodings, shared HTM implementation settings, comparator-network configurations, and prediction-error metrics used throughout the experiments. It emphasizes sparse distributed representations and complementary evaluation of point accuracy and sequence probability.

  • Data encoding: Artificial categorical symbols were encoded as random sparse distributed representations with 40 active bits in 2,048-bit vectors.Specialized encoders were required to convert real-world data into SDRs.
  • Data encoding: For NYC taxi data, HTM received passenger count, time of day, and day of week through scalar encoders that produced clustered ON bits in binary vectors.LSTM received the same information as input.
  • HTM implementation: The HTM sequence-memory model used one identical parameter set across all experiments, with a complete parameter list provided in the appendix.The implementation source code was made available on GitHub.
  • Comparator implementations: Comparator configurations included online sequential ELM, hand-tuned online ESN, and LSTM networks trained either in batches or online with truncated backpropagation through time.ELM and ESN settings differed by task, while LSTM architectures differed between the artificial and NYC taxi tasks.
  • Evaluation metrics: Prediction accuracy was evaluated with MAPE and negative log-likelihood, combining single-best prediction error with the probability of alternative future sequence elements.MAPE is less sensitive to outliers than root mean squared error, whereas negative log-likelihood evaluates sequence probability distributions.
Loading 1512.05463v2…