PaperlayerView exact revision on arXiv

Source-linked AI summary

Representation of linguistic form and function in recurrent neural networks

Ákos Kádár, Grzegorz Chrupała, Afra Alishahi

arXiv:1602.08952v2cs.CLcs.LG

TL;DR

The paper addresses the limited interpretability of linguistic structure in RNN hidden representations. It introduces omission-score and top-k-context analyses for a multi-task GRU model, finding that visual and textual pathways learn different linguistic sensitivities and contain specialized units for task-relevant patterns.

  • Problem

    RNN hidden representations encode linguistic information in nonlinear distributed vectors whose components are difficult to interpret, and such analysis has been less developed for language models.

  • Method

    The paper analyzes a multi-task IMAGINET model with shared word embeddings and separate GRU pathways for image-vector prediction and next-word prediction using omission scores and top-k contexts.

  • Results

    VISUAL emphasizes information structure and semantically informative lexical and grammatical functions, TEXTUAL is more syntactically sensitive, and both pathways contain specialized units encoding task-useful patterns.

  • Takeaways & Limitations

    RNN hidden activations can be analyzed as encoding task-specific linguistic structure, including context-sensitive word functions and longer dependencies carried across time.

Abstract

from arXiv · show

We present novel methods for analyzing the activation patterns of RNNs from a linguistic point of view and explore the types of linguistic structure they learn. As a case study, we use a multi-task gated recurrent network architecture consisting of two parallel pathways with shared word embeddings trained on predicting the representations of the visual scene corresponding to an input sentence, and predicting the next word in the same sentence. Based on our proposed method to estimate the amount of contribution of individual tokens in the input to the final prediction of the networks we show that the image prediction pathway: a) is sensitive to the information structure of the sentence b) pays selective attention to lexical categories and grammatical functions that carry semantic information c) learns to treat the same input token differently depending on its grammatical functions in the sentence. In contrast the language model is comparatively more sensitive to words with a syntactic function. Furthermore, we propose methods to ex- plore the function of individual hidden units in RNNs and show that the two pathways of the architecture in our case study contain specialized units tuned to patterns informative for the task, some of which can carry activations to later time steps to encode long-term dependencies.

1 Introduction

The paper develops methods for interpreting linguistic structure in RNN hidden activations and applies them to a multi-task architecture with visual and textual pathways. The analyses show task-specific differences in which linguistic information the pathways use and how individual units encode structure.

  • Motivation: RNN representations encode variable-length expressions in fixed-size vectors, but their nonlinear hidden components are difficult to interpret.The paper addresses the relative lack of visualization and linguistic analysis methods for RNNs compared with computer-vision models.
  • Contribution: The paper introduces methods for analyzing hidden activation patterns and the syntactic generalizations learned by RNNs.The focus is on hidden activations rather than word embeddings, with pointers toward extending the analysis to other settings.
  • Case study: IMAGINET contains parallel VISUAL and TEXTUAL GRU pathways with shared word embeddings, enabling comparison under fixed training data and embeddings.VISUAL predicts a visual-scene representation, whereas TEXTUAL models language sequentially.
  • Macro analysis: Omission-score analyses find that VISUAL attends to semantically informative categories, information structure, and grammatical functions, while TEXTUAL is more sensitive to local syntax.The method estimates each input token’s contribution to the networks’ final prediction.
  • Micro analysis: Top-k-context analyses identify hidden units tuned to syntactic, dependency, semantic, lexical, and structural patterns, including units that carry activations across time.These units can encode longer dependencies and task-useful linguistic features.

2 Related work

Prior work developed ways to inspect RNN activations, token salience, attention, and individual units, but this paper emphasizes linguistic structure learning and comparative analysis across RNN pathways.

  • Early RNN analysis: Elman’s early work analyzed hidden activations in recurrent networks trained on synthetic sentence data.These studies established recurrent architectures as objects of activation-level linguistic analysis.
  • RNN interpretability: Li et al. measured token salience with loss derivatives and examined temporal hidden-unit activations in LSTMs trained for sentiment analysis.Their methods also visualized local semantic compositionality.
  • Positioning: The present work differs by focusing more directly on structure learning and comparative analysis between RNNs.This comparison is framed against prior activation-visualization approaches.
  • Attention: Attention mechanisms assign varying weights to decoder hidden states so neural translation models can focus on different source phrases.This related approach analyzes selective focus during decoding.
  • Unit analysis: Karpathy et al. identified interpretable hidden dimensions in character-level language models while emphasizing error analysis and training dynamics.The present paper also studies individual hidden-unit functions from a linguistic perspective.
  • Field context: Computer vision has more extensively studied learned filters and saliencies, whereas computational linguistics has comparatively less work on these issues.The paper situates its contribution within this cross-field difference.

3 Models

The model is a multi-task IMAGINET architecture with shared embeddings and separate GRU pathways for visual prediction and next-word prediction. Its objective jointly trains both outputs.

  • GRU architecture: GRUs retain information across time by combining the previous hidden state with newly computed information through gating.The update gate controls how much new information is mixed into the current state.
  • GRU architecture: The reset gate controls how much of the current input is mixed with the previous state when forming the candidate activation.The candidate activation is a component of the GRU state update.
  • IMAGINET: IMAGINET maps each image-description sentence into two hidden-state sequences produced by TEXTUAL and VISUAL GRUs sharing a word-embedding matrix.The model receives paired image descriptions and corresponding images.
  • IMAGINET: TEXTUAL predicts the next sentence word from its current hidden state, while VISUAL predicts an image vector from its final hidden representation.The pathways therefore support language modeling and visual-scene representation learning.
  • Training objective: The multi-task loss combines cross-entropy for word prediction with cosine distance for image prediction.Both textual and visual targets contribute to training.

4 Experiments

The experiments analyze linguistic regularities learned by IMAGINET’s TEXTUAL and VISUAL pathways at both the final-vector and individual-unit levels.

  • Macro-level analyses: Macro-level experiments analyze the final hidden activation vectors of the recurrent pathways from a linguistic perspective.These analyses examine regularities learned from word-level input.
  • Micro-level analyses: Micro-level experiments investigate linguistic features encoded by individual hidden units.Together, the two experiment types cover pathway-level and unit-level representations.

5 Analysis of hidden activation vectors

The section introduces omission scores to measure how much individual tokens contribute to a sentence representation and compares linguistic sensitivity across VISUAL and TEXTUAL pathways. The analyses show that VISUAL emphasizes semantically informative categories, grammatical functions, and information structure, while TEXTUAL is relatively more uniform and syntactically oriented.

  • 5.1 Computing Omission Scores: Omission scores estimate a token’s contribution by measuring how much removing it changes the final sentence representation.The method compares the end-of-sentence representation of the original sentence with that of the sentence containing the omitted token.
  • 5.1 Computing Omission Scores: In example captions, VISUAL focuses on scene-relevant words such as pizza, baby, and bed, whereas TEXTUAL distributes attention more evenly across time steps.Removing the highest-scoring word changes the image retrieved from the sentence, illustrating the visual pathway’s selective token sensitivity.
  • 5.2 Omission score distributions: VISUAL assigns high omission scores mainly to nouns, followed by adjectives and verbs, while TEXTUAL shows smaller differences across word types.VISUAL is relatively more sensitive to nouns, numerals, and adjectives; TEXTUAL is relatively more sensitive to prepositions, some verbs, determiners, and particles.
  • 5.2 Omission score distributions: VISUAL emphasizes grammatical functions filled by semantic content, whereas TEXTUAL attends relatively more to purely grammatical functions.The reported dependency-label contrasts include VISUAL sensitivity to ROOT, DOBJ, NN, AMOD, and NUM, and TEXTUAL sensitivity to DET, MARK, AUX, PREP, POSS, CC, and COP.
  • 5.3 Beyond Lexical Cues: VISUAL omission scores depend on a word’s grammatical function and sentence position, not only on its lexical identity.Adding dependency labels, positions, and interactions explains more variance for VISUAL than lexical features alone; selected words also receive different scores across dependency roles.
  • 5.3.2 Sensitivity to information structure: VISUAL learns information-structure sensitivity by assigning greater importance to sentence-initial topic words, with some influence from recency.In image captions, topics often identify depicted objects, and the position coefficients support greater attention to sentence-initial words and relatively high attention to the final position.

6 Analysis of hidden units

The analysis identifies specialized hidden units and compares how VISUAL and TEXTUAL encode linguistic information. VISUAL units reflect topical and semantic patterns, while TEXTUAL units are more associated with syntactic constructions.

  • 6.1 Top K contexts: The top-k-contexts method ranks corpus contexts by hidden-unit activation to reveal linguistic patterns encoded by individual dimensions.It stores hidden activations for each token and sorts contexts by activation magnitude.
  • 6.2 Specialized hidden units: VISUAL and TEXTUAL units become sensitive to distinct topical, syntactic, and semantic templates.VISUAL examples include home-electronics contexts, whereas TEXTUAL examples include vehicles followed by transportation verbs and other combined syntactic-semantic patterns.
  • 6.3 Units predictive of a grammatical function: Hidden units can predict dependency relations beyond lexical n-gram cues, indicating that some dimensions encode more general grammatical information.The regression models identify units whose coefficients remain informative after including n-gram predictors.
  • 6.4 Units carrying over information: Some VISUAL units carry activation across time steps, encoding information from preceding tokens needed for later dependency predictions.Examples include units predicting CONJ and POBJ contexts where the relevant information comes from earlier tokens.
  • 6.5 Comparison of models based on top contexts: VISUAL emphasizes contentful and sentence-initial topic words, whereas TEXTUAL generally gives greater salience to words near the sentence end and grammatical-function words.These differences reflect the models’ distinct task objectives and are consistent with the reported comparison between information-structure and syntactic sensitivity.
  • 6.5 Comparison of models based on top contexts: The relative difference between TEXTUAL and VISUAL median mutual-information scores is much larger for dependency-label contexts than for word contexts.The bootstrap distributions for dependency-label conditions do not overlap.

7 Conclusion

The paper introduces linguistic analyses of RNN hidden activations and applies them to the two IMAGINET pathways. The analyses show task-specific attention patterns and specialized units that can encode structural information across time.

  • 7 Conclusion: The paper proposes methods to measure token importance and investigate linguistic features encoded beyond lexical cues.Its case study uses the IMAGINET VISUAL and TEXTUAL pathways.
  • 7 Conclusion: VISUAL learns an abstract representation of information structure and selectively attends to lexical categories and grammatical functions carrying semantic information.TEXTUAL is comparatively sensitive to features of a more syntactic nature.
  • 7 Conclusion: Both pathways contain specialized units tuned to lexical and structural patterns useful for their tasks.Some units carry activations to later time steps to encode long-term dependencies.
  • 7 Conclusion: The proposed techniques are intended for application to recurrent models trained with other objectives, including neural machine translation and sentence embedding systems.The paper presents this as future work rather than an evaluation in the current study.
Loading 1602.08952v2…