Getting aligned on representational alignment

Representational alignment

• Representational alignment is the extent to which two or more systems’ internal representations agree, shaped by differences in stimuli, experience, tasks, goals, and architecture.The concept appears under several names across fields and is also implicit in areas such as knowledge distillation, disentanglement, and concept-based models.Source: §1 · p. 4

Motivation

• Researchers lack a shared standardized language for alignment, limiting knowledge transfer across fields, duplicating efforts, and slowing progress; RSA does not capture the full range of work.The paper argues that RSA is not applied alike across disciplines.Source: §1 · p. 4

Framework and cross-disciplinary contribution

• The framework organizes a broad cross-disciplinary literature review around five recurring components and three objectives, providing a common language for synthesizing insights across fields.The paper also maps existing studies through the framework and identifies open problems and challenges across disciplines.Source: §1 · pp. 4–5

Objective 1 – Measuring

• Measuring alignment compares representational structures at an abstract information-processing level, helping validate one system as a model of another or identify differences between systems.The systems are typically static, and measurements use paired data from the same stimuli.Source: §1 · p. 4

Objective 2 – Bridging

• Bridging establishes a correspondence between systems’ representations to enable direct comparison and pooling, typically by projecting at least one system into a new shared space.The systems are typically static, and the data may be paired or unpaired.Source: §1 · pp. 4–5

Objective 3 – Increasing

• Increasing alignment updates at least one system’s representations to make its processing more like another’s, either as an end in itself or to improve downstream performance.Typically, at least one system is dynamic, while the data may be paired or unpaired.Source: §1 · p. 5

2 Background and review

• Researchers across cognitive science, neuroscience, and machine learning study representational alignment from differing perspectives but often converge on similar techniques, motivating a unifying framework and review of future-work gaps.The review refers to Table 2 in motivating its framework.Source: §2 · p. 5

2.1 Cognitive Science

• Cognitive science asks whether people share representations across cultures and development, typically inferring internal representations and algorithms indirectly from behavior.These approaches measure alignment between people or characterize representational change over time.Source: §2.1 · p. 5

2.1.1 Similarity judgments and multidimensional scaling

• MDS embeds similarity judgments in spaces, while relational methods compare structures; concepts can be aligned without identifying matches in advance [Shepard, 1962, 1980; Goldstone and Rogosky, 2002].MDS can also capture individual differences and developmental change, while natural environments support alignment of everyday concepts and children’s early concepts appear to exploit these regularities [Roads and Love, 2020].Source: §2.1.1 · p. 6

2.1.2 Human-machine alignment

• Human similarity judgments correlate with final-layer CNN activations, but better-performing models can align worse, and student-transfer performance varies U-shapedly with alignment [Peterson et al., 2018; Roads and Love, 2021; Sucholutsky and Griffiths, 2023].Most tested vision models showed low human alignment out of the box, but a learned linear transformation could reduce the gap [Muttenthaler et al., 2023a]; greater alignment does not necessarily improve performance or behavior.Source: §2.1.2 · p. 7

2.1.3 Semantic representations

• Semantic alignment tracks representation change in learning and neurodegeneration, while textual embeddings can proxy cross-modal human similarity judgments on par with domain-specific neural networks [Rogers and McClelland, 2004; Ralph et al., 2017; Saxe et al., 2019; Marjieh et al., 2023a].This work also uses alignment to investigate the neuro-anatomical basis of semantic processes and suggests ways to bridge human and computational representational spaces.Source: §2.1.3 · p. 7

2.1.4 Alignment across individual participants’ behavior

• Across individuals, differences in word meanings predict communication failures, so conflict resolution must account for both opinion divergence and representational alignment [Oktar et al., 2023].Representational differences have been documented for semantic categories, social groups, and complex concepts [Hoffman, 2018; Marti et al., 2023; Xie et al., 2021; Brandt, 2022].Source: §2.1.4 · p. 7

2.1.5 Alignment across cultures

• Across 39 groups in 15 countries, rhythm categories showed universal small-integer-ratio bias but culture-dependent emergence [Jacoby et al., 2021a]; structured semantic domains aligned more across languages [Thompson et al., 2020].Thompson et al. analyzed 1,010 meanings in 41 languages; number and kinship aligned more than natural kinds and common actions, whose meanings were more culturally contingent.Source: §2.1.5 · pp. 7–8

2.2.1 Alignment across heterogeneous measurements

• RSA compares representational dissimilarity matrices to align neural activity across heterogeneous measurements, including human and macaque inferotemporal cortex and signals collected across modalities [Kriegeskorte et al., 2008b; Cichy et al., 2014].RDMs capture pairwise stimulus differences while abstracting away from the space in which representations are defined; cross-modal RSA can capture spatiotemporal patterns unavailable from either modality alone.Source: §2.2.1 · p. 8

2.2.2 Alignment across individuals

• Hyperalignment and shared response modeling map individual neural responses into common spaces, enabling group-level comparisons and zero-shot decoding [Haxby et al., 2011, 2020; Chen et al., 2015b].Hyperalignment uses Procrustes transforms, while shared response modeling separates individual-specific and shared components; common spaces have supported object classification, eye-movement prediction, and image reconstruction.Source: §2.2.2 · p. 8

2.2.3 Alignment between brain activity and model systems

• Alignment links brain regions to computational models, from semantic representations and visual hierarchies to word-by-word alignment with language models [Pobric et al., 2010; Yamins et al., 2014; Wehbe et al., 2014b].Research over the past decade has reported high brain alignment for newer AI models in vision and language.Source: §2.2.3 · p. 9

2.2.4 Alignment for hypothesis testing

• Alignment tests hypotheses about brain information processing, but whether next-word prediction is necessary for language-model alignment remains unclear, with syntax and lexical semantics also implicated [Merlin and Toneva, 2022; Oota et al., 2023].Candidate-hypothesis comparisons have been used to investigate task-dependent visual and language processing and the processing of stimulus properties such as object manipulability and size.Source: §2.2.4 · p. 9

2.2.5 Alignment for stimulus selection or design

• Alignment can select or design stimuli: controversial examples distinguish competing mechanistic accounts, while unusual stimuli can drive or suppress activity in the human language network [Groen et al., 2018; Golan et al., 2022; Tuckute et al., 2023].These approaches use alignment to optimize stimulus space and are only beginning to be explored in neuroscience and cognitive science.Source: §2.2.5 · p. 9

2.2.6 Alignment as communication

• Speaker-listener neural alignment emerges during successful communication, and alignment also accompanies joint play, nonverbal social cues, and improved teacher-student learning outcomes [Stephens et al., 2010; Piazza et al., 2020; Koul et al., 2023; Meshulam et al., 2021].During narration, alignment spans temporal, parietal, auditory, and prefrontal cortices and appears only when the listener understands the speaker’s utterance.Source: §2.2.6 · p. 9

2.3.1 Model-to-model alignment

• Model-to-model alignment can reveal effects of objectives, tasks, or initialization, but reducing alignment is also useful for ensemble diversity when models otherwise converge to similar minima.Representational similarity analysis is one measurement approach; linear mappings and representational similarity itself can also bridge model spaces.Source: §2.3.1 · p. 10
• Multimodal systems jointly optimize modality-specific representations, while cross-model alignment can use linear regression or representational similarity to bridge spaces.Examples include CLIP’s contrastive alignment of visual and textual embeddings and systems that translate between modalities or create shared spaces.Source: §2.3.1 · p. 10
• Knowledge distillation aligns a smaller student to a teacher’s outputs, representational geometry, or pairwise similarities, transferring prior knowledge and serving as compression or regularization.Soft labels can produce tighter class clusters and improve data efficiency relative to hard labels because they convey information about class relationships.Source: §2.3.1 · pp. 10–11

2.3.2 Learning human-like representational geometries

• Aligning neural representations with human similarity judgments or visual strategies has improved few-shot learning, anomaly detection, retrieval, local structure, and object categorization.Learning a linear transformation or fine-tuning can be cheaper than optimizing an information-processing function from scratch while helping explain factors behind alignment.Source: §2.3.2 · p. 11

2.3.3 Interpretability and explainability

• Interpretability research bridges network representations with human-understandable concepts through neuron analysis, concept-based methods, learned human-aligned concepts, linear vectors, and circuits.Mechanistic interpretability seeks procedures describing how a network performs a task, rather than only identifying a concept it represents.Source: §2.3.3 · p. 11

2.3.4 Behavioral alignment

• Behavioral alignment compares system outputs, but identical behavior can arise from different representations and mappings, whereas behavioral differences imply a difference in one or both.Fine-grained error patterns can reveal representational differences; RLHF uses human ratings to train a scorer for preferred outputs, though suitable feedback remains an open question.Source: §2.3.4 · pp. 11–12

2.3.5 Value alignment

• Behavior-based value alignment is hard to define and measure, and output evaluations may miss misalignment beyond constrained settings; representation engineering instead manipulates dimensions linked to valued behaviors like honesty.Representational alignment may offer benefits for value alignment and downstream tasks, at least as pre-conditioning for targeted interventions.Source: §2.3.5 · p. 12

2.3.6 Human-robot interaction

• Robots can learn task representations through explicit structures or implicit input-to-behavior mappings, but each approach risks misrepresenting user intent or retaining irrelevant information and spurious correlations.Because collected data may not represent the end user, robotics methods may benefit from treating people as weak supervision signals in alignment rather than static data sources.Source: §2.3.6 · pp. 12–13

• Research on representational alignment remains fragmented across the three fields, with disparate definitions and insufficient knowledge sharing.Source: §3 · p. 13
• This fragmentation has led to rediscovery of ideas under different names, repeated mistakes, and underused opportunities for cross-disciplinary collaboration.Source: §3 · p. 13
• The proposed general formalism aims to unify these communities and accelerate progress on open problems in representational alignment research.Source: §3 · p. 13

• The framework identifies five components: alignment data, systems, measurements, extracted or reconstructed embeddings, and an alignment function comparing embeddings.The data may be sensory or higher-level cognitive content; measurements include behavioral judgments and neural or network activity.Source: §3.1 · p. 14
• The framework assumes static data samples, but extends to dynamic interaction by treating environment states as data.Systems interface with data through experimenter- and system-dependent processes; task objectives can affect representations and outputs may change the data distribution.Source: §3.1 · p. 14
• Alignment studies compute a score, while bridging or increasing alignment uses that score to update embedding functions, internal representations, or their measurements.Source: §3.1 · p. 14
• The framework aims to provide a general language for communicating methods and results, mapping diverse literature examples, and formalizing study components and decisions.The paper also uses the framework to describe nine highlighted examples from Figure 1.Source: §3.1 · p. 14

3.2 Formalizing representation spaces

• A dataset comprises n trials whose stimuli may be single items, sets, sequences, or videos, and may be dynamic environment states for interacting agents.Most case studies concern the simplified case in which systems do not modify the environment.Source: §3.2 · pp. 14–15
• The framework represents two systems as functions that map stimuli to internal states, treating task context as part of the stimuli and the environment as stationary.Systems may actively engage with stimuli and modify the environment; most case studies instead use a simplified setting in which systems do not modify it.Source: §3.2 · p. 15
• Measurements summarize each system’s states across all trials, either through parameterized measurement functions or, in some cases, by directly stacking the internal states.Measurement parameters can reflect choices such as the machine-learning layer measured or human fMRI scanning and processing procedures.Source: §3.2 · p. 15
• Optional linear or nonlinear embedding functions can map measurements into possibly lower-dimensional spaces so similarity can be quantified; identity maps suffice when embeddings are unnecessary.Embeddings can support denoising or increase representational alignment in some situations.Source: §3.2 · p. 15
• Representations are flattened into vectors for simplicity, but measurements may have other shapes or types if an appropriate alignment measure exists.Examples include matrices, graphs, programs, and strings.Source: §3.2 · p. 15

3.3 Measuring alignment

• An alignment function maps two representations to a scalar dissimilarity, with zero denoting equivalence; it need not be a metric and may be descriptive or differentiable, symmetric or directional.The framework also distinguishes similarity- from dissimilarity-quantifying functions.Source: §3.3 · pp. 15–16

3.3.1 Similarity or dissimilarity quantifying

• Bounded similarity scores such as correlations and cosine similarity offer interpretable reference points, whereas nonnegative dissimilarities such as norms or entropy are harder to interpret but can be minimized.A dissimilarity can also serve as a loss to maximize a similarity score in contrastive learning [Chen et al., 2020; Radford et al., 2021; Muttenthaler et al., 2023b].Source: §3.3.1 · p. 16

3.3.2 Descriptive or differentiable

• Descriptive measures characterize observed alignment, while differentiable measures can be optimized to increase it; rank-correlating RDMs is non-differentiable but invariant to monotonic nonlinear transforms, unlike a differentiable Pearson alternative.Rank correlation is used in Representational Similarity Analysis, with Kendall’s τa or ρa.Source: §3.3.2 · p. 16
• Differentiable alignment can use a learned transformation with model parameters frozen or fine-tune the model itself; fine-tuning requires access to updateable parameters and held-out generalization, supported by few-shot data or regularization.Transformations range from identity through linear to constrained nonlinear mappings.Source: §3.3.2 · p. 17

3.3.3 Symmetric or directional

• Symmetric functions preserve the same score when input order is reversed and suit descriptive comparisons; directional functions measure one representation relative to the other and can serve as optimization losses.Cosine similarity and Pearson correlation are symmetric examples, while cross-entropy and KL divergence are directional and generally difficult to interpret descriptively.Source: §3.3.3 · p. 18

3.3.4 Different measures afford different inferences

• Different measures can yield different conclusions: symmetric measures may hide information or noise differences, while asymmetric or fitted measures can reveal distinctions but also introduce failures or excessive flexibility.The authors recommend comparing multiple measures and checking whether conclusions generalize; examples include regression and RSA producing contrasting judgments for systems encoding the same information with different activity proportions.Source: §3.3.4 · p. 18

3.3.5 What does it take to unambiguously specify a similarity measure?

• Similarity scores are difficult to compare across studies unless implementation conventions are specified, since alternative HSIC estimators and CKA variants can substantially change the score [Murphy et al., 2024; Cloos et al., 2024b].Cloos et al. [2024a] developed a Python package to benchmark and standardize similarity-measure implementations.Source: §3.3.5 · pp. 18–19

4 Representational alignment in diverse communities

• The framework provides a common language for highlighting similarities in the goals and approaches of diverse fields studying intelligent-system alignment.The section presents this as a way to help researchers connect topics and transfer practices across communities.Source: §4 · p. 19
• Selected research projects are presented with conceptual summaries followed by formal mathematical descriptions structured using the framework.The examples are visualized in Figure 1.Source: §4 · p. 19
• Table 2 concisely maps additional related literature onto the unifying framework.Source: §4 · p. 19

4.1.1 Measuring representational alignment (Figure 1a)

• Jacoby and McDermott [2017] used serial reproduction to reveal rhythm categories near integer ratios; across 39 groups in 15 countries, prototypes persisted but category weights varied.Under experimentally verifiable conditions, the paradigm converges to a sample from the perceptual prior and uses minimal verbal instructions; American and Canadian children differed little from adults.Source: §4.1.1 · pp. 20–21

4.1.2 Bridging representational spaces (Figure 1d)

• Hebart et al. [2020] used 1.46 million human judgments to identify 49 interpretable dimensions across 1854 object categories that were highly predictive of single-trial choices.Identifying core dimensions provides a route to directly compare candidate dimensions underlying alignment across individuals or modalities.Source: §4.1.2 · p. 21

4.1.3 Increasing representational alignment (Figure 1g)

• Muttenthaler et al. [2023b] transformed neural-network representations toward human odd-one-out behavior, producing representations more aligned with human perception and improving downstream computer-vision performance.An additional contrastive objective preserved local similarity structure from the original neural-network space.Source: §4.1.3 · pp. 22–23

4.2.1 Measuring representational alignment (Figure 1b)

• Kriegeskorte et al. [2008b] used RSA to compare monkey and human IT representational geometry, encoding image dissimilarities from neural responses and correlating the resulting vectors with Spearman rank correlation.Monkey responses were measured with multi-array electrophysiology and human responses with fMRI.Source: §4.2.1 · p. 23

4.2.2 Bridging representational spaces (Figure 1e)

• O’Connell and Chun [2018] mapped individuals’ fMRI into a CNN-defined common space, then compared brain-derived spatial-priority maps with eye fixations to identify regions predictive of gaze.They used linear decoding for the shared space and a computational salience model to derive the spatial-priority map.Source: §4.2.2 · pp. 23–24

4.2.3 Increasing representational alignment (Figure 1h)

• Khosla and Wehbe [2022] optimized CNNs to predict human fMRI responses in visual regions; models trained on high-level visual regions recapitulated classification and human-aligned similarity judgments.The work aimed to increase alignment by directly optimizing CNNs to predict fMRI responses.Source: §4.2.3 · p. 24

4.3.1 Measuring representational alignment (Figure 1c)

• CKA is a simple, widely used measure of neural-network representational similarity, with kernel matrices enabling comparisons when embeddings are expensive or infinite-dimensional.Other neural-network similarity measures have also been proposed.Source: §4.3.1 · p. 25

4.3.2 Bridging representational spaces (Figure 1f)

• Aligning image and text representations improved cross-task transfer from visual recognition to VQA over standard multitask learning, with added recognition benefits for under-labeled categories common in queries.The alignment objectives distinguish object words from attribute words and use different losses for each.Source: §4.3.2 · pp. 25–26

4.3.3 Increasing representational alignment (Figure 1i)

• Representation alignment extends distillation beyond behavioral matching, using a learned teacher–student alignment function whose critic objective lower-bounds mutual information between representations.The critic is trained alongside the student using matching and non-matching input pairs; the relative frequency of non-matching pairs is controlled by a hyperparameter.Source: §4.3.3 · pp. 26–27

5.1 Selecting data and stimuli

• Alignment estimates depend strongly on dataset choice: restricted or confounded stimuli can inflate similarity or bias interpretations, so diverse naturalistic and controlled data plus held-out categories are needed.Naturalistic stimuli can confound features, while overly simple controlled stimuli can miss complex processing; the authors recommend testing generality across held-out stimulus categories.Source: §5.1 · p. 27

5.2 Defining, probing, and characterizing representations

• Presentation timing and order, measurement choices, and which components or brain regions are sampled can substantially alter representations, while exhaustive pairwise analysis is often infeasible.Presentation formats should align between systems where possible and randomize factors that cannot be aligned; indirect measures such as fMRI or EEG can distort or enhance features.Source: §5.2 · p. 28

5.2.1 Eliciting representations from black-box systems

• Under empirically validated conditions, MCMC elicitation converges to participants’ hidden representational prior, while serial reproduction parallels diffusion processes [Marjieh et al., 2023b].Serial reproduction uses Gibbs sampling to elicit representations through recall or interactive object modification, and has been used to derive prototypes across domains.Source: §5.2.1 · pp. 28–29

5.2.2 The relationship between representation and computation

• Implementation biases and dynamic context complicate computational interpretations of representations; a pragmatic approach is supported by behavioral utility, but analyzing temporal dynamics may yield deeper insight.Ostrow et al. [2024] propose Dynamical Similarity Analysis and find it more accurately identifies similarities among recurrent networks on various tasks.Source: §5.2.2 · p. 29

5.3 Measuring alignment

• Different metrics can yield different conclusions, so comparisons should use multiple measures; distance-based metrics may misweight distinctions when representational variance differs from computational importance.The concern includes variance differences linked to inductive biases or learning dynamics [Lampinen et al., 2024], which may make measured distance a poor proxy for computational importance.Source: §5.3 · pp. 29–30

5.4 Will representational alignment help improve the alignment of behavior?

• Aligned outputs can occur without aligned representations, and representational similarity does not guarantee output alignment; nevertheless, their coupling makes representational alignment a useful diagnostic and complement to direct behavior-alignment strategies.Initial representations constrain what a system learns to output, while learned outputs shape representations; this connection may matter when designing human-centric AI thought partners [Collins et al., 2024b].Source: §5.4 · p. 30

5.5 Possible risks of representational alignment

• Greater representational alignment may make model outputs harder to distinguish from human outputs, while choosing biological targets can embed implicit choices with downstream bias risks, motivating safeguards.The authors call for further work to characterize these risks and develop frameworks to guard against negative ramifications.Source: §5.5 · p. 30

6 Conclusion

• Across cognitive science, neuroscience, and machine learning, researchers measure alignment between systems, bring representations into a shared space, and seek to increase their alignment.These efforts lack a clear common language, leaving researchers often unaware of related ideas, methods, and empirical results.Source: §6 · p. 30
• The Perspective builds bridges across fields by aligning terminology and methods and highlighting their histories and recent developments.It aims to increase sharing of related ideas and methods and awareness of common challenges and open questions.Source: §6 · p. 30
• The authors hope these varied perspectives will inspire researchers to apply representational-alignment ideas and tools to understand or build more intelligent systems.Source: §6 · p. 30