Source-linked AI summary
Disentangling Computation in Multi-Task Neural Networks with the Green's Operator
James Hazelden
TL;DR
The paper asks how computation is organized and reused across tasks and time, a question not directly answered by activity geometry or local perturbation-growth analyses. It studies first-order source-to-destination responses with the finite-horizon Green’s operator and its task- and time-level reductions. In a flexible multitask RNN, those reductions reveal task response structure and distinct temporal routing patterns across computations.
Problem
The paper asks how computation is organized and reused across tasks and time, since activity geometry and Lyapunov analyses do not directly represent where perturbation influence travels or whether tasks reuse that response.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The source-to-destination response view complements fixed points, activity geometry, and Lyapunov analysis as a matrix-free map of task reuse and temporal routing.
Source: §6 · pp. 4–5The analysis is first-order and finite-horizon, and its reductions discard information.
Source: §6 · pp. 4–5
Abstract
from arXiv · showhide
How is computation organized and reused across tasks and time in a trained recurrent network? Most analyses emphasize the geometry of neural activity, dynamical motifs, or local perturbation growth. We instead study the network's global first-order perturbation response. The finite-horizon Green's operator maps perturbations at each source along a trajectory to their downstream state-space responses and therefore directly represents perturbation routing. Simple reductions of this operator provide task-to-task and time-to-time views of the same computation, while matrix-free products make these views accessible without constructing the full operator. In a flexible multitask recurrent network, task reductions reveal structured reuse of known computational motifs, while temporal reductions reveal causal pathways and how they emerge during training. Our main point is simple: the Green's operator provides a global response geometry for mapping the organization of learned dynamical computation.
1. Introduction
The paper asks how a recurrent network organizes and reuses computation across tasks and time, and how to study that organization globally. It uses the finite-horizon Green’s operator to represent source-to-destination perturbation responses, which activity and local-spectrum analyses do not directly describe.
1. Introduction
- The central question is how computation is organized and reused across tasks and time without cataloguing every trajectory, fixed point, task condition, interval, and perturbation direction.Recurrent networks can solve many tasks using the same state space and parameters, and can reuse dynamical motifs across related computations.Source: §1 · p. 1
- Activity analyses describe trajectory and representation geometry, while Lyapunov analyses describe perturbation growth or decay; neither directly answers where an injected perturbation’s influence appears later or whether another task reuses it.Source: §1 · p. 1
- The finite-horizon Green’s operator P maps perturbations applied along a trajectory to resulting first-order state perturbations, with blocks encoding causal responses between source and destination points.Source: §1 · p. 1
- The paper isolates how routing differs from activity and local spectral stability, then uses task- and time-level reductions to map computation in a flexible multitask RNN.Source: §1 · p. 2
2. The Green’s operator: a global response map
The Green’s operator represents first-order perturbation responses from source times to destination times. Task- and time-level reductions expose different views of that response, and matrix-free products support those summaries without constructing the full operator.
2. The Green’s operator: a global response map
- The linearized recurrent system is driven by injected perturbations, and stacking perturbations and downstream state responses over a finite horizon yields the global response representation.Source: §2 · p. 2
- For t > s, P’s block maps a perturbation at source time s to its first-order effect at destination time t, with the identity on the diagonal under the stated convention.Source: §2 · p. 2
- A forward recurrence computes Pu and a reverse recurrence computes P∗v, supporting randomized low-rank and reduced summaries without forming the full task/trial/time/state response tensor.Task-level reductions remove nuisance axes while retaining response geometry for task comparison; time-level reductions retain source and target time.Source: §2 · p. 2
- Figure 2 presents task-level reductions through known motif organization, pairwise Green-response similarity, inferred relations, and a low-dimensional embedding; Green-response similarity and hidden-state covariance are related but non-equivalent.The figure also distinguishes task pairs from the same versus different families.Source: Figure 2 · p. 3, §2 · p. 3
3. A minimal example: same activity, same spectrum, different routing
Matching unperturbed trajectories and eigenvalues does not determine source-to-destination perturbation routing. The toy example isolates how a single route can change downstream response.
3. A minimal example: same activity, same spectrum, different routing
- Two stable triangular linear RNNs have the same trajectory from the chosen initial condition and the same eigenvalues, but an off-diagonal y → x route makes a perturbation to y persistently affect x in one system and not the other.The example isolates routing as distinct from the information supplied by activity trajectories and local spectra.Source: §3 · p. 3
4. Task organization in a flexible multitask RNN
In a 256-unit leaky RNN trained on 15 tasks, pairwise Green-response similarity reveals block structure aligned with known sub-computations. Response routing and hidden-state covariance are related but non-equivalent.
4. Task organization in a flexible multitask RNN
- The analysis uses a 256-unit leaky RNN jointly trained on the 15-task family, evaluated at step 680,000.Prior work identifies motif reuse in this task family.Source: §4 · p. 3
- Pairwise task comparisons use normalized Frobenius similarity between reduced responses.The analysis asks whether known motif reuse is visible in perturbation-response geometry.Source: §4 · p. 3
- The task-conditioned response-similarity matrix shows block structure aligned with known sub-computations.Hidden-state covariance and response routing are related but non-equivalent; the motif organization serves as an external reference, not a claim of universal superiority.Source: §4 · pp. 3–4
5. Temporal organization emerges during training
Temporal reductions retain source and target time to map when perturbation influence is routed. Training leaves ReactPro comparatively local while memory tasks develop longer-lasting pathways.
5. Temporal organization emerges during training
- After training, MemoryPro and DMS develop delay-spanning structure, while ReactPro remains comparatively local and ContMemDecision2 develops a more restricted persistent pathway.Before training, responses lie near the causal diagonal.Source: §5 · p. 4
- MemoryPro gains substantially more long-lag response than ReactPro, while both begin near the diagonal at initialization.Training organizes temporal routing differently for computations with different persistence demands.Source: §5 · p. 4
- An exact basis-invariant Frobenius reduction gives the same qualitative conclusion as Figure 3’s signed uniform-direction projection.The figure’s projection is not a mathematical neuron trace.Source: §5 · p. 4
6. Discussion
The discussion frames task and time reductions as complementary views of response routing, with a deliberately narrow claim. The appendix connects the operator to path sums and parameter-to-state learning operators.
6. Discussion
- Task-to-task and time-to-time reductions ask what computation is reused and when influence is routed, respectively.The analysis is first-order, finite-horizon, and each reduction discards information; the claim is that global response geometry provides a useful matrix-free map.Source: §6 · pp. 4–5
Appendix A. General operator formulation
- When DhF is invertible, the formulation defines the finite-network Green’s operator; h collects model state, x inputs, and θ parameters.Source: Appendix A · p. 6
- For an explicit acyclic computation graph, the Green’s operator expands as I + L + L^2 + ··· + L^K, with each term collecting paths of a given length.For recurrent trajectories unrolled over a finite horizon, this expansion is equivalent to the block products in Eq. (3).Source: Appendix A · pp. 6–7
- Differentiating the constraint with respect to parameters connects parameter-to-state learning operators to parameter-selected sketches of global response geometry.The passage notes related empirical operator factorizations in Hazelden et al. (2026).Source: Appendix A · p. 7
Appendix B. Matrix-free products and randomized summaries
Matrix-free forward and reverse recurrences compute products with the Green’s operator and its adjoint. These products support randomized summaries without storing the full block operator.
Appendix B. Matrix-free products and randomized summaries
- A forward recurrence computes y = Pu by propagating the input perturbation through the recurrent trajectory.It initializes y0 = u0 and updates yt+1 = Jt yt + ut+1.Source: Appendix B · p. 7
- A reverse recurrence computes the adjoint product z = P∗v.Source: Appendix B · p. 7
- The product cost is linear in horizon times Jacobian-vector or vector-Jacobian product cost, without storing the full T^2N^2 block operator.The products support randomized range finding and SVD, Gram products, and stochastic reduced traces or block-energy estimators.Source: Appendix B · p. 7
Appendix C. Toy derivation
The toy systems show that matching trajectories and local eigenvalue spectra do not determine perturbation routing. The off-diagonal term gives the exact lagged response from a perturbation in y to a response in x.
Appendix C. Toy derivation
- The off-diagonal term is the exact lag-k response from a perturbation in y to a response in x.For a ≠ b, it is equivalently c(a^k−b^k)/(a−b).Source: Appendix C · p. 7
- The two systems differ in source-to-destination routing at every lag despite sharing the selected unperturbed trajectory and local eigenvalue spectrum.Their Green matrix is block lower triangular, with blocks A^(t−s) for t ≥ s.Source: Appendix C · p. 7
Appendix D. Experimental details and reduction controls
The appendix reports the trained network and matrix-free analysis setup, then checks task comparisons and temporal conclusions against sampling and reduction controls. The controls support structured task and time organization while qualifying the response representation's comparative scope.
Appendix D. Experimental details and reduction controls
- The analysis uses a trained 15-task leaky RNN with 256 hidden units at converged training step 680,000.The Green operator is evaluated on tensors indexed by trial, time, and hidden unit using forward and adjoint products rather than explicit construction.Source: Appendix D · pp. 7–8
- 0.932 ROC-AUC and 0.795 average precision summarize the 16-trial exact Green comparison across 1,106 cross-task condition pairs.In the fair eight-trial comparison, Green similarity gives AUC 0.920 and AP 0.765; hidden covariance gives 0.852/0.591, Laura-style task variance 0.941/0.812, and local gain spectrum 0.785/0.573 (AUC/AP). The epoch-preserving semantic-label null AUC is 0.812 ± 0.017 versus observed 0.920 (p = 2 × 10^-4); Green response is a mechanistic complement, not a universally superior representation.Source: Appendix D · p. 8
- With two disjoint, coverage-matched sets of 16 trials per condition, pairwise Green maps correlate at 0.959 and mean matched-condition cosine is 0.953.The two recovery AUCs are 0.936 and 0.920; eight trials per half are less stable, so detailed pair claims use the larger stratified sample where available.Source: Appendix D · p. 8
- The historical Figure 3 temporal visualization is a uniform-direction response projection, not a mathematical partial trace; the temporal conclusion is checked with basis-invariant Frobenius response RMS.Source: Appendix D · p. 8
- At step 680,000, MemoryPro has long-range Frobenius-energy fraction 0.353 and near-diagonal fraction 0.167, versus ReactPro's 0.017 and 0.660.At initialization, both long-range fractions are near 5 × 10^-4 and both near-diagonal fractions are near 0.83; across all 15 tasks, persistent computations carry substantially more long-range response after training.Source: Appendix D · p. 8
Appendix E. Additional scope and robustness notes
The response representation is presented as a descriptive complement to activity- and stability-based descriptions, with limits on its generality and predictive interpretation.
Appendix E. Additional scope and robustness notes
- Task-level Green organization is model-dependent, and the response representation is not a universal replacement for activity or stability-based descriptions.Source: Appendix E · p. 8
- Task-level Green similarity does not uniquely predict transfer or interference after controlling for task variance.Source: Appendix E · p. 9