Iterated learning for emergent systematicity in VQA

1 INTRODUCTION

• Ground-truth programs can support systematic generalization in NMNs, but creating them is costly and jointly learned programs often fail to generalize systematically.The cited work reports worse performance with learned programs than with human-designed programs.Source: §1 · p. 1
• The paper treats NMN programs as samples from an emergent “layout language” and applies iterated learning to encourage compositional structure in VQA.Iterated learning models language change through generations acquiring a previous generation’s language from limited samples.Source: §1 · pp. 1–2
• Using 100 ground-truth programs, the method achieves CLEVR performance comparable to methods using 18,000 and 1,000 programs, respectively.The experiments also report improved generalization to unseen questions from training templates and to unseen question templates.Source: §1 · p. 2
• The authors introduce SHAPES-SyGeT, a benchmark for systematic generalization based on the SHAPES dataset.Their experiments span SHAPES-SyGeT, CLEVR, and CLOSURE.Source: §1 · p. 2

2 RELATED WORK

• On SQOOP, NMNs robustly solved test problems only with a fixed tree-structured layout; robust learned layouts emerged only with a strong prior.That work concluded explicit regularization and stronger priors were needed to develop the right layout structure.Source: §2 · pp. 2–3
• CLOSURE tests novel combinations of CLEVR linguistic concepts, and many existing models fail to generalize systematically to it.A substantial gap was reported between performance with ground-truth and learned layouts.Source: §2 · p. 3
• Research on emergent communication has used cooperative referential games, including multi-stage variants, to study learned communication protocols.Methods include reinforcement learning, straight-through estimation, and backpropagation for continuous languages.Source: §2 · p. 3
• Prior studies report that environmental pressure can foster compositionality, but generalization pressure alone does not guarantee it.The community still lacks strong evidence about necessary or sufficient conditions for compositional language emergence.Source: §2 · p. 3
• Iterated learning has been linked to a learning bottleneck in which limited supervised data can favor structured languages.Experiments with human participants demonstrated artificial languages developing structure under iterated learning.Source: §2 · p. 3
• Neural iterated-learning studies found benefits associated with compositionality, but their experiments used extremely simple object and message structures.Separate ablations found reinitialization between generations necessary for benefits in some settings, while seeded IL was proposed to counter language drift in translation games.Source: §2 · p. 3

3 METHOD

• The dataset pairs questions, programs, images, and answers, with test examples drawn from unseen question templates.Question templates group questions with the same structure but different primitive values.Source: §3 · p. 4
• The program generator maps questions to programs, and the execution engine combines image inputs with those programs to predict answers.The task evaluates systematic generalization on question templates absent from training.Source: §3 · p. 4

3.1 MODEL ARCHITECTURE

• Tensor-FiLM-NMN combines FiLM-conditioned global operations with tensor-valued feature maps, joining features of Vector-NMN and Tensor-NMN.The paper explores Tensor-NMN, Vector-NMN, and this hybrid execution engine.Source: §3.1 · p. 4

3.2 ITERATED LEARNING FOR NMNS

• The authors treat programs as messages in an emergent layout language and use generational knowledge transfer to favor concepts that are easier to learn.Their stated hypothesis is that structure is easier for neural networks to learn than idiomatic non-compositional rules.Source: §3.2 · pp. 4–5
• The method cycles through interacting, transmitting, and learning phases, then starts the next generation with another interacting phase.During transmission, ground-truth programs are preserved when available to maintain an inductive bias toward desirable programs.Source: §3.2 · p. 5, Figure 2 · p. 5
• During interaction, agents jointly optimize answer prediction, while REINFORCE uses clipped negative cross-entropy as the generator’s reward.When ground-truth programs are available, weighted cross-entropy with teacher forcing supplies additional supervision; the authors report that small amounts of this supervision help generalization.Source: §3.2 · p. 5
• In the learning phase, limited generator updates create a bottleneck: too few risk underfitting, while too many risk overfitting to the previous generation’s idiosyncrasies.The new execution engine is trained using programs from the new generator to support stable training in the next interacting phase.Source: §3.2 · pp. 5–6

• SHAPES has 15,616 unique-image data points but only 244 unique questions, and its validation and test questions share training-question forms.Source: §4 · p. 6
• SHAPES-SyGeT divides 12 question templates into 7 training templates and 5 test templates to evaluate systematic generalization.Source: §4 · p. 6
• SHAPES-SyGeT uses remaining questions under training templates for Val-IID and evaluation-template questions for Val-OOD.Source: §4 · p. 6

5 EXPERIMENTS

• Experiments use three SHAPES-SyGeT seeds and eight CLEVR seeds; SHAPES-SyGeT uses raw images, while CLEVR uses pre-trained ResNet-101 image representations.Baselines use REINFORCE for program-generator gradients and cross-entropy for available program supervision; model selection and early stopping use Val-IID. The paper uses generic modules, making results hard to compare directly with SHAPES work using operation-specific modules.Source: §5 · p. 6, §5.1 · p. 7

5.1 SHAPES-SYGET

• Only IL produces gradually increasing systematic generalization during training, while all models reach perfect training accuracy in under 5000 steps.With IL, program accuracy rises through the learning bottleneck, indicating increasing structure in the program language.Source: §5.1 · pp. 7–8, Figure 4 · p. 8
• On SHAPES-SyGeT, seeded or unreinitialized execution engines harm generalization, while retaining the previous program generator without retraining greatly harms the best model.Spectral normalization alone has only a marginal generalization effect, but improves Val-IID and Val-OOD accuracy with IL in the standard execution-engine setting.Source: §5.1 · p. 8
• When the execution engine is not reinitialized, spectral normalization and retaining the program generator have small or reversed effects, which the authors associate with easier overfitting to images when programs are noisy.This behavior is observed on the small SHAPES-SyGeT dataset with a partially trained execution engine.Source: §5.1 · p. 8

5.2 CLEVR AND CLOSURE

• On CLEVR, retraining the execution engine from scratch each generation is computationally infeasible, and leaving it unreinitialized generalizes better than seeded IL.The execution engine takes over a day to reach 95% CLEVR validation accuracy even with ground-truth programs and no program generator.Source: §5.2 · p. 8

5.1 SHAPES-SYGET; 5.2 CLEVR AND CLOSURE

• IL improves generalization across SHAPES-SyGeT model configurations and supervision levels, and improves CLOSURE generalization and program accuracy while bringing CLEVR validation performance close to ProbNMN with fewer programs.On CLEVR, Tensor-NMN generalizes substantially better with IL; the experiments use 100 ground-truth programs.Source: §5.1 · p. 7, Table 1 · p. 7, Figure 3 · p. 7

6 CONCLUSION

• IL improves performance with less supervision and facilitates systematic generalization in VQA tasks using neural module networks.The authors report that learning-phase regularization, suitable architectures, and IL dynamics reduce the required ground-truth supervision; they single out spectral normalization for further study.Source: §6 · p. 9
• The conclusion states that fully unsupervised emergence of compositional language in VQA is thus far infeasible because jointly optimizing the program generator and execution engine is difficult.Source: §6 · p. 9

A HYPERPARAMETERS

• Table 3 details the experiment hyperparameters for SHAPES-SyGeT and CLEVR.Source: A HYPERPARAMETERS · p. 14

B.1 PROGRAM GENERATOR

• A seq2seq program generator translates questions into programs, serializing computation trees in Polish (prefix) notation because seq2seq models cannot directly model graphs.The implementation uses Bahdanau et al. (2019a)'s Seq2SeqAtt model with attention over encoder states; preliminary experiments found better generalization on held-out in-distribution questions than Johnson et al. (2017b).Source: §B.1 · p. 14

B.2 EXECUTION ENGINE

• The execution engine assembles the generated program into a neural network that maps image input x to an answer.Each program token corresponds to a reusable module; the work explores Tensor-NMN, Vector-NMN, and Tensor-FiLM-NMN architectures.Source: §B.2 · pp. 14–15

C ITERATED LEARNING ALGORITHM

• Algorithm 1 presents the full iterated-learning algorithm described in Section 3.2.Source: C ITERATED LEARNING ALGORITHM · p. 15

D SHAPES-SYGET TEMPLATES AND SPLITS

• SHAPES-SyGeT keeps the same primitives while creating splits with different combinations of them across question templates.The images use a 3 × 3 grid, and the dataset categorizes questions into 12 standard templates.Source: D SHAPES-SYGET TEMPLATES AND SPLITS · p. 15

E ABLATION EXPERIMENTS

• The SHAPES-SyGeT ablations use 20 ground-truth programs and focus primarily on Tensor-FiLM-NMN, which benefits more from IL than Tensor-NMN.The experiments use SHAPES-SyGeT because its computational requirements are an order of magnitude lighter than CLEVR/CLOSURE; results are presented in Table 6.Source: E ABLATION EXPERIMENTS · p. 15

F CLEVR VALIDATION CURVES

• Models trained with IL show higher systematic-generalization performance on CLOSURE, while CLEVR validation curves resemble the training curves.Source: F CLEVR VALIDATION CURVES · p. 15

G PRELIMINARY EXPERIMENTS WITH GQA

• The GQA setup uses question text, spatial features from a pretrained ResNet-101, and programs, without object annotations; programs are serialized in Polish notation.This serialization can duplicate token sequences because GQA programs do not always form strict directed acyclic graphs, and the stated limitations prevent competitive dataset results.Source: G PRELIMINARY EXPERIMENTS WITH GQA · p. 15
• Algorithm 1 divides learning into interacting, transmitting, and program-generator learning phases, with samples drawn from the training dataset and generated examples added to D′.The supplied steps also show an execution-engine learning phase that samples training tuples.Source: G PRELIMINARY EXPERIMENTS WITH GQA · p. 16
• The GQA experiment aims to test whether IL retains an advantage for neural module networks with limited ground-truth program supervision, rather than to pursue state-of-the-art performance.The greater complexity of GQA requires some modifications and special considerations.Source: G PRELIMINARY EXPERIMENTS WITH GQA · p. 17

G.1 CHANGES TO THE MODEL ARCHITECTURE

• Unlike SHAPES-SyGeT and CLEVR, GQA program steps include operations, optional sub-operations, arity, and argument tuples with negation flags.The example question is encoded as three tokens with arities 1, 1, and 0.Source: §G.1 · p. 17
• GQA program steps are represented as (Op, Subop, Args, ArgNegs, Arity) tuples rather than single operations.Short argument lists are padded to length three, enabling fixed-size logits for token distributions at multiple granularities.Source: §G.1 · pp. 17–18
• Table 6 names ablations that vary spectral normalization, program-generator retraining, and execution-engine resetting or seeding.The table caption defines FullSN, NoSN, NoRetrain, Seeded, and NoReset configurations.Source: Table 6 · p. 18
• Vector-NMN combines embeddings for Op, Subop, Args, and ArgNegs to create timestep-specific FiLM embeddings, avoiding dedicated modules for every program-step combination.The number of possible combinations is described as very large.Source: §G.1 · p. 18

G.2 CHANGES TO THE METHOD

• GQA's diverse structures and larger vocabularies make training the program generator from scratch in every iterated-learning generation impractical.The paper describes training the generator to decent performance as prohibitively slow even with ground-truth programs.Source: §G.2 · pp. 18–19
• Table 7 presents vocabulary sizes for SHAPES-SyGeT, CLEVR, and GQA token types after preprocessing.The supplied passage gives the table's scope but not its cell values.Source: Table 7 · p. 19
• The generator retains question and program input/output embeddings while its other parameters are reset between generations.The authors hypothesize that learning embeddings at multiple token levels accounts for a large portion of generator training time; this reset strategy generalizes better than no reset and an interpolation alternative.Source: §G.2 · p. 19
• The execution engine also retains its FiLM embeddings and resets its other parameters, a strategy used to combat overfitting on GQA images.The authors report that selected Vector-NMN hyperparameters tend to overfit despite a hyperparameter search.Source: §G.2 · p. 19

G.3 RESULTS

• GQA IL experiments use 4,000 ground-truth programs, or 0.4% of the available training programs.Results are reported on the balanced validation split as means and standard deviations over three trials.Source: §G.3 · p. 19
• IL shows a clear advantage in program-structure accuracy at multiple granularities on GQA.Figure 7 plots program accuracies at different granularities for models trained with 4,000 ground-truth programs.Source: Figure 7 · p. 20, §G.3 · p. 21
• Improved program accuracy correlates with improved GQA generalization, although iterated models have similar training accuracy.Figure 8 presents GQA training and validation curves for models trained with 4,000 ground-truth programs.Source: §G.3 · p. 21, Figure 8 · p. 21