PaperlayerView exact revision on arXiv

Source-linked AI summary

Relational inductive bias for physical construction in humans and machines

Jessica B. Hamrick, Kelsey R. Allen, Victor Bapst, Tina Zhu, Kevin R. McKee, Joshua B. Tenenbaum, Peter W. Battaglia

arXiv:1806.01203v1cs.LGcs.AIstat.ML

TL;DR

The paper asks whether relational reasoning can help deep learning systems handle interactive physical construction, a task requiring choices among objects and their relations. It evaluates people on a block-gluing task and develops a graph-network deep RL agent with object- and relation-centric representations. The agent outperformed humans and agents without relational inductive bias, supporting the relevance of relational representations in this setting.

  • Problem

    Deep RL systems rarely represent relations explicitly and often struggle with structured combinatorial problems such as selecting glue placements across variable block configurations.

    Source: Introduction · p. 1

  • Method

    The authors introduce the gluing task, measure human performance, and develop a deep RL agent whose graph-network policy operates on object- and relation-centric representations.

    Source: Introduction · p. 2, Graph Networks · p. 3

  • Results

    The object- and relation-centric agent solved the gluing task better than humans, while an agent without relational inductive bias performed far worse.

    Source: Discussion · p. 6

  • Takeaways & Limitations

    The results suggest relational inductive bias can support physical interaction even without an explicit model of physical dynamics.

    Source: Discussion · p. 6

    The reward structure may be unfair to humans because it was chosen to incentivize agents to complete the task quickly.

    Source: Results · p. 2

Abstract

from arXiv · show

While current deep learning systems excel at tasks such as object classification, language processing, and gameplay, few can construct or modify a complex system such as a tower of blocks. We hypothesize that what these systems lack is a "relational inductive bias": a capacity for reasoning about inter-object relations and making choices over a structured description of a scene. To test this hypothesis, we focus on a task that involves gluing pairs of blocks together to stabilize a tower, and quantify how well humans perform. We then introduce a deep reinforcement learning agent which uses object- and relation-centric scene and policy representations and apply it to the task. Our results show that these structured representations allow the agent to outperform both humans and more naive approaches, suggesting that relational inductive bias is an important component in solving structured reasoning problems and for building more intelligent, flexible machines.

Introduction

The paper argues that flexible physical interaction depends partly on reasoning over objects and their relations, a capacity its authors call relational inductive bias. It introduces the gluing task and a graph-network deep RL agent to examine that claim.

Introduction

  • Relational inductive bias builds assumptions about the relational structure of the world into a learning system.The paper defines inductive bias as assumptions that guide a learning algorithm toward one hypothesis over another independently of observed data.Source: Introduction · p. 1
  • The gluing task requires reasoning over variable object configurations, variable action spaces, and combinatorial glue choices, and the authors argue relational knowledge is harder to acquire than physical knowledge.They use the task to examine whether relational, rather than physical, inductive bias is most important for solving it.Source: Introduction · p. 1
  • The authors instantiate this bias in a deep RL agent using a graph network and contrast it with holistic approaches lacking explicit object and relation representations.They distinguish their interactive construction focus from prior physical-reasoning work centered on prediction or inference.Source: Introduction · p. 1
  • The paper aims to characterize inductive biases for physical construction, not to model human solutions precisely or claim state-of-the-art gluing performance.Its contributions include introducing the task, measuring human performance, developing an object- and relation-centric agent, and comparing it with alternatives and humans across the gluing and control tasks.Source: Introduction · p. 2

The Gluing Task

The gluing task measures how people stabilize unstable block towers by choosing which contacting pairs to glue. Participants performed above chance, with evidence of systematic strategies and a tendency toward unnecessary glue.

The Gluing Task

  • Participants completed 144 trials in one-hour sessions, judging towers of 2–10 blocks and earning points for stable blocks while losing a point per attempted glue pair.The 135 experimental towers were divided into five sets, and using the minimum glue necessary earned a 10-point bonus.Source: Participants · p. 2, Stimuli and Design · p. 2
  • The authors characterize these human findings as a preliminary exploration that provides a baseline for artificial agents.The reward structure penalized glue use to better match agent incentives to complete the task quickly, a choice the authors note may be unfair to humans.Source: Results · p. 2
  • Participants averaged 900 points out of 1977, with scores ranging from 468 to 1154, and showed only a small, nonsignificant learning effect across trials.The learning correlation was r = 0.15, 95% CI [−0.01,0.30].Source: Results · p. 3
  • Participants were slower on their first gluing action than later actions, suggesting they may decide where to glue before acting or first encode a useful scene representation.They were also slower to select the first block in a pair than the second.Source: Results · p. 3
  • Ten of 25 participants reported selecting glue locations top-to-bottom, and another three sometimes alternated direction; action-height slopes also generally indicated top-to-bottom selection.The fitted slope was β = −0.07, 95% CI [−0.08,−0.06].Source: Results · p. 3
  • People were more likely to apply unnecessary glue than omit necessary glue, with unnecessary applications accounting for 73% of errors; invalid non-contact gluing occurred 1.3% of the time.Participants unglued connections in 0.29% of actions; the authors suggest penalty-free ungluing as a future question.Source: Results · p. 3

Leveraging Relational Representations

The agent represents blocks and glue relations in a graph network and uses relational reasoning for construction decisions. In the gluing task, graph agents outperform less structured approaches, with correct relational structure improving performance further.

Leveraging Relational Representations

Graph Networks

  • The tower graph represents blocks as nodes, block pairs as edges, and position, orientation, glue, and global properties as graph features.The agent decodes edge-specific actions and a stop action from updated graph representations.Source: Graph Networks · pp. 3–4

Supervised Learning Experiments

  • Sparse graph inputs learned stability more efficiently, and one recurrence dramatically improved optimal-glue prediction.The five-block figure’s findings were also consistent for stability prediction on towers with 6–9 blocks.Source: Figure 3 · p. 4, Supervised Learning Experiments · p. 4

Sequential Decision Making Experiments

  • GN-FC earned M = 883.60, 95% CI [719.40,1041.00], more points on average than MLP.The comparison is shown in Figure 4a.Source: Figure 4 · p. 5, The effect of relational structure · p. 5
  • GN earned M = 183.20, 95% CI [73.20,302.40], more points on average than GN-FC when given the correct relational structure.GN-FC chose invalid edges 31% of the time, including nonadjacent and self-edges.Source: Figure 4 · p. 5, The effect of relational structure · p. 5
  • GN showed no detectable degradation on held-out 7- and 10-block towers, while GN-FC struggled to extrapolate to 10-block towers.The MLP was trained separately for each tower size and could not generalize to new sizes by design.Source: The effect of relational structure · p. 5
  • The simulation agent earned M = 156.20, 95% CI [70.80,249.60], more points on average than GN, but used a perfect simulator and a greedy policy.The authors note that a more realistic simulator would likely fare somewhat worse.Source: The effect of physical knowledge · p. 5
  • Human participants’ average reward fell between the MLP and GN-FC agents, and their errors more often involved unnecessary glue than the GN agent’s errors.The authors suggest perceptual uncertainty may lead people to overestimate tower instability.Source: Comparison to humans · p. 5

Discussion

The paper reports that object- and relation-centric policies can outperform humans on the gluing task, while agents without relational inductive bias perform worse. It also discusses how relational representations may apply beyond physical scenes.

Discussion

  • Humans performed above chance and used systematic strategies, including working top-to-bottom and reasoning about the whole glue configuration before acting.The best-performing agent was the simulation agent, which used both relational and physical knowledge.Source: Discussion · pp. 5–6
  • The object- and relation-centric agent solved the gluing task better than humans, while an agent without relational inductive bias performed far worse.The authors conclude that relational bias can be effective even without an explicit model of physical dynamics.Source: Discussion · p. 6
  • Relational reasoning may also apply to cognitive maps, scheduling, and social interaction, where entities or events participate in relationships.The paper suggests graph networks could help model these cognitive domains.Source: Discussion · p. 6
  • The authors conclude that relational inductive biases can improve deep RL and support formal cognitive models of complex interactive behaviors.Source: Discussion · p. 6

Supplementary Material

The supplementary material describes the agents’ graph-network architecture and training setup. Graph networks were trained across tower sizes, whereas the MLP used separate agents for each size.

Architectural Details

  • The GN agent represents glue with a sparse vector for contacting block pairs and is told which blocks are in contact; the GN-FC agent instead matches the MLP’s inputs and outputs.Source: GN Agents · p. 6
  • The graph-network agents encode nodes, edges, and global features, use recurrent processing and a second decoder, and decode edge and global values into outputs.The second graph-network decoder helps decompose the task, for example by detecting contacts before selecting pairs to glue.Source: GN Agents · p. 6

Training Procedure

  • Both agent types trained for 300k episodes on 100k scenes per tower size, but nine separate MLP agents were trained while GN agents used a curriculum across sizes.The GN curriculum added the next tower size every 10k episodes while retaining previous sizes.Source: Training Procedure · p. 6
Loading 1806.01203v1…