What can you do with a rock? Affordance extraction via word embeddings

1 Introduction

• Affordances are actions feasible in a given situation; agents that miss them may waste effort on impossible or futile behaviors.The paper contrasts this with people, who detect affordances automatically, often subconsciously.Source: §1 · p. 1
• Wikipedia-trained word embeddings serve as a common-sense knowledge base that can be queried with linear algebra to extract object-specific affordances.The approach uses linguistic information in online corpora to connect words and common-sense knowledge.Source: §1 · p. 1
• In a text-based environment, affordance extraction speeds learning by identifying actions an object affords and distinguishing manipulable objects from those that can only be observed.The agent explores actions that make sense and discovers valuable behaviors more quickly than a comparable brute-force agent.Source: §1 · p. 1
• The agent eliminates extraneous actions without discarding beneficial ones in most cases.This qualification describes the reported action-selection behavior, not a guarantee for every case.Source: §1 · p. 1

2 Related Work

• Word2vec encodes words from their contexts, and prior work reports semantic structure such as gender and geography [Mikolov et al., 2013b; Mikolov et al., 2013c].The paper gives vector[‘king’] − vector[‘man’] + vector[‘woman’] = vector[‘queen’] as an example, following [Mikolov et al., 2013a].Source: §2 · p. 1
• Word-vector properties have been used for sentence and paragraph encoding, image categorization, retrieval, semantic segmentation, biomedical document retrieval, and matching movie scripts to source texts [Kiros et al., 2015; Le and Mikolov, 2014; Karpathy et al., 2014; Brokos et al., 2016; Zhu et al., 2015].The paper says its work is most similar to, but uses skip-gram semantic structure instead of a Markov Logic Network for an explicit knowledge base.Source: §2 · p. 1
• Affordance detection has been attempted with visual features, haptic data, visuomotor simulation [Schenck et al., 2016], repeated experimentation, and knowledge bases.For text-based game play, the paper contrasts its approach with an LSTM state representation and DQN action selection [Narasimhan et al., 2015]; it reports that Q-value clarity broke down as environment complexity and alphabet size grew, leaving negative overall reward.Source: §2 · pp. 1–2

3 Wikipedia as a Common Sense Knowledge Base

• Although Wikipedia is unstructured and can contain irrelevant or erroneous data, word2vec produces a compact knowledge representation that can encode relationships and does not require explicit maintenance as the corpus updates.The paper contrasts it with curated resources such as ConceptNet, Cyc, and WordNet.Source: §3 · p. 2
• Because semantic relationships are not readily apparent in raw vectors or PCA reductions, the paper uses mathematical queries; projecting gendered terms onto word-difference axes makes masculine and feminine terms linearly separable.Figure 1 uses vector[‘king’] − vector[‘queen’] and vector[‘woman’] − vector[‘man’] to define the projection space.Source: §3 · p. 2, Figure 1 · p. 2
• The paper uses “affordant” more narrowly for actions that seem reasonable, rather than all physically possible actions.For example, eating a pencil is physically possible but does not make sense.Source: §3 · p. 2

3.1 Verb/Noun affordances

• Verb–noun analogies use canonical pairs to ask what action is affordant for a target noun; averaging multiple canonical values reportedly gives better results than using one.The method averages verb-minus-noun vectors from canonical pairs and searches for verbs near the target noun plus that affordance vector.Source: §3.1 · pp. 2–3
• For “sword,” the analogy query returns verbs including “vanquish,” “duel,” “unsheathe,” and “wield,” and subjective examination finds Wikipedia-derived affordances at least as relevant as traditional methods.Figure 2 compares the sword associations with strict co-occurrence counts using a nine-word window and ConceptNet’s CapableOf relationship.Source: §3.1 · p. 3, Figure 2 · p. 3
• The algorithm is not resilient to polysemy: for “apple,” software-company associations dominate the results over fruit-related actions.The paper attributes this example to “Apple, the software company” being more strongly represented in the corpus than “apple, the fruit.”Source: §3.1 · p. 3

3.2 Identifying graspable objects

• For object selection, the paper treats affordant nouns as manipulable in meaningful ways, such as being pushed, pulled, grasped, transported, or transformed.This distinguishes manipulable objects from objects such as a sunset or cliff.Source: §3.2 · p. 3
• Word-difference projections locate an object along manipulability axes; Figure 3 places small manipulable objects lower-left and large, abstract, or background objects upper-right.The paper says manipulability can be roughly estimated from an object's position along either defining axis.Source: §3.2 · p. 3, Figure 3 · p. 3

4 Test Environment: A World Made of Words

• Agents must balance a broad action set against a growing search space, while avoiding reductions that risk losing information.The goal is to explore only actions likely to produce results, as a human might.Source: §4 · p. 4

4.1 Learning algorithm

• Q-learning updates use α = 1 and a constrained update intended to retain behaviors that have produced reward at least once.This is meant to prevent the agent from unlearning behaviors that may be useful in a later training round.Source: §4.1 · p. 4
• The agent hashes game text as its state and represents each action as a verb/object pair, with verbs drawn from V and nouns from O.It issues a “look” command every second iteration and strips numerals from the text before hashing.Source: §4.1 · p. 4
• The affordance agent reduces its manipulation list from 1000 verbs to 30 and its object list to at most 15 nouns per state.The baseline uses common Wikipedia verbs, navigation and essential-manipulation lists, and nouns extracted from game text with preceding adjectives.Source: §4.1 · pp. 4–5
• The agent has no inventory or score representation and cannot construct prepositional commands such as “put book on shelf.”These omissions were made to focus the research on affordance-based search-space reduction.Source: §4.1 · p. 5

5 Results

• The evaluation used 50 Z-machine-compatible games with little or no information about goals or rewarding actions, training for 1,000 epochs of 1,000 steps each.The game restarted and its score reset after each epoch, while the agent retained learned Q-values.Source: §5 · p. 5
• Affordance-based reduction scored points on 16/50 games; peak performance was 23.40% for verbs, 4.33% for objects, and 31.45% combined, versus baseline 12/50 and 4.45%.Across Figure 7's five techniques and 100 epochs, affordance reduction improved overall performance on 12/16 games and decreased it on one; results average five runs and are normalized per game.Source: §5 · p. 5, Figure 7 · p. 6
• Two games with termination errors were excluded from subsequent analysis, and 32 games yielded no points because of command, event-sequence, or object-naming demands.The authors report that reduction methods still outperformed the baseline in both peak performance and average reward in the excluded partial results.Source: §5 · p. 5

5.1 Alternate reduction methods

• Compared with a 1,000-verb baseline, affordance-based verb selection improved performance in most games, while the other reduction methods sometimes pruned actions essential for reward.The alternatives included intrinsic rewards, random verb reduction, ConceptNet CapableOf filtering, and Wikipedia verb–noun co-occurrence filtering.Source: §5.1 · p. 6

5.2 Fixed-length vocabularies vs. Free-form learning

• Free-form learning constructed a state-specific manipulation list from the top 15 responses, and its sampled actions appeared reasonable and inventive in the game context.The paper suggests that further research in this direction may enable one-shot learning for text-based adventure games.Source: §5.2 · p. 6

• Wikipedia's implicitly encoded common-sense knowledge may open new opportunities for autonomous agents.Source: §6 · p. 6
• In text-based adventure games, the approach efficiently navigated previously intractable search spaces and outperformed several intuitive methods.Source: §6 · p. 6
• Experiments were limited to text environments, while the authors say the principles apply wherever mappings can be formed between words and objects.They identify self-driving cars, domestic robots, and UAVs as potential application domains.Source: §6 · p. 6