Source-linked AI summary
Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control
Zihan Ye, Jiayi Liu, Puze Liu, Jiayun Li, Georgia Chalvatzaki, Jan Peters, Kristian Kersting
TL;DR
Opaque, task-specific safety formulations are difficult to interpret and transfer across tasks. NEUPRO learns visual safety predicates through differentiable reasoning over symbolic rules; experiments on REASON report cross-task generalization and explicit explanations of safety violations.
Problem
Existing robot safety formulations can be difficult to interpret and are often tied to specific tasks, while grounding visual safety requirements in explicit constraints remains challenging.
Source: §1 · p. 1Method
NEUPRO uses differentiable reasoning over expert-specified symbolic safety rules to train a visual predicate model that grounds observations in interpretable safety concepts.
Source: §1 · p. 2Results
On REASON, learned safety predicates generalize to novel tasks sharing safety semantics and provide explicit explanations of safety violations.
Source: §1 · p. 2Takeaways & Limitations
NEUPRO offers a transparent and reusable alternative to task-specific scalar costs and opaque safety representations.
Source: §6 · p. 7NEUPRO depends on the coverage of its safety rules and background knowledge, focuses on static visual observations, and relies on Grounding DINO to detect objects.
Source: §5 · p. 7
Abstract
from arXiv · showhide
As robots are increasingly deployed in everyday environments, ensuring their safety has become a central challenge. Existing methods often encode safety requirements as opaque mathematical/logical formulations or dense cost functions. While effective in specific tasks, they remain difficult to interpret, tightly coupled to individual tasks, and offer limited insight into why a robot action is considered safe or unsafe. To address this limitation, we propose ``Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control'' (NEUPRO), which leverages a differentiable reasoner that can learn reusable safety representations from human-specified safety knowledge. NEUPRO allows practitioners to express task-related safety requirements as transparent symbolic rules, while enabling gradients to propagate through these rules to a feature extractor that maps raw observations to safety-relevant concepts. As a result, the learned feature extractor is (softly) grounded in human-understandable semantics, supports transparent constraint evaluation, and is transferable across tasks. By coupling interpretability with differentiability, NEUPRO moves beyond opaque cost design toward reusable safety reasoning. To evaluate NEUPRO's capability, we collect and release REASON, the first real robot benchmark dataset for interpretable robot safety specification. Experiments on REASON show that NEUPRO learns safety-critical features that generalize across tasks, mitigate the interpretability limitations of conventional black-box cost formulations, and provide explicit explanations of safety violation.
1 Introduction
Existing safety formulations are difficult to interpret, transfer, and ground in visual observations. NEUPRO learns visual safety predicates through differentiable reasoning over symbolic rules, and the introduction presents REASON as a benchmark for evaluating interpretable safety reasoning.
1 Introduction
- Existing hand-crafted constraints are task-specific and costly to adapt, while learned representations can be policy-entangled and VLM predictions lack verification and formal guarantees.Abstract visual requirements such as not pointing a knife at a human are difficult to express as explicit mathematical constraints.Source: §1 · pp. 1–2
- NEUPRO uses transparent symbolic safety rules and differentiable reasoning to train visual predicates, with domain experts supplying the rules rather than learning them.Gradients flow from rule satisfaction to a predicate model that grounds observations in interpretable safety concepts.Source: §1 · p. 2
- REASON is introduced as a real-robot interpretable safety dataset linking perceptual observations to safety constraints across human–robot and robot–environment scenarios.Its structured annotations connect observations with interpretable constraints.Source: §1 · p. 2
- On REASON, learned safety predicates generalize to novel tasks sharing safety semantics and provide explicit explanations of violations, unlike conventional black-box costs.The reported results emphasize cross-task transfer and explanations of safety violations.Source: §1 · p. 2
- NEUPRO’s graph-based inference supports scalable predicate learning and safety reasoning, while its symbolic rules enable flexible reasoning from visual inputs.The introduction presents these capabilities as contributions of the framework.Source: §1 · p. 2
2 Related Work
Related work spans safe robot learning, differentiable logic, and neuro-symbolic planning. NEUPRO applies symbolic rules to explain safety satisfaction from visual observations, rather than using symbolic structure primarily for planning or skill composition.
2 Related Work
- Table 1 compares safety-reasoning paradigms on interpretability, verifiability, and flexibility, using symbols to mark strong, partial, or limited support.The caption defines ✓, ●, and ✗ as strong, partial, and limited support, respectively.Source: Table 1 · p. 2
- Tensor-based differentiable reasoning bridges symbolic entailment and gradient learning but is memory-intensive; NEUPRO’s graph reasoning targets memory efficiency and representation learning, not rule induction.The passage contrasts NEUPRO’s focus with graph-based work that induces logical rules.Source: §2 · p. 2
- Unlike neuro-symbolic approaches centered on planning and skill composition, NEUPRO uses symbolic rules to assess whether visual scenes satisfy safety requirements and why.Safety supervision also shapes its learned perceptual representation.Source: §2 · p. 2
- NEUPRO’s overview shows a perception module grounding safety-relevant atoms from images and a graph-based reasoner evaluating them with rules and commonsense knowledge.The reasoner produces a safety prediction from the grounded atoms and knowledge.Source: Figure 2 · p. 3
- Safe robot learning commonly uses manual constraints, control-theoretic mechanisms, or learned costs, which are often opaque, task-specific, or entangled with policies; NEUPRO instead uses interpretable FOL rules.The cited paradigms include constrained reinforcement learning, barrier functions, reachability analysis, and shielding.Source: §2 · pp. 2–3
3 Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control (NEUPRO)
NEUPRO grounds supplied symbolic safety rules in visual observations using a learned predicate model and a differentiable reasoner. Its sparse reasoning graph supports end-to-end learning while retaining links between predicates, rules, and safety predictions.
3 Neuro-Symbolic Predicate Learning for Semantic Safe Robot Control (NEUPRO)
- NEUPRO combines a Logical Predicate Model that maps images to soft grounded-atom truth values with a graph-based reasoner that evaluates first-order safety rules.The graph-based design avoids dense tensor-based differentiable reasoning.Source: §3 · p. 3
Problem Formulation
- The framework assumes symbolic safety rules are given and uses rule-level or partial predicate supervision to learn reusable predicate grounding instead of labeling every predicate in every scene.The target is accurate, interpretable prediction of rule satisfaction from observations.Source: Problem Formulation · pp. 3–4
Perceptual Symbol Grounding
- Grounding DINO (Liu et al. 2023b) provides object features selected by rule-term class, and the LPM outputs continuous valuations for grounded safety predicates.These valuations initialize differentiable reasoning; an atom can represent whether collision with an obstacle is likely.Source: Perceptual Symbol Grounding · p. 4
Reasoning Graph for NEUPRO
- A sparse graph aggregates grounded literals into conjunctions and rule-level disjunctions, while differentiable operations pass gradients to perception and preserve links to final safety predictions.The graph predicts each observation’s probability of satisfying each safety rule.Source: Reasoning Graph for NEUPRO · p. 4
Learning Objective
- NEUPRO trains predicted rule-satisfaction probabilities against binary satisfaction labels, masking out rules that are inactive for each observation.The objective shapes perception with symbolic safety semantics and provides an interpretable, end-to-end trainable bridge from observations to constraint satisfaction.Source: Learning Objective · p. 5
4 Experiments
Experiments assess predicate grounding, safety prediction and explanations, transfer, flexible reasoning, and scalability. They use REASON, a real-robot benchmark covering robot–human and robot–environment safety scenarios.
Experimental setup
- REASON pairs robot–human and robot–environment observations with safety predicates, symbolic constraints, and labels for evaluating the perception-to-reasoning pipeline.Evaluations use RGB images and compare NEUPRO with six baseline families; the tasks include object properties, spatial configurations, manipulation states, and human-directed motion.Source: §4 · p. 5
RQ1
- NEUPRO achieved the highest predicate-grounding accuracy, with VLMs less consistent on relational and human-interaction predicates.The comparison evaluates eight predicates from images.Source: §4 · p. 5, Figure 3 · p. 6
RQ2
- NEUPRO showed strong safety-classification accuracy across tasks, while VLMs achieved substantially lower accuracy even when given task rules.Table 2 compares safety-classification accuracy across REASON tasks; results are averaged over five test groups.Source: §4 · p. 6, Table 2 · p. 6
RQ3
- NEUPRO explained an unsafe prediction by composing grounded atoms for holding scissors and being close to a person through a logic rule.The explanation is illustrated qualitatively as first-order logic rules and atoms.Source: §4 · p. 6, Figure 4 · p. 6
RQ4
- NEUPRO maintained strong performance when predicates were applied to unseen objects or transferred to different tasks requiring the same semantic concepts.Object-level and task-level evaluations use objects and tasks different from those seen in training.Source: Figure 5 · p. 6, §4 · p. 7
RQ5
- NEUPRO distinguished safety outcomes when identical spatial relations involved objects with different properties, rather than relying on fixed visual associations.The evaluated contrasts include hard versus soft objects above a laptop and sharp versus non-sharp objects near a person.Source: §4 · p. 7, Table 3 · p. 7
RQ6
- At 102 rules, graph-based NEUPRO matched tensor-based accuracy while training 9.1× faster, using 14.3× less peak GPU memory, and inferring 1.6× faster.The authors expect the efficiency advantage to grow as the number of rules increases.Source: §4 · p. 7, Figure 6 · p. 7
5 Limitations
NEUPRO’s reasoning is bounded by its knowledge base and visual inputs, and currently addresses static rather than temporal safety. Its inference also uses a single view, despite REASON containing multiple viewpoints.
5 Limitations
- Reasoning depends on the quality and coverage of safety rules and background knowledge; missing conditions or object properties may leave violations unidentified.Source: §5 · p. 7
- NEUPRO currently reasons from static visual observations and relies on Grounding DINO, so missed object detections leave it without information about those objects.Source: §5 · p. 7
- The current method does not model temporal constraints such as motion trends or future consequences; temporal predicates and predictive world models are proposed directions.Source: §5 · p. 7
- Although REASON contains images from multiple viewpoints, inference uses one image, which may limit robustness when that view is occluded.Source: §5 · p. 7
6 Conclusion
NEUPRO uses graph-based reasoning over interpretable first-order logic rules to support flexible safety reasoning from visual observations. REASON experiments validate its interpretability, flexibility, and scalability, positioning it as a transparent, reusable alternative to task-specific scalar costs and opaque representations.
6 Conclusion
- NEUPRO represents safety specifications as interpretable first-order logic rules and uses graph-based reasoning to support scalable predicate learning from raw images.Source: §6 · p. 7
- Experiments on REASON validate NEUPRO’s interpretability, flexibility, and scalability for safety reasoning.Source: §6 · p. 7
- NEUPRO offers a transparent and reusable alternative to task-specific scalar costs and opaque safety representations.Source: §6 · p. 7