PaperlayerView exact revision on arXiv

Source-linked AI summary

Modular Design Patterns for Hybrid Learning and Reasoning Systems: a taxonomy, patterns and use cases

Michael van Bekkum, Maaike de Boer, Frank van Harmelen, André Meyer-Vitali, Annette ten Teije

arXiv:2102.11965v2cs.AIcs.LG

TL;DR

Hybrid AI research has grown rapidly, but its diverse systems lacked a shared conceptual framework. This paper proposes a taxonomy and modular design patterns that compose elementary building blocks; the patterns group otherwise distinct systems under common architectures and are applied to two realistic use cases.

  • Problem

    Hybrid AI systems combine complementary data-driven and knowledge-driven approaches, but their diverse literature lacked a conceptual framework for discussing, comparing, configuring, and combining them.

    Source: §1 · p. 1

  • Method

    The paper develops a taxonomical vocabulary and modular elementary and compositional design patterns for describing hybrid systems and applies them in two use cases.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    The patterns capture shared architectures across distinct approaches, including ILP, MLN, PSL, and knowledge-graph embeddings, and the use cases demonstrate communicable blueprints of complex hybrid systems.

    Source: §3 · pp. 5–6, §7 · p. 15

  • Takeaways & Limitations

    The boxology aims to support understanding and communication across AI communities and to provide a blueprint for complex systems integrating different AI technologies.

    Source: §1 · p. 2, §7 · p. 15

    Actors are not explicitly used in the current pattern collection, with distributed AI and human-agent interaction left for future work.

    Source: §2.4 · p. 4

Abstract

from arXiv · show

The unification of statistical (data-driven) and symbolic (knowledge-driven) methods is widely recognised as one of the key challenges of modern AI. Recent years have seen large number of publications on such hybrid neuro-symbolic AI systems. That rapidly growing literature is highly diverse and mostly empirical, and is lacking a unifying view of the large variety of these hybrid systems. In this paper we analyse a large body of recent literature and we propose a set of modular design patterns for such hybrid, neuro-symbolic systems. We are able to describe the architecture of a very large number of hybrid systems by composing only a small set of elementary patterns as building blocks. The main contributions of this paper are: 1) a taxonomically organised vocabulary to describe both processes and data structures used in hybrid systems; 2) a set of 15+ design patterns for hybrid AI systems, organised in a set of elementary patterns and a set of compositional patterns; 3) an application of these design patterns in two realistic use-cases for hybrid AI systems. Our patterns reveal similarities between systems that were not recognised until now. Finally, our design patterns extend and refine Kautz' earlier attempt at categorising neuro-symbolic architectures.

A PREPRINT: UNDER REVIEW

The preprint lists its authors, affiliations, and keywords.

A PREPRINT: UNDER REVIEW

1 Introduction

The introduction frames hybrid AI as a growing but conceptually fragmented area, then proposes modular design patterns to describe, compare, and engineer systems that combine learning and reasoning.

1 Introduction

  • The paper proposes modular design patterns to describe hybrid systems combining data-driven and knowledge-driven AI components.The patterns are intended to abstract system architectures from technical details and reveal commonalities across systems.Source: §1 · p. 1
  • The framework addresses a literature gap: existing surveys cover narrow system families, but no conceptual framework was available to discuss, compare, configure, and combine hybrid systems.The authors motivate the framework by the complementary strengths and weaknesses of data-driven and knowledge-driven approaches.Source: §1 · p. 1
  • The patterns aim to bridge communities whose differing terminology hampers communication about hybrid AI systems.The authors compare their engineering purpose to software-engineering design patterns.Source: §1 · p. 2
  • The proposed composition patterns offer modular building blocks for systems in which trained networks and logic communicate.Source: §1 · p. 2
  • The paper contributes a taxonomy, elementary and compositional patterns, and two realistic use cases covering skill matching and robot action selection.The patterns describe systems from recent literature and elaborate the authors’ earlier proposal [63].Source: §1 · p. 2

2 A Taxonomical Vocabulary

The taxonomy provides a shared vocabulary for describing hybrid systems through their instances, models, processes, and actors, with abstraction levels adjustable to design needs.

2 A Taxonomical Vocabulary

  • The boxology uses rectangles for instances, hexagons for models, ovals for processes, and triangles for actors, with type annotations such as model:stat:NN specifying detail.Descriptions should abstract from implementation details while documenting applicable design choices; greater specificity is appropriate closer to implementation.Source: §2 · p. 2

2.1 A Taxonomy of Instances

  • Instances include data and symbols: symbols designate interpreted objects, classes, or relations and support compositional operations, whereas data items do not designate in that way.The passage contrasts a symbol designating a person with a picture that is only itself.Source: §2.1 · pp. 2–3

2.2 A Taxonomy of Models

  • Models are either statistical, representing dependencies among variables, or semantic, representing meanings and relationships through a knowledge base.Examples include neural networks and Bayesian networks for statistical models, and ontologies, knowledge graphs, and rulebases for semantic models.Source: §2.2 · p. 3

2.3 A Taxonomy of Processes

  • Processes generate, transform, or infer over instances and models; induction moves from specific instances to a model, while deduction applies a general model to specific instances.Classification and prediction are deductive processes in this taxonomy.Source: §2.3 · p. 3

2.4 A Taxonomy of Actors

  • Actors can be humans, software agents, or robots, but actors are not explicitly used in the current pattern collection.The authors reserve distributed AI and human-agent interaction for future inclusion.Source: §2.4 · pp. 3–4

3 Elementary Patterns

The elementary patterns describe learning and inference as reusable building blocks, which can be specialized and composed to capture diverse hybrid architectures.

3 Elementary Patterns

  • The train pattern produces a model from data or symbols, and an actor such as a domain expert can also create a semantic model.Source: §3 · p. 4
  • Transformations can prepare the required data type, while infer patterns use models with data or symbols to derive conclusions.The elementary patterns also include transformations of semantic models into other data structures.Source: §3 · pp. 4–5
  • The inputs constrain the types of models and results in these elementary patterns.Source: §3 · p. 5
  • The paper defines hybrid systems as combinations of elementary patterns other than the standard machine-learning combination (1a with 2a) and the knowledge-representation pattern (2b).Source: §3 · p. 5
  • The elementary patterns group Inductive Logic Programming, Markov Logic Networks, and Probabilistic Soft Logic as approaches that learn over symbolic structures.Despite differing algorithms and representations, the passage identifies architectural pattern 1b as applying to all three.Source: §3 · p. 5
  • The learning and prediction patterns compose into full train-and-infer architectures, including ordinary neural networks whose feed-forward phase the authors classify as deductive reasoning.Learning may operate on symbols as well as data; the resulting patterns describe systems including Inductive Logic Programming, knowledge-graph embeddings, Probabilistic Soft Logic, and Markov Logic Networks.Source: §3 · pp. 5–6
  • A generic instance-based pattern abstracts over data and symbols, while type annotations recover their specific forms and expose shared architecture across approaches.The passage names ILP, MLN, PSL, and knowledge-graph embeddings as approaches that follow the same schema.Source: §3 · pp. 5–6

4 A collection of compositional patterns

The section builds compositional patterns by stitching or specializing elementary patterns. It illustrates systems that learn symbolic outputs and systems that generate explanations from learned models or their predictions.

4 A collection of compositional patterns

  • Compositional patterns are formed by connecting elementary patterns or by specifying them at a more detailed or abstract level.The section combines work from several fields within these compositions.Source: §4 · p. 6

4.1 Learning from data with symbolic output

  • Ontology learning extracts subject–verb–object relations from text, then applies semantic rules to produce an ontology with relation hierarchies and axioms.The relations are an intermediate representation, and the ontology is the process’s output rather than an input model.Source: §4.1 · p. 6
  • Text-mining can produce only class/instance labels instead of a full ontology, while retaining the same architectural pattern.The variation changes the symbolic output’s content, not the pattern’s architecture.Source: §4.1 · p. 6
  • Some ontology-learning systems begin with given relations and infer a hierarchy, using only the second half of the text-to-ontology pipeline.These systems apply pattern (2b) without first extracting relations from text.Source: §4.1 · p. 6

4.2 Explainable learning systems through rational reconstruction

  • Pattern 5a generates a post-hoc symbolic justification for each learned prediction, but that justification need not reflect the machine learner’s internal computation.The symbolic reasoner uses background knowledge to reconstruct a rationale for an input/output pair.Source: §4.2 · p. 7
  • Pattern 5a explains individual results, while a model-level variant learns first-order logic formulas from a classifier’s input/output behavior.The variant takes the trained model as input to explanation, rather than explaining only an individual prediction.Source: §4.2 · p. 7

4.3 Learning an intermediate abstraction

Intermediate representations can connect learning stages or feed deductive reasoning, while symbolic knowledge can guide learning. The section also describes variations that reuse, constrain, or iteratively improve these representations and learning processes.

4.3 Learning an intermediate abstraction

  • Learning an intermediate symbolic representation before addition is described as more robust than end-to-end bitmap-to-sum training, and related abstractions support spatial or temporal tasks and transfer learning.The cited work reports higher transfer rates after trivial changes to inputs such as handwritten digits or floor-space bitmaps.Source: §4.3 · p. 8
  • In AlphaGo, a learned evaluation function gives rise to a symbolic search tree that Monte Carlo tree search traverses to derive a next move.This composes learned abstraction with deductive reasoning; pattern 5a is described as a specialization of this pattern.Source: §4.3 · p. 8
  • Ontology learning treats linguistically extracted relations as an intermediate abstraction for rules that construct the final ontology, with pretrained semantic models allowing training phases to be omitted.The pattern is presented as a variation on learning an intermediate abstraction.Source: §4.3 · p. 8

4.4 Informed learning with prior knowledge

  • Pattern 7 uses symbolic inference to provide prior information that guides model training, including semantic loss functions that penalize violations of background knowledge.Examples include knowledge-graph priors, weighted model counting, approximate constraint satisfaction, logical rules, and type hierarchies.Source: §4.4 · p. 9
  • Logic Tensor Networks jointly minimize neural-network loss and maximize first-order theory satisfaction, an analogy to semantic loss functions highlighted by the pattern analysis.The paper notes that this analogy was not mentioned in the original LTN papers.Source: §4.4 · p. 9
  • Symbolic constraints can constrain reinforcement-learning exploration for safety, and symbolic plans can guide policy learning; one cited system reports improved sampling frequency for a combinatorial task.LYRICS provides an interface for arbitrary first-order background knowledge to constrain weight learning.Source: §4.4 · p. 9
  • A symbolic prior may be an explicit knowledge base rather than an inference result, yielding a simplified pattern; cited examples use ontologies or knowledge graphs as priors.One cited classifier performs better on ontology-abstracted symbolic data than on raw data.Source: §4.4 · p. 9
  • An iterative variant alternates between producing symbolic knowledge and using it to inform learning, then uses the learned model to produce improved symbolic knowledge.The paper cites IterefinE as another example of this pattern.Source: §4.4 · p. 9

4.5 From symbols to data and back again

These patterns cover transformations between symbolic and statistical representations, embedding logical knowledge for neural learning, and training neural networks to perform reasoning tasks.

4.5 From symbols to data and back again

  • Knowledge-graph link prediction embeds graph structure in a high-dimensional vector space, then uses geometric regularities to predict missing edges.The paper identifies this as pattern (8), a task at the boundary of symbolic and statistical representations.Source: §4.5 · p. 10

4.6 Learning logical representations for statistical inferencing

  • Pattern 9 tensorizes semantic knowledge as neural-network input or constraints, enabling learning with embedded logic and examples including Logic Tensor Networks, graph neural networks, and multi-hop inference.The pattern extends training on data to training on transformed representations of relational data.Source: §4.6 · p. 10

4.7 Learning to reason

  • Pattern 10 trains neural networks to perform reasoning tasks using symbolic input tuples and inference results from a symbolic reasoner.The pattern combines symbolic reasoning with statistical-model training and is motivated by neural networks’ potential scalability and robustness to noisy inputs.Source: §4.7 · p. 11
  • Neural reasoning approaches include RDF, description-logic, and logic-programming tasks; examples learn class membership, relation existence, or query entailment from normalized triples.Although current efforts focus on deductive reasoning over FOL and DL knowledge bases or fragments, the pattern could apply to other inference tasks and mechanisms.Source: §4.7 · p. 11

4.8 Meta reasoning for control

The section relates meta-reasoning to configuring and guiding learning systems. It describes knowledge-based hyperparameter selection with iterative performance feedback, curriculum sequencing, and subsymbolic guidance of theorem proving.

4.8 Meta reasoning for control

  • Meta-reasoning systems reason about or learn from the behavior of another system.The passage situates this as a longstanding topic in AI and cognitive architectures.Source: §4.8 · p. 11
  • Pattern 11 uses machine-learning configuration knowledge to deduce hyperparameters, train a model, and inspect its performance to adapt settings in the next iteration.Alpine Meadow is cited as an example.Source: §4.8 · pp. 11–12
  • Curriculum-guided learning fits the pattern when a symbolic component supplies training instances in an efficient sequence to maximize learning speed.The passage defines the curriculum problem in terms of finding the most efficient sequence of learning situations.Source: §4.8 · p. 12
  • Unlike pattern 7’s one-time symbolic prior, pattern 11 inspects learning to adjust guidance; the reverse direction is also possible, with a subsymbolic system learning search strategies for a symbolic theorem prover.The cited theorem-proving line of work dates back to the 1990s.Source: §4.8 · p. 12

5 Two Use-cases

The boxology makes hybrid-system designs easier to communicate and refine, and the two use cases show how it can describe concrete learning-and-reasoning architectures.

5 Two Use-cases

  • The boxology clarifies complex hybrid-system designs by separating processes and data and supporting step-wise refinement toward implementation choices.The authors present this as a way to build or understand systems using reusable patterns and components.Source: §5 · p. 12

5.1 Skills Matching

  • The boxology made the skills-matching architecture more readable and supported discussion of both elementary patterns and specific model inputs and outputs.It also helped consider patterns that might later be added, substituted, or removed.Source: §5.1 · p. 13
  • In skills matching, embeddings turn vacancies and a human-engineered skills ontology into tensors used to train a neural network to identify and map skills.At application time, the system predicts a skill and uses the ontology to derive a more understandable label.Source: §5.1 · p. 13

5.2 Robot in Action

  • For the robot, an object detector, tracker, and handcrafted world ontology feed reasoning that predicts the best action at each timestep.The system uses tracks and the world model to induce semantic rules before predicting an action.Source: §5.2 · p. 13
  • In the robot use case, the boxology helped identify similarities between the object detector and tracker and determine the detailed process flow.Source: §5.2 · p. 13

6 Related Work

Related surveys and taxonomies offer finer distinctions in particular areas, while the boxology focuses on configuring modular components into hybrid-system architectures. Its patterns also refine Kautz’s categories by specifying internal modular architectures.

Informed machine learning

  • The survey of over 100 informed-machine-learning systems analyzes where knowledge enters learning in detail, while the boxology covers those systems under pattern 7.Source: §6 · p. 14

Kautz taxonomy

  • Kautz’s six system types share the goal of describing neural-symbolic interaction, but the boxology refines them into subtypes with internal modular architectures.Some Kautz types map to multiple boxology patterns, while Type 6 remains unclear in the authors’ account.Source: §6 · p. 14

Hybrid-systems survey

  • The surveyed hybrid-systems framework distinguishes dimensions such as graphical-model direction and inference type, which the boxology groups within shared components.For example, directed and undirected graphical models are captured by one semantic-model component, and inference forms by one KR component.Source: §6 · p. 14

Related taxonomies

  • De Raedt et al. make finer distinctions inside knowledge-representation and model components, whereas the boxology addresses how components are configured for system functionality.The approaches could be combined in a future version.Source: §6 · pp. 14–15

7 Conclusion and Future Work

The paper presents the boxology as a visual language for representing learning and reasoning systems, with two use cases demonstrating communicable blueprints. Future work includes extending its concepts and developing it into a reusable architectural toolkit.

Conclusion

  • The boxology represents learning and reasoning systems, and two use cases demonstrate communicable blueprints for complex systems combining AI technologies.Source: §7 · p. 15

Future Work

  • The authors expect broader use to extend the taxonomy and visual language, and say implementation guidelines remain to be documented.Source: §7 · p. 15
  • Future extensions include actor concepts for modeling autonomous entities, human-in-the-loop systems, and interactions such as federated learning or multi-robot coordination.The paper also points to an extension with human feedback-provider and feedback-consumer patterns.Source: §7 · p. 15
  • The authors aim to develop the boxology into reusable design, implementation, and deployment tools, including a generative grammar and logic calculus for composing and verifying patterns.Source: §7 · p. 15
  • A coherent methodology is expected to make hybrid-system design easier to understand and maintain and to support explainability, responsibility, reliability, and predictability.The authors frame trustworthy-by-design systems and quality control through evaluation and certification as aims.Source: §7 · p. 15

Appendix: Taxonomy

The supplied appendix passages identify a hybrid-AI taxonomy table and include brief glossary-style definitions.

Appendix: Taxonomy

  • The glossary defines a label as a descriptive or identifying word or phrase and a trace as evidence of something past.Source: Appendix: Taxonomy · p. 20
  • The appendix includes a table titled “Hybrid AI taxonomy.”Source: Table 1 · p. 20

Related papers

Loading 2102.11965v2…