PaperlayerView exact revision on arXiv

Source-linked AI summary

HyDI: A hybrid Deep Learning-Inductive Logic Programming ensemble for multi-label classification

Simon Flügel, Till Mossakowski

arXiv:2609.37740v1cs.LGcs.LO

TL;DR

Deep Learning models can be difficult to explain, and hierarchical multi-label classification also faces increasing class rarity deeper in the hierarchy. HyDI combines a DL classifier with ILP-generated rules for leaf classes and applies the ensemble to ChEBI. The DL baseline performs slightly better than HyDI on both reported F1 aggregates, while HyDI provides rules and explanations for selected classes.

  • Problem

    Deep Learning models are difficult to explain, and hierarchical multi-label classification faces class imbalance as classes become rarer deeper in the hierarchy.

    Source: §1 · p. 1

  • Method

    HyDI combines a multi-label DL classifier with ILP-generated classifiers for leaf classes, selecting between them by class.

    Source: §1 · p. 1

  • Results

    The DL model performs slightly better than HyDI by 0.6% for both micro- and macro-F1 scores.

    Source: §4.2 · p. 12

  • Takeaways & Limitations

    For 314 of 825 ChEBI leaf classes, HyDI replaces DL predictions with ILP rules, some of which are chemically meaningful.

    Source: §5 · p. 13

    Validation and test sets can be very small for leaf classes, making F1 differences of 50% or more between them statistically plausible.

    Source: §4.2 · p. 13

Abstract

from arXiv · show

While attaining remarkable results for many applications, Deep Learning models are notoriously difficult to explain. This work introduces HyDI, a hybrid ensemble architecture for hierarchical multi-label classification. It combines a Deep Learning (DL) model with rule-based classifiers generated by Inductive Logic Programming (ILP). For leaf classes of the label hierarchy, the rule-based classifiers replace the DL model, leading to more transparent classification results. HyDI is applied to the Chemical Entities of Biological Interest (ChEBI) ontology, providing ILP-generated rules for 314 classes. For these classes, HyDI can generate global explanations as well as local explanations that combine visual and text-based descriptions.

1 Introduction

Deep Learning can be difficult to explain and hierarchical multi-label classification faces increasing class rarity deeper in the hierarchy. HyDI combines a DL model with ILP-generated leaf classifiers and applies the ensemble to ChEBI.

1 Introduction

  • HMLC assigns multiple labels arranged in a hierarchy, where classes farther down the hierarchy have fewer associated objects.The resulting class imbalance is identified as a major challenge.Source: §1 · p. 1
  • HyDI combines a multi-label DL classifier for all classes, ILP-generated binary classifiers for individual leaf classes, and an aggregator that chooses which model predicts each class.The goal is to maintain DL classification performance while making results more explainable.Source: §1 · pp. 1–2
  • HyDI is applied to the Chemical Entities of Biological Interest ontology and provides global and local explanations combining natural-language text with visualizations.Source: §1 · p. 2

2 Related work

Prior work combines ILP and Deep Learning in post-hoc explanation or manually specified chemical rules, while this work learns rules from molecule data and integrates them into classification. HyDI also connects its rule explanations to chemical visualizations.

2.1 ILP explainability

  • Unlike post-hoc approaches, this work learns ILP rules directly from input data and uses them in an ensemble to explain classes predicted by those rules.The authors favor this direct explanation for their structured organic-chemistry use case, while noting that it does not reveal the DL model’s inner workings.Source: §2.1 · p. 2
  • For chemical compounds, HyDI automates visual explanations by highlighting atoms relevant to an ILP rule, extending a technique previously shown as a proof of concept.Source: §2.1 · p. 3

2.2 Multi-label classification for ChEBI

  • Existing ChEBI symbolic approaches use manually constructed rules, motivating this work’s ILP-based automation of rule construction from input data and local explanations.Source: §2.2 · p. 3
  • Unlike Chemical Classifier Programs, which use natural-language ChEBI definitions, this work learns rules from molecule samples.Source: §2.2 · p. 3
  • HyDI differs from Chebifier 2 by assigning each label to one predictor, either a DL model or an ILP rule, rather than aggregating weighted predictions.The paper argues this makes the result attributable to a single model.Source: §2.2 · p. 4

3 Methodology

The methodology uses Popper to learn ILP hypotheses and extends its cost function to favor false negatives or false positives. HyDI assigns DL prediction to non-leaf classes and chooses among DL and ILP predictions for leaf classes.

3.1 ILP

  • Popper learns hypotheses by iteratively testing them against examples and background knowledge, then adding ASP constraints based on misclassifications to guide later hypotheses.The selected hypothesis optimizes a minimal-description-length cost function.Source: §3.1 · p. 4
  • The ILP cost function combines hypothesis size with false-negative and false-positive counts, and this work adds weights to bias their relative costs.The three evaluation settings are ILP-FP (α = 1, β = 3), ILP-FN (α = 3, β = 1), and ILP-REG (α = 1, β = 1).Source: §3.1 · p. 4

3 Methodology

  • Non-leaf classes are predicted by the DL model, while leaf classes may be predicted by the DL model or an ILP rule depending on performance.Source: Figure 1 · p. 5

3.1 ILP

  • The three cost settings are intended to produce a broader range of hypotheses from which the ensemble can select.Source: §3.1 · p. 5

3.2 The HyDI ensemble

HyDI uses a native multi-label DL classifier for non-leaf classes and selects a predictor for each leaf class using validation F1. The selection accounts for the dependence of leaf predictions on their superclass predictions.

3.2 The HyDI ensemble

  • A native multi-label DL classifier predicts non-leaf classes, avoiding successive independent classifiers whose errors could compound along deep or multiply inherited hierarchies.The paper illustrates that independent classifiers with recall 0.95 would yield about 0.60 recall across ten successive levels.Source: §3.2 · p. 5
  • An ILP leaf rule is evaluated conditional on correct DL predictions for its direct superclasses, so its effective performance depends on those superclass predictions.Source: §3.2 · pp. 5–6
  • For leaf classes, HyDI compares the DL prediction with ILP predictions using per-class validation F1 and selects the highest-scoring predictor.ILP wins ties against DL; ties among ILP rules are broken using the mean F1 across rules from each ILP configuration.Source: §3.2 · p. 6

3.3 Application to ChEBI

The ChEBI25-3STAR dataset frames molecule classification as assigning molecules their ontology superclasses, while the augmented GAT supplies the Deep Learning model. The dataset uses human-curated molecule classes and a shared train/validation/test split.

ChEBI25-3STAR dataset

  • ChEBI classification assigns molecules all their superclasses, including indirect ones, in the hierarchy.The ontology ranges from broad concepts to specific classes and individual molecules.Source: ChEBI25-3STAR dataset · p. 6
  • The ChEBI25-3STAR dataset contains 1,808 labels, including 825 leaf classes, and 51,670 samples from human-curated molecule classes.The DL and ILP models use the same 80/10/10 train/validation/test split.Source: ChEBI25-3STAR dataset · p. 6

DL model

  • The augmented GAT represents molecules across connected atom, motif, and global-node levels initialized with chemical properties.It performs four message-passing steps, sums atom representations, and classifies with a fully connected head.Source: DL model · pp. 6–7, Figure 2 · p. 7

ILP system

The ILP system represents molecules as logic programs and learns a rule for each label class from positive and negative examples. Negative samples are selected from nearby classes in the label taxonomy.

ILP system

  • Molecules are encoded as logic programs using predicates for bond-, atom-, and molecule-level properties.The methanol example assigns the molecule and its carbon and oxygen atoms separate identifiers.Source: ILP system · p. 7, Table 1 · p. 8
  • Each class uses up to 200 positive and negative training samples, with negatives selected by breadth-first search through nearby taxonomy classes.The search starts with sibling classes, expands outward if fewer than 25 negatives are available, and stops at 200.Source: ILP system · pp. 7–8
  • ILP seeks a rule that separates positive from negative examples; the primary-alcohol example infers methanol as positive but not propan-2-ol.The illustrated rule uses carbon and oxygen atoms, a single bond, and at least two hydrogens on the carbon.Source: ILP system · p. 8

3.4 Global explanations

HyDI translates symbolic ILP rules into natural-language explanations by incorporating superclass conditions, ChEBI class names, and readable predicate expressions. The examples show both conjunctions within rules and alternatives across multiple clauses.

3.4 Global explanations

  • An algorithm translates ILP rules into natural language by adding implicit superclass conditions, retrieving ChEBI class names, and replacing predicates with natural-language expressions.The authors say this supports discussion of rules with experts for verification and possible improvement.Source: §3.4 · pp. 8–9
  • The pyrazole example renders a rule as a diazole superclass condition plus two nitrogen atoms joined by an aromatic bond.The rule's variables are reordered when needed to keep literals concerning the same variable close together.Source: §3.4 · pp. 8–9
  • Rules with several clauses become an “or” condition, as in the alkali-metal example listing lithium, sodium, or potassium atoms.The displayed clauses each require the molecule to have an atom of the specified element.Source: §3.4 · p. 9

3.5 Local explanations

Local explanations use examples to connect an ILP rule with the atoms in a molecule. For N-methylpyrazole, xclingo traces which atoms ground the rule's variables.

3.5 Local explanations

  • Local explanations offer an example-based way to understand ILP rules.The authors describe this as a more intuitive approach to understanding the rules.Source: §3.5 · p. 9
  • For a given molecule, xclingo traces which atoms ground the rule's variables.The paper names xclingo1 as the tool used for this tracing.Source: §3.5 · p. 9
  • The atom grounding connects a global rule explanation to the molecular graph, illustrated for pyrazoles and N-methylpyrazole.The example is the local explanation shown in Figure 4.Source: §3.5 · p. 9

3.6 Experimental configuration

The augmented GAT model is trained on ChEBI25-3STAR for 200 epochs. ILP rules are learned for each leaf class and cost-function configuration using Popper with bounded runtime and rule complexity.

3.6 Experimental configuration

  • The augmented GAT model trains on ChEBI25-3STAR for 200 epochs with a 10^-3 learning rate, binary cross-entropy loss, and batch size 512.Training used an NVIDIA A100 GPU and the ChEB-AI Graph library.Source: §3.6 · p. 10
  • Popper learns an ILP rule for each leaf class and each of the ILP-FP, ILP-FN, and ILP-REG configurations, with a 60-second timeout.The setup uses the NuWLS anytime solver and limits rules to 6 variables, 2 clauses, and 8 body literals.Source: §3.6 · p. 10
  • The Popper implementation uses a forked version with a parametrised cost function and the NuWLS anytime solver enabled.Rules are constrained to at most 6 variables, 2 clauses, and 8 body literals.Source: §3.6 · p. 10

4 Results

The ChEBI25-3STAR results compare ILP rules and the HyDI ensemble with the augmented GAT baseline. HyDI is close to the baseline overall, while rule performance varies by configuration and selected subset.

4.1 ILP rule analysis

  • Across 825 leaf classes, no configuration produced a rule for 149 classes, while one or two configurations produced rules for 168.For 508 classes, all configurations produced rules.Source: §4.1 · p. 10, Table 2 · p. 11
  • ILP-FN produced 666 rules versus 520 for ILP-FP; ILP-FP macro F1 was 4% / 4.1% lower than ILP-FN / ILP-REG overall.On shared classes, the macro-F1 gaps widened to 8.8% / 7.4% versus ILP-FN / ILP-REG.Source: §4.1 · p. 10, Table 2 · p. 11
  • On shared classes, ILP-FP had the highest precision but recall 21% lower than ILP-FN; their cost functions favor fewer false positives and false negatives, respectively.ILP-FN’s stronger false-negative weighting encourages fewer false negatives and can allow more false positives, while ILP-FP’s weighting works in the opposite direction.Source: §4.1 · p. 11

4.2 Ensemble performance

  • The DL model performs 0.6% better than HyDI on both micro- and macro-F1; 314 leaf classes use ILP rules and 511 use DL.Table 3 reports performance for the DL model and HyDI across all classes.Source: Table 3 · p. 12, §4.2 · p. 12
  • ILP-REG scores are -19.9% / -13.2% versus ILP-FN / ILP-FP, and -19.3% versus DL on the ILP-REG-predicted subset.The passage notes that validation-set performance did not generalize to the test set for this subset.Source: Table 4 · p. 12, §4.2 · p. 12

4.1 ILP rule analysis

  • The example 6β-hydroxy steroid rule is overly specific when it requires a neighboring atom with 2 hydrogen atoms, yielding precision 1 and recall 0.67.The example rule includes steroid position and stereochemistry along with the neighboring-atom hydrogen condition.Source: §4.1 · pp. 11–12

4.2 Ensemble performance

  • Small leaf-class splits can contain only 2 or 3 positive examples, making validation-to-test F1 differences of 50% or more statistically plausible.Some classes are not structurally defined, and others require features—such as an exact carbon-atom count—not feasible in the current ILP setting.Source: §4.2 · p. 13

5 Conclusions

HyDI combines upper-layer DL predictions with leaf-level ILP rules, replacing the DL model for 314 of 825 ChEBI leaf classes. The evaluation reports a 0.6% macro-F1 loss alongside enhanced interpretability, while the authors identify rule-quality issues and future extensions.

5 Conclusions

  • 314 of 825 leaf classes use ILP rules, with a 0.6% loss in macro F1 and enhanced interpretability through natural-language rules and molecule visualizations.The DL model predicts upper hierarchy layers; ILP rules replace it for selected leaf classes.Source: §5 · p. 13
  • Some learned rules are chemically meaningful, while others show signs of overfitting or target concepts that available predicates cannot formalise.Source: §5 · p. 13
  • Future work will add functional groups and ring structures, predicate invention and negation, and iterative rule refinement with human experts.The proposed explanations are intended to help experts identify rule issues and make changes that can then be automatically verified.Source: §5 · p. 13
Loading 2609.37740v1…