PaperlayerView exact revision on arXiv

Source-linked AI summary

LLMs as Post-hoc Auditors of Physiological Plausibility in Symbolic Regression: A Clinician-Evaluated Case Study

Jorge López-Varela, J. Ignacio Hidalgo, José-Manuel Muñoz, Omar Costilla-Reyes, Esther Maqueda, Jesus Moreno-Fernandez, Tomás González-Vidal, J. Manuel Velasco, Oscar Garnica

arXiv:2609.11431v1cs.AI

TL;DR

Symbolic-regression equations can be accurate yet difficult to interpret or physiologically justify, motivating post-hoc LLM auditing with clinician oversight. The study evaluates four body-fat models using three LLMs and clinician assessment, finding comparative rankings more useful than isolated equation interpretations while retaining important reliability limits.

  • Problem

    Symbolic-regression models may be complex or scientifically inconsistent, creating a need to assess interpretability and physiological plausibility beyond predictive accuracy.

  • Method

    The study applies three LLMs to interpret and rank four grammar-guided symbolic-regression models, repeating each prompt three times and evaluating outputs with three clinicians.

  • Results

    Clinicians rated comparative model rankings more favorably than isolated equation interpretations across the three LLMs.

  • Takeaways & Limitations

    LLMs are most immediately useful as comparative auditing filters for selecting similarly accurate models, rather than as autonomous validators.

  • Takeaways & Limitations

    The audited LLMs produced physiologically and mathematically questionable interpretations, including unsupported claims about body-fat relationships and predictors.

Abstract

from arXiv · show

Genetic Programming and its variants, such as grammatical evolution, are widely used in Symbolic Regression to derive mathematical expressions from multivariate data. In addition to predictive accuracy, models are appreciated for their potential to provide interpretability, offering explicit equations that relate input variables to outcomes. However, achieving interpretability and plausibility remains challenging, as evolved models may be complex or scientifically inconsistent. In this study, we explore whether Large Language Models, can assist in improving the explainability of Symbolic Regression models generated by evolutionary computation methods. Building upon our previous work on estimating body fat percentage using grammar-based Genetic Programming , we investigate the use of LLMs as post-processing tools to analyze and rank evolved expressions according to their interpretability and medical plausibility. Four symbolic expressions are analysed by three LLMs over three repeated runs, and the resulting interpretations and rankings are assessed by a panel of three clinicians. Across the three LLMs, comparative model-ranking outputs received more favorable clinician assessments than isolated term-level interpretations. However, the LLMs also produced physiologically and mathematically questionable explanations, indicating that they are better suited to comparative auditing under expert oversight than to autonomous validation.\blfootnote{The present work is an extended version of a paper submitted into a journal.

1 Introduction

Symbolic regression offers explicit equations, but evolved expressions can be difficult to interpret and physiologically implausible. This study examines LLMs as post-processing tools for evaluating and selecting such models with medical expert input.

  • Interpretability and plausibility: Symbolic regression can support interpretability because it expresses relationships among problem variables as explicit mathematical equations.However, equation complexity can make direct interpretation difficult.
  • Interpretability and plausibility: Physiological plausibility requires alignment with scientific evidence, whereas interpretable equations may still contain unsupported relationships or inconsistent units.These problems are especially consequential in healthcare, where users may lack domain knowledge or mathematical expertise.
  • Motivation and contribution: LLMs are proposed to analyze symbolic-regression solutions for interpretability and plausibility, then select among models with similar predictive quality.The work extends grammar-guided GP models for body-fat estimation, where prior selection relied only on RMSE.
  • Study design: The paper builds on prior comparisons of GE, CFG-GP, and DSGE models for body-fat estimation, where final selection used the lowest mean square error.The present work adds interpretability and plausibility analysis to that selection process.
  • Study design: The study combines LLM analyses with evaluation by three physicians, who assess both the candidate solutions and the LLMs’ interpretations.The authors report that this integration may support more interpretable and plausible symbolic models.

2 Related Work

Related work has addressed symbolic-regression complexity and LLMs within evolutionary workflows, but post-hoc auditing of symbolic models’ semantic and biomedical plausibility remains largely unexplored.

  • Complexity and interpretability: GP bloat increases syntactic-tree complexity without proportional accuracy gains, compromising human interpretability.Existing responses include parsimony pressure, multi-objective optimization, and age-fitness strategies.
  • LLMs in evolutionary computation: Prior LLM–evolutionary-computation paradigms mainly use LLMs to generate or enhance equation search for accuracy or efficiency rather than audit semantic validity.The proposed work instead treats auditing as a separate post-processing task.
  • Explainability gap: Model-agnostic tools such as LIME and SHAP identify influential variables but do not expose the functional structure needed to assess physiological plausibility.The paper positions symbolic expressions and post-hoc LLM auditing as addressing this scientific-discovery limitation.
  • Research gap: The paper targets a gap in using LLMs as post-hoc evaluators of symbolic models’ semantic and biomedical plausibility.This differs from using LLMs to discover or optimize equations within the evolutionary process.

3 Large Language Models and Prompt Engineering

The paper describes LLM capabilities, hallucination risks, model-interpretation prompting, and a reasoning-oriented prompt design tailored to medical symbolic expressions.

  • 3 Large Language Models and Prompt Engineering: LLMs generate natural-language outputs from large-scale training, but their text can be false when required information is absent from training data.The paper therefore treats hallucination as an inherent possibility rather than a rare exception.
  • 3.1 Using LLMs for model interpretation: The study uses LLMs to translate symbolic-regression expressions into natural-language interpretations that medical experts can understand without reading equations directly.This approach is motivated partly by the scarcity of domain-specific fine-tuning datasets.
  • 3.1 Using LLMs for model interpretation: Prompt engineering is used because task specificity and expression complexity require carefully structured instructions.The authors emphasize that prompts should provide precise and understandable information.
  • 3.1 Using LLMs for model interpretation: Reasoning prompting is selected for mathematical expressions and medically specific tasks, while reasoning models are motivated by evidence of stronger performance on reasoning-oriented clinical tasks.The paper acknowledges that these advanced models require more time and resources.
  • 3.2 How to engineer proper prompts for this task: Few-shot prompting is described as using examples to shape outputs, but the study leaves it for future work because suitable resources are unavailable.The implemented approach instead relies on reasoning-prompting techniques.
  • 3.2 How to engineer proper prompts for this task: The prompt assigns the LLM the role of a medical mathematical-expression interpreter and directs it to produce outputs for medical experts rather than mathematicians.This is intended to improve understandability while avoiding unnecessary mathematical elaboration.
  • 3.2 How to engineer proper prompts for this task: The prompt should explain the expression’s variables, origin, and relevant context, then specify a precise task and desired output structure.The authors prefer precise instructions over unnecessary prompt complexity because input-output examples are unavailable.

4 Methodology

The methodology combines previously generated grammar-guided symbolic-regression models with filtering, LLM-based plausibility and explainability auditing, and medical validation using NHANES body-composition data.

  • Methodology: The study evaluates four previously generated symbolic-regression models for total body-fat percentage and applies a four-phase workflow from candidate generation through expert validation.The phases are candidate selection, filtering and simplification, multi-perspective LLM auditing, and human medical validation.
  • Data Selection and Preprocessing: The preprocessing retains adults aged 18–59 with complete anthropometric and DXA measurements, excludes pregnant females and records with missing demographic information, and produces 2,403 individuals.The split is an 80%/20% random stratified train/test partition.
  • Methodology: The workflow diagram distinguishes prior-work candidate models, a new two-stage LLM evaluation protocol, and expert medical validation.These stages operationalize the paper’s post-hoc audit rather than replacing symbolic-regression generation.
  • Data Selection and Preprocessing: The models use NHANES 2017–2018 body-composition data, with DXA-measured total body-fat percentage as the target and nine anthropometric or demographic inputs.The dataset is described as nationally representative health and nutrition data collected by the CDC.
  • Grammar-Guided GP Models: The study reports candidate-model performance and hyperparameter configurations in separate tables, while complete grammars and convergence details are referenced from prior work.The cited tables cover candidate metrics and configurations for the two grammar-guided GP variants.

4.3 Filtering and Simplification

The study filters previously evolved symbolic-regression models by accuracy before asking LLMs to assess interpretability and plausibility. Selected expressions had competitive performance around R2 ≈0.85.

  • 4.3 Filtering and Simplification: RMSE served as the fundamental accuracy and fitness metric, with lower values indicating better fit and guiding candidate selection.The evolutionary process minimized differences between predicted and actual values.
  • 4.3 Filtering and Simplification: R2 ≈0.85 was used to identify expressions with competitive performance for LLM interpretation.The selected models came from earlier work in which accuracy was prioritized.

Model CFG-M9

The CFG-M9 section presents symbolic expressions built from anthropometric, age, and sex-related variables. These expressions use simple arithmetic combinations, including products and squared terms.

  • Model CFG-M9: The displayed candidates use compact arithmetic structures but also include nonlinear terms such as BMXWAIST2 and products between measurements.Examples include 1000 · BMXWAIST + 3 · BMXWAIST2 and BMXHT · BMXWAIST · RIAGENDR.
  • Model CFG-M9: Several expressions incorporate age or sex-related variables, including RIDAGEYR and RIAGENDR.These variables appear both alone and within additive or multiplicative expressions.

4.4 Multi-Perspective Plausibility and Interpretability Audit

The audit uses multiple LLM types and repeated runs to interpret and rank symbolic models across clinical and structural criteria. It separates individual explanation from comparative evaluation and tracks output stability.

  • 4.4 Multi-Perspective Plausibility and Interpretability Audit: The protocol evaluates Gemma3, DeepSeek-R1, and GPT-5.1 Thinking as non-reasoning or reasoning-oriented models with different inference-cost profiles.Gemma3 provides a deterministic baseline, while DeepSeek-R1 and GPT-5.1 Thinking represent reasoning models.
  • 4.4 Multi-Perspective Plausibility and Interpretability Audit: Individual interpretation decomposes each expression, explains its mathematics, supplies physiological justifications, and synthesizes the result for clinicians.This task was applied to Gemma3 and DeepSeek-R1.
  • 4.4 Multi-Perspective Plausibility and Interpretability Audit: Comparative ranking scores interpretability, physiological plausibility, simplicity, and practical clinical use on a 1-to-5 scale with justifications and an overall recommendation.The ranking is intended to compare models beyond data-based performance alone.
  • 4.4 Multi-Perspective Plausibility and Interpretability Audit: Each model was run three times, using repetition to confirm deterministic baseline behavior or measure run-to-run variability in reasoning models.The first run was submitted for clinical evaluation, while the other runs characterized ranking stability.

Evaluating non-deterministic LLMs:

The evaluation uses structured prompts to standardize LLM interpretations and comparisons for both technical and clinical audiences. Its sign-based explanation rule is explicitly limited because terms can occur in nonlinear subexpressions.

  • Evaluating non-deterministic LLMs:: The prompt assigns the LLM a statistician-and-medical-knowledge role that bridges engineers and physicians.The context explicitly targets audiences with different mathematical and medical backgrounds.
  • Evaluating non-deterministic LLMs:: The requested output interprets four models through important subexpressions, then provides a brief summary in simple medical language.The format is designed for physicians without advanced mathematical knowledge.
  • Evaluating non-deterministic LLMs:: The comparison instructions require consistent formatting, criterion-by-criterion rankings, detailed reasoning, and rankings of all four models.The reasoning is intended to be understandable to readers without advanced mathematical knowledge.
  • Evaluating non-deterministic LLMs:: The sign heuristic describes individual terms as increasing or decreasing the outcome, but it does not establish a variable’s global marginal effect in the full nonlinear model.Variables may occur in multiple nonlinear subexpressions.
  • Evaluating non-deterministic LLMs:: GPT-5.1 Thinking used individual interpretation prompts rather than a separate comparison prompt, and those individual interpretations were not discussed.This choice was made to preserve structural comparability for benchmarking outputs.

4.5 Analysis Methodology

The methodology evaluates LLM-generated interpretations and model comparisons through clinician ratings, focusing on consistency, expert alignment, and relationships between model characteristics and perceived plausibility.

  • The core dimensions are agreement among LLM rankings and rationales, alignment with physicians, and links between model properties and perceived plausibility.
  • The analysis compares LLM rankings across runs and summarizes clinician ratings descriptively alongside qualitative comments.
  • Three clinicians rated LLM outputs using task-specific statements on a 1-to-5 agreement scale.They evaluated individual interpretations, model comparisons, and final conclusions.
  • Clinicians assessed whether interpretations were plausible, comprehensible, relevant, and useful beyond what physicians could detect at a glance.
  • For model comparisons, clinicians judged ranking appropriateness, reasoning soundness, relevance, added information, and usefulness for comparing models.

5 Evaluation and Comparison generated by Gemma 3

Gemma3 interprets four symbolic body-fat models and ranks them by interpretability, plausibility, simplicity, and practical utility. Clinician assessments favor its comparative ranking task over several individual explanations, while identifying recurring physiological errors and omissions.

  • 5.1 Individual Interpretation: Gemma3 describes DSGE-M21 using body dimensions and central-obesity relationships, but clinicians find the interpretation implausible and difficult to comprehend.
  • 5.1 Individual Interpretation: Gemma3 explains DSGE-M18 in natural language without evaluating the plausibility of its terms.
  • 5.1 Individual Interpretation: Clinicians identify recurring errors involving sex differences, height, waist and arm circumference, and unexamined or misleading body-fat relationships.They also note that prompting may contribute to components being left unanalyzed.
  • 5.2 Individual Interpretation: CFG-M9 is characterized as highly complex and nonlinear, while CFG-M3 emphasizes interactions among body size, proportions, and gender.Clinicians broadly agree on the CFG-M9 assessment, whereas CFG-M3 receives the lowest overall interpretation assessment.
  • 5.3 General Rankings and Conclusions of Gemma3: DSGE-M21 is ranked the best overall because it balances physiological plausibility, interpretability, and practical utility, with DSGE-M18 a close second.The analysis notes that DSGE-M18’s greater complexity does not necessarily produce a significant accuracy improvement.
  • 5.3 General Rankings and Conclusions of Gemma3: Clinician assessments of Gemma3’s comparison task are clearly higher than assessments of its individual interpretations, especially for ranking appropriateness and reasoning soundness.Agreement is weaker for relevance, and only three evaluators limit claims about small differences between dimensions.

6 Evaluation and Comparison generated by DeepSeek-R1

DeepSeek-R1 produced richer physiological interpretations and favored DSGE-M21 overall, but clinicians identified complexity, questionable terms, and run-to-run ranking variability.

  • 6.1 Individual Interpretations: DeepSeek-R1 offered deeper, physiologically oriented explanations than Gemma3, improving the natural-language account for readers without mathematical expertise.It also inferred possible model intentions, such as distinguishing muscular from fatty arms.
  • 6.1 Individual Interpretations: Clinicians rated DeepSeek-R1 higher in physiological plausibility but lower in understandability than Gemma3 for one comparison.In another evaluation, it outperformed Gemma3 on plausibility and two statements, while lacking understandability.
  • 6.1 Individual Interpretations: Clinicians found some interpretations coherent but criticized difficult terms, excessive complexity, and explanations that were hard to follow.One evaluation described the interpretation as possibly the worst in the task, with minimum scores and high inconsistency.
  • 6.2 Comparison of Models: DeepSeek-R1 favored waist-to-hip relationships as clinically recognized while distrusting arm- and leg-based terms lacking clear physiological support.This criterion lowered the plausibility of DSGE-M18 relative to Gemma3’s assessment.
  • 6.3 General Rankings and Conclusions of DeepSeek-R1: DSGE-M21 was ranked best across fields, while DeepSeek-R1 associated CFG-GP models with greater complexity and overfitting risk.This preserved the broader ordering in which DSGE models were judged superior overall.

7 Comparison of the models generated by GPT-5.1 Thinking

GPT-5.1 Thinking compares four symbolic-regression models across interpretability, physiological plausibility, simplicity, and practical use. It generally favors DSGE-M21, while clinicians particularly value its explicit ranking criteria and justifications.

  • Physiological plausibility: The models receive mixed plausibility assessments: DSGE-M21 retains coherent waist, hip, height, and sex effects but includes a counter-intuitive negative weight interaction, while CFG models contain several physiologically dubious weight or limb effects.GPT-5.1 Thinking reports DSGE models as less plausible and CFG-GP models as more plausible than in the previous experiment, although individual explanations remain critical.
  • Simplicity: The comparison favors simpler models, ranking DSGE models ahead of CFG-GP models, while CFG-M3 remains somewhat more regular than CFG-M9 despite still having many terms.The simplicity criterion is inferred from the preference for fewer and simpler terms and interactions.
  • Practical use: DSGE-M18 has good practical utility as a compact bridge between engineers and clinicians, whereas CFG-M9 and CFG-M3 have low or moderate-to-low utility because their core mechanics are difficult to defend or explain.GPT-5.1 Thinking treats practical use as a combination of interpretability, simplicity, plausibility, and implementation considerations.
  • Interpretability: DSGE-M21 is compact and reasonably interpretable, whereas CFG-M9 and CFG-M3 are harder to explain because of numerous interactions, ratios, and higher-order terms.DSGE-M18 is also compact but is judged physiologically problematic, especially in its treatment of large hips and weight.
  • Clinical assessment: Clinicians favor GPT-5.1 Thinking’s comparison because it makes evaluation criteria explicit and justifies ranking decisions, even though its ordering is not markedly different from prior models.The clinicians validate the reasoning behind the rankings without assigning low scores to its comparison quality.
  • General rankings and conclusions: GPT-5.1 Thinking keeps DSGE-M21 as the leading model in most dimensions, diverging mainly in physiological plausibility.The ranking variability concentrates in middle positions, while the leading position remains comparatively stable.

8 Conclusions

The study evaluates LLMs as post-hoc auditors of symbolic-regression models with comparable predictive quality. Comparative rankings are judged more useful than isolated equation interpretations, but physiological errors and ranking variability require clinician oversight.

  • Contribution: LLMs can act as a qualitative auditing layer that complements quantitative performance metrics by surfacing models that appear interpretable and physiologically plausible.The authors do not present this role as a replacement for clinical judgment.
  • Main finding: Clinicians rated comparative model rankings more favorably than term-by-term interpretations of individual equations across the three language models.The ranking task was considered sounder and more useful for weighing candidate expressions against explicit criteria.
  • Limitations: Gemma3 repeatedly produced the physiologically incorrect claim that men tend to have higher body-fat percentages than women, despite fluent presentation.Clinicians immediately flagged this error, illustrating why expert review remains necessary.
  • Reliability: Reasoning-model rankings varied across repeated runs mainly in middle positions, while the top position remained stable and the deterministic baseline reproduced the same ordering.Clinical assessments of the three models were nevertheless largely indistinguishable.
  • Scope of use: The results do not support autonomous biomedical validation; physiological-plausibility judgments require mathematical verification and clinician oversight.LLMs appear more suitable for organizing comparative assessments of structural properties such as simplicity and interpretability.
  • Future work: Future work should broaden clinical evaluation and provide LLMs with precomputed analytical information about variable effects to improve plausibility judgments.The proposed information includes the direction and magnitude of each variable’s effect.
Loading 2609.11431v1…