PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM4CKD: Large Language Models for Early Stage Chronic Kidney Disease Screening

Muhammad Ashad Kabir, Sirajam Munira

arXiv:2609.04013v2cs.AIcs.LG

TL;DR

Early CKD screening is difficult when diagnostic resources, stable feature sets, and labeled data are limited. LLM4CKD evaluates zero-shot and few-shot LLM screening with clinically selected structured prompts against ML, DL, tabular foundation-model, and screening-tool baselines. LLMs achieve competitive performance with few examples, whereas supervised and tabular models improve more consistently as training data grow.

  • Problem

    CKD screening is constrained by limited diagnostic resources, heterogeneous feature availability, and conventional methods' reliance on labeled data and stable feature spaces.

    Source: §I · p. 1

  • Method

    LLM4CKD evaluates five LLMs using clinically selected tabular features and structured prompts under zero-shot and few-shot settings, comparing them with ML, DL, tabular foundation-model, screening-tool, and independent-dataset baselines.

    Source: §I · p. 2, A. Dataset Preparation · p. 3, D. Large Language Models Used for… · p. 4

  • Results

    LLMs achieve competitive balanced accuracy with very few examples, while supervised ML and tabular models generally improve more predictably as labeled training data increase.

    Source: B. Low-Data Comparison with ML, DL,… · p. 6

  • Takeaways & Limitations

    LLMs may complement CKD screening when labeled data are limited, whereas conventional and tabular models remain advantageous when sufficient training data and infrastructure are available.

    Source: B. Low-Data Comparison with ML, DL,… · p. 6, §IV · p. 10

    Performance estimates are limited by the relatively small primary cohort, and LLM reliability varies with prompts, feature serialization, model versions, and deployment settings.

    Source: G. Limitations · p. 9

Abstract

from arXiv · show

Early screening of chronic kidney disease (CKD) is critical for timely intervention, yet most machine learning (ML) and deep learning (DL) approaches require labeled data and model training, limiting their use in real-world screening settings. This study evaluates the effectiveness of large language models (LLMs) for CKD screening under zero-shot and few-shot in-context learning settings and compares them with traditional ML and DL methods. We propose a framework that uses clinically selected tabular features and structured prompt templates to enable LLM-based inference without task-specific training. LLM performance is evaluated across multiple prompt styles, feature configurations, and data settings, and compared with standard ML, DL, and tabular foundation model (TFM) baselines, and existing CKD screening tools. The results show that LLMs can achieve competitive performance using only a small number of examples, often matching or outperforming traditional approaches in low-data settings. However, their performance remains model-dependent and less stable as input complexity increases. In contrast, ML, DL, and TFM models show more consistent improvement with larger training data. Overall, the findings highlight a trade-off between data efficiency and stability, suggesting that LLMs may serve as a flexible complementary approach for CKD screening when labeled data are limited. To facilitate further research and reproducibility, the code has been made publicly available at https://github.com/akabircs/LLM4CKD

I. INTRODUCTION

CKD is often silent and difficult to screen at scale where diagnostic resources and stable feature sets are unavailable. LLM4CKD evaluates LLMs as a low-resource, data-efficient alternative alongside established predictive approaches and screening tools.

  • CKD affects approximately 850 million people worldwide, with nearly 80% of cases residing in low- and middle-income countries.Source: §I · p. 1
  • Early-stage CKD can be clinically silent, while timely detection can support interventions that slow disease progression.Source: §I · p. 1
  • Conventional ML and DL methods rely on fixed feature spaces, curated datasets, and stable deployment conditions that real-world screening often lacks.Source: §I · p. 1
  • LLMs offer zero-shot and few-shot in-context learning, reducing reliance on large labeled datasets for heterogeneous screening settings.Source: §I · p. 1
  • LLM4CKD benchmarks LLMs against conventional ML, DL, and tabular foundation-model baselines in low-resource early-stage CKD screening.Source: §I · p. 2
  • The study additionally examines clinical plausibility through feature-ranking comparisons, established CKD tools, and independent-dataset evaluation.Source: §I · p. 2

A. Related Work and Research Gaps

Prior CKD screening research has not systematically established how LLMs perform beyond zero-shot inference, relative to diverse baselines, or across datasets and screening tools. LLM4CKD addresses these gaps through comparative and cross-dataset evaluation.

II. METHODOLOGY

The LLM4CKD methodology combines structured feature processing, harmonized datasets, selected-feature analysis, and comparative evaluation. It uses a Bangladeshi primary cohort and an independent Indian UCI cohort for cross-dataset assessment.

  • The framework serializes structured features, constructs prompts, performs LLM inference, analyzes clinically grounded feature rankings, and evaluates performance against tools and an independent dataset.Source: §II · p. 2

Dataset Preparation

  • The primary evaluation uses a community-based Bangladeshi cohort, while the independent evaluation uses the hospital-based Indian UCI CKD dataset.Source: A. Dataset Preparation · pp. 2–3
  • Dataset-2 contains 400 records, including 250 CKD cases and 150 non-CKD controls, and excludes serum creatinine to reduce label leakage.Source: A. Dataset Preparation · p. 3
  • Feature harmonization standardizes names and value encodings so clinically equivalent variables support consistent prompts and comparable evaluation across datasets.Source: A. Dataset Preparation · p. 3
  • The selected-feature subset comes from prior ML analysis and is mapped to Dataset-2 using variables with equivalent clinical interpretations.Source: A. Dataset Preparation · p. 3

B. Feature Serialization and Prompt Style

LLM4CKD converts harmonized tabular patient records into text while preserving feature semantics, then compares list-based and text-based serialization templates. These templates provide alternative structured representations for prompt-based inference.

C. Prompt Construction and Templates

The framework serializes harmonized clinical features into structured prompts and evaluates instruction- and chat-style inference with list- and text-based templates under zero- and few-shot settings.

C. Prompt Construction and Templates

D. Large Language Models Used for Inference

E. Feature Ranking and Explainability

The study analyzes clinically plausible LLM feature prioritization alongside model evaluation procedures that use controlled prompting, probabilistic outputs, and repeated comparisons.

E. Feature Ranking and Explainability

  • A surrogate ML model and SHAP estimates are used to compare LLM risk-factor prioritization with data-driven feature importance.Source: E. Feature Ranking and Explainability · p. 4
  • Zero-shot models are evaluated without labeled examples, training, or parameter updates, directly on the full dataset.Source: F. Model Evaluation · p. 4
  • Fig. 2 evaluates all- versus selected-feature inputs across prompt styles and templates using balanced accuracy and paired Brier-loss permutation tests.Source: Figure 2 · p. 5, F. Model Evaluation · p. 5
  • Open-weight models use deterministic decoding, while probabilistic classification converts output-token log-probabilities into normalized CKD and non-CKD likelihoods.Source: F. Model Evaluation · p. 5
  • TabPFN and SAINT implementations use fixed model-internal random states where applicable, with source code publicly available for the experimental pipeline.Source: F. Model Evaluation · p. 5

III. RESULTS AND DISCUSSION

Results show that selected features can improve zero-shot LLM screening, while few-shot LLMs are data-efficient but less consistently scalable than supervised tabular models.

A. Zero-Shot Analysis of Feature Selection and Prompt Design

B. Low-Data Comparison with ML, DL, and Tabular Foundation Models

  • Across selected-feature comparisons, LLMs use varying shot counts while ML and DL/TFM baselines train on corresponding labeled subsets.Source: Figure 3 · p. 6

A. Zero-Shot Analysis of Feature Selection and Prompt Design

B. Low-Data Comparison with ML, DL, and Tabular Foundation Models

C. Pairwise Brier Loss Comparison

Pairwise Brier-loss comparisons identify Qwen-3 as the strongest overall LLM, while TabPFN and NODE lead tabular models and gain advantage as labeled data increase.

LLM comparisons

Tabular model comparisons

  • 0.820 balanced accuracy and 0.122 Brier loss make TabPFN the best overall top-model result at 32 context samples.NODE remains strong with balanced accuracy of 0.797 and Brier loss of 0.142.Source: NODE – SAINT · p. 7

Data-size effects

  • TabPFN and NODE benefit more consistently from additional labeled data than LLMs, particularly for probabilistic calibration.Selected-feature LLM prompting remains strong in low-resource settings.Source: NODE – SAINT · pp. 7–8

Conventional ML comparisons

  • MLP remains a competitive supervised baseline, but its Brier loss ranges from 0.160 to 0.175 across training sizes.Its probabilistic calibration is weaker than the strongest LLM and tabular foundation-model baselines.Source: NODE – SAINT · p. 7

D. Feature Ranking and Clinical Plausibility

LLM-derived feature rankings are clinically plausible because they emphasize recognized CKD risk factors, although their weighting differs from the MLP reference and epidemiological patterns.

Reference ranking

Ranking agreement

  • The feature-importance heatmap compares Spearman rank correlations between LLMs and the MLP model.Darker cells indicate stronger agreement.Source: Figure 5 · p. 8

LLM rankings

Comparative plausibility

E. Comparison with CKD Screening Tools

Zero-shot LLMs achieve competitive performance against established CKD screening tools, while independent-dataset results show that feature selection effects vary by model.

Screening-tool comparison

Independent-dataset evaluation

  • Feature selection improves or maintains performance for most models on Dataset-2, but its effect remains model-dependent.Source: F. Independent Dataset Evaluation · p. 9
  • 0.73 to 0.88 balanced accuracy is Llama-3’s gain on Dataset-2 with selected features (p < 0.01).Mistral and GPT-4o-mini improve slightly without statistically significant differences, while Qwen-3 decreases from 0.82 to 0.81 (p < 0.01).Source: F. Independent Dataset Evaluation · p. 9

G. Limitations

The study’s conclusions are constrained by cohort size, model and prompt dependence, feature-selection assumptions, limited cross-dataset evidence, and untested clinical deployment factors.

Study limitations

  • The relatively small primary cohort may limit performance-estimate stability, despite repeated runs and independent-dataset evaluation.Larger and more diverse cohorts are needed for stronger generalization claims.Source: G. Limitations · p. 9
  • LLM reliability depends on prompt design, feature serialization, model version, and token-level log-probability availability.Results may vary across future model releases and deployment settings.Source: G. Limitations · p. 9
  • The selected feature subset was derived from prior ML analysis and may not be optimal across populations or clinical contexts.Source: G. Limitations · p. 9
  • Independent-dataset results provide preliminary cross-dataset evidence rather than definitive clinical validation.The datasets differ in sampling setting, feature availability, and case-control composition.Source: G. Limitations · p. 9, §IV · p. 10
  • Robustness to patient-level missing features, heterogeneous inputs, deployment efficiency, clinical safety, cost, and prospective validation was not directly evaluated.These factors require evaluation before clinical use.Source: G. Limitations · p. 9

Related papers

Loading 2609.04013v2…