Lymphocyte Mimicry Correction via Region-Level Tissue Reasoning and Unbalanced Optimal Transport

Xiang Li; Yuqi Wang; Casey C. Heirman; Jihye Heo; Kyle J. Lafata

Source-linked AI summary

Lymphocyte Mimicry Correction via Region-Level Tissue Reasoning and Unbalanced Optimal Transport

Xiang Li, Yuqi Wang, Casey C. Heirman, Jihye Heo, Kyle J. Lafata

arXiv:2608.17151v1Aug 17, 2026cs.CVcs.LG

TL;DR

Cell-type mimicry makes TIL quantification difficult because morphologically similar cells require tissue context for distinction. Loki-OT transfers region-level MLLM reasoning to cell predictions through Unbalanced Optimal Transport and achieves lower patient-level MAE than PanopTILs on TCGA-BRCA.

Problem
TIL quantification remains difficult because morphologically similar cells and complex tissue environments challenge morphology-based cell classification and region-level models cannot predict individual cells.
Method
Loki-OT uses Unbalanced Optimal Transport to propagate MLLM-derived region-level lymphocyte density priors into globally consistent cell-level assignments.
Results
Loki-OT achieves lower patient-level MAE than the fully supervised PanopTILs model on TCGA-BRCA.
Takeaways & Limitations
Region-level biological priors can complement dense cell annotations when TIL errors arise from tissue context rather than detection.
Takeaways & Limitations
Evaluation is limited to breast-cancer TIL quantification on one held-out cohort, while the asymmetric correction prioritizes precision over recall.

Abstract

from arXiv · show

Cell mimicry arises when different cell types appear morphologically similar. Human pathologists resolve this ambiguity using surrounding tissue context, whereas current vision models either lack contextual reasoning (cell foundation models) or cannot operate at the cell level (pathology MLLMs). We present Loki-OT, which propagates region-level tissue reasoning to individual cell predictions via Unbalanced Optimal Transport, using MLLM-derived density priors as soft guidance for ambiguous cell reassignment. Loki-OT is motivated by the observation that pretrained cell foundation model features already encode discriminative information, including tissue context, but standard cell-level supervision fails to use tissue context effectively. The resulting transport plan is distilled into a lightweight student MLP classifier that learns context-aware decision boundaries within the pretrained feature space. On the independent TCGA-BRCA cohort, Loki-OT achieved lower patient-level MAE than the fully supervised in-domain PanopTILs classifier and improved F1 in epithelium-rich mimicry tissues, using 278 weak region-level MLLM estimates built on a general-domain cell foundation model. Code: https://github.com/xiangli980/Lymphocyte_Mimicry_Correction_via_Loki_OT

1 Introduction

Accurate TIL quantification remains difficult because non-lymphocyte nuclei can mimic lymphocytes in complex tissue, causing systematic overestimation. Loki-OT addresses this limitation by propagating region-level biological supervision to cell-level predictions through Unbalanced Optimal Transport.

Motivation: TIL quantification on H&E-stained tissue is challenging because morphologically similar cells become difficult to distinguish in complex tissue environments.TILs are established prognostic and predictive biomarkers across multiple cancer types.
Problem: Cell mimicry occurs when non-lymphocyte nuclei visually resemble lymphocytes, contributing to systematic overestimation in automated cell classification.Confounding tissue morphology remains an unresolved challenge for automated TIL quantification despite advances in model design and detection performance.
Limitations of existing approaches: Local-morphology models such as HoVer-Net and CellViT can fail when different cell types share similar morphology.These approaches achieve precise localization but rely primarily on local cellular morphology.
Contribution: Loki-OT formulates cell mimicry correction as learning cell-level predictions from region-level biological supervision.The framework treats region-level tissue reasoning as a soft biological constraint and propagates it to individual cell predictions through Unbalanced Optimal Transport.

2 Method

Loki-OT combines CellViT++ morphology and tissue-context features with MLLM-derived region-level lymphocyte density priors. Unbalanced optimal transport corrects ambiguous cell predictions during training, and a two-stage curriculum distills the resulting targets into an inference-only MLP.

Cell and context features: CellViT++ with a SAM-H backbone extracts nuclear morphology features and context embeddings from mean-pooled ViT patch tokens surrounding each cell.The context window spans approximately ±5 tokens, or 80 pixels, around each cell’s bounding box.
Cell and context features: Concatenated morphology and context features make morphologically identical cells separable across different tissue environments.
Region-level priors: An MLLM estimates region-level lymphocyte density as soft supervision during training, while the distilled classifier requires neither MLLM queries nor tissue masks at inference.
Unbalanced optimal transport: UOT combines baseline confidence costs with an epithelium-specific logical penalty and uses the MLLM density as the target marginal to reassign cells selectively.The logical penalty increases when a cell is highly similar to its surrounding epithelium, discouraging lymphocyte labels in epithelium-rich subtypes T1, T3, and T4.
Unbalanced optimal transport: UOT permits deviations from source and target distributions through KL penalties, buffering MLLM uncertainty while preserving confident per-cell predictions unless regional density requires correction.The method uses τ=ρ=3.0 and ϵ=0.1 with entropy regularization for efficient Sinkhorn-Knopp optimization.
Two-stage distillation: A two-stage curriculum first solves regions jointly and then distills cell-level UOT assignments, applying correction only when baseline lymphocyte density exceeds the MLLM estimate by at least 10%.The final teacher target uses the UOT assignment in corrected regions and otherwise falls back to the Stage-1 prediction; Stage 2 combines temperature-softened KL distillation with hard-label cross-entropy.

3 Experiments

Experiments use expert-annotated TIGER training ROIs and an independent TCGA-BRCA test cohort to evaluate context supervision, optimal-transport variants, and Loki-OT’s two-stage training. Results show that MLLM-guided context reduces systematic over-counting, while UOT distillation further lowers MAE through context-based cell reassignment despite reduced recall.

Datasets: The study trains on 135 ROIs from two hospitals and tests on 1,741 zero-overlap TCGA-BRCA ROIs from 124 patients.Training includes ∼80K nuclei and 278 (ROI, subtype) regions; testing includes 64K nuclei, with seven shared tissue subtypes but differing composition.
Context supervision: Context-soft improves every per-patient metric over Lizard, with the largest gain in MAE.The coarse MLLM tissue-level prior directs Stage 1 to suppress density where Lizard’s false positives exceed tissue-level estimates.
Evaluation: Loki-OT versus PanopTILs yields a Wilcoxon-Holm corrected patient-level MAE p=0.031 on the independent TCGA-BRCA cohort.Table 2 reports globally micro-averaged F1, recall, and precision, with patient-level MAE and 1,000-bootstrap 95% confidence intervals.
OT distillation: 32.8% reduction from Context-soft in MAE accompanies lower recall and higher precision after UOT suppresses some true lymphocytes alongside morphological mimics.The transport plan assigns different labels to morphologically similar cells in the same region using tissue context, moving predictions toward MLLM-estimated regional density.
Training stages: Stage 1 removes easy false positives while largely preserving true lymphocytes, whereas Stage 2 enables harder context-dependent mimic correction through region-to-cell supervision.Stage 2 further reduces false positives but introduces additional true-positive suppression.
Teacher variants: OT-Teacher achieves the lowest MAE and highest false-positive reduction but suffers severe true-positive loss, while UOT-Teacher relaxes marginal constraints through KL penalties.Loki-OT uses selective distillation rather than simply replicating UOT-Teacher; the supplied passage truncates the subsequent consequence.

4 Conclusion

Loki-OT transfers region-level tissue reasoning to cell-level predictions through Unbalanced Optimal Transport, correcting lymphocyte mimicry on TCGA-BRCA with weak region-level MLLM estimates. Evaluation was limited to breast-cancer TIL quantification on one held-out cohort, motivating broader validation across priors, prompts, cancers, cell types, and architectures.

Conclusion: Loki-OT transfers region-level tissue reasoning to cell-level predictions through Unbalanced Optimal Transport and corrects lymphocyte mimicry using weak region-level MLLM estimates.On TCGA-BRCA, it achieved lower patient-level MAE than the fully supervised PanopTILs model.
Limitations and future work: The framework’s evaluation was limited to breast-cancer TIL quantification on one held-out cohort.Future work will test robustness to different priors and prompts and validate performance across cancer types, cell types, and multiscale architectures.
Limitations and future work: The asymmetric correction mechanism prioritizes precision over recall and cannot resolve errors when the baseline and prior agree.These constraints motivate robustness evaluation with different priors and prompts.
Data and ethics: The study used only publicly available, de-identified human tissue datasets from the TIGER challenge dataset, with no new human data collected.Institutional review board approval was therefore not required.

A MLLM Prompting Protocol

The protocol queries Claude Sonnet 4.5 once per ROI–tissue block to assess lymphocytes in a blue-outlined region using tissue-specific labels. It separates anatomy-based presence probability from visually assessed density and requests structured JSON output.

Query setup: Claude Sonnet 4.5 is queried once per (ROI, tissue) block in a single-turn multimodal dialog.The tissue label is mapped from the ROI–tissue PNG filename to seven predefined categories, including invasive tumor, tumor stroma, necrosis, and inflamed stroma.
Query setup: Tissue labels are assigned by matching the tissue index parsed from each PNG filename to T1–T7 label descriptions.The mapping includes invasive_tumor, tumor_stroma, in_situ_tumor, benign_glands, necrosis, inflamed_stroma, and other_tissue.
Assessment rules: Presence Probability is based primarily on anatomical compartment definitions, whereas Density, Pattern, and Description combine visual observation with anatomical context.This anatomy-versus-visual split is intended to reduce hallucination by using compartment knowledge for binary presence and visual evidence for quantitative density.
Assessment rules: The multimodal user message asks the model to analyze lymphocytes within the blue-outlined region labeled by its tissue description.Requested fields include presence probability, density category and percentages, rationale, predominant location, overall pattern, and confidence.
Output format: The response is required in structured JSON format.A representative inflamed-stroma response reports Presence Probability 0.95, High density, 52% area density, 58% cell-count density, and Overall confidence 0.85.

B Per-Subtype MAE

Table 4 reports per-ROI MAE by tissue subtype on the TCGA-BRCA test set, with 95% bootstrap confidence intervals. Because several small-n subtypes have wide, overlapping intervals, no single per-subtype winner is highlighted.

Per-subtype MAE: Table 4 reports lymphocytes-per-ROI MAE by tissue subtype on TCGA-BRCA, using 1,000 ROI-level bootstrap resamples for 95% confidence intervals.The small-n subtypes are T4 (n=19), T5 (n=58), and T3 (n=73).
Per-subtype MAE: PanopTILs attains the lowest MAE in sparse-positive epithelial subtypes T1, T3–T5, and T7.The passage attributes this pattern to PanopTILs’ tendency to suppress positive predictions everywhere.
Per-subtype MAE: Wide, overlapping confidence intervals in T4, T5, and T3 prevent declaring a single per-tissue-subtype winner.The table explicitly declines to bold one winner because of uncertainty in these small-n subtypes.