PaperlayerView exact revision on arXiv

Source-linked AI summary

Language-encoded network topology enables large language models to reason about complex networks

Ucchwas Talukder Utsha, Sakib Mostafa, James Zou, Md Tauhidul Islam

arXiv:2609.03229v1cs.LG

TL;DR

LLMs often struggle to answer structural questions about networks presented as connections or measurements because topology must be inferred from text. BioGlyph compiles topology into interpretable structural roles, evidence, and consequences without changing the network or LLM, improving structural reasoning across domains and revealing biological associations. Its advantages are strongest in dense, community-structured networks, while sparse networks and context limits remain boundaries.

  • Problem

    LLMs often struggle with network questions requiring structural reasoning when graphs are presented as edge lists, sentences, or measurement tables.

  • Method

    BioGlyph uses network-analysis algorithms and fixed rules to describe nodes and interactions through interpretable structural roles, supporting evidence, and structural consequences, leaving the network and LLM unchanged.

  • Results

    Across networks from multiple domains, BioGlyph improved structural reasoning relative to edge lists, adjacency sentences, raw measurements, and unsupervised graph representations, particularly in dense and community-structured networks.

  • Takeaways & Limitations

    In yeast, cross-community connectors, community cores, and hubs were enriched for essential proteins, while peripheral proteins were depleted, linking named structural roles to biological organization.

  • Takeaways & Limitations

    Performance depends on network density and prompt budget: alternatives can perform comparably in sparse networks, and BioGlyph descriptions occasionally exceed the context limit.

Abstract

from arXiv · show

Networks describe systems in biology and beyond, from protein interactions and social relationships to power grids and citation records. Reasoning about such systems requires understanding their structure: which elements are central, which connections bridge separate communities, and how it changes when elements are removed. Although large language models (LLMs) excel at natural language, they struggle with such questions when networks are given as edge lists, sentences or measurement tables, because their structural meaning must be inferred. Here we introduce BioGlyph, which compiles network topology into an interpretable and transferable language of structural roles. BioGlyph combines graph partitioning and structural measurements to identify roles such as hubs, community cores and cross-community connectors, and fixed rules to translate them into a universal vocabulary. The representation describes each element through its structural role, supporting evidence and semantic consequences, leaving both the network and the LLM unchanged. Across twenty networks spanning five domains, BioGlyph substantially improves open LLMs' ability to answer structural reasoning questions, outperforming edge-based, numerical and learned representations by up to 26 percentage points in system accuracy. Ablations show that the gain comes from explicitly encoding structural roles in semantically interpretable terms. The gain is more prominent in dense, community-structured networks and diminishes in sparse networks whose topology is more readily inferred from text. In a budding-yeast protein-interaction network, BioGlyph exposes biological organization: cross-community connectors are enriched for essential genes, whereas peripheral proteins are depleted. BioGlyph thus provides an interpretable representation for both language models and scientists to reason about network structure.

Introduction

BioGlyph addresses the difficulty of structural network reasoning by translating topology into interpretable roles, evidence, and consequences while leaving the network and language model unchanged. Across biological, social, and information networks, it improved reasoning over direct and numerical representations, especially for dense networks, and revealed biologically relevant structural associations.

  • Motivation: Network reasoning challenges LLMs when structural properties must be inferred from edge lists, sentences, or measurement tables.Relevant questions include connectivity after perturbation, cross-community connections, and node importance.
  • Method: BioGlyph translates network topology into named structural roles with supporting measurements and consequences of removing nodes or interactions.It uses fixed rules and established network-analysis algorithms without biological labels, task-specific training, or changes to the LLM.
  • Evaluation: 75.6% accuracy was achieved on the yeast benchmark with BioGlyph, compared with 47.3% without network information, 40.1% for edge lists, 37.5% for sentences, and 36.1% for raw graph measurements.Every BioGlyph description fitted within the 24,576-token context window, whereas many alternative representations exceeded it.
  • Evaluation: 79.5% accuracy versus 73.7% for raw-measurement tables, 74.6% versus 57.8% for sentences, 75.0% versus 53.1% for edge lists, and 78.1% versus 47.1% for GraphSAGE roles remained when both representations fit the context window.This pairwise analysis separated representational differences from context-window truncation.
  • Ablation: 82.7% accuracy followed from adding structural consequences to role descriptions, while replacing meaningful role names with meaningless tokens reduced overall accuracy from 66.8% to 51.1%.The ablation results implicate explicit semantic role names and consequence statements in the representation’s performance.
  • Biological relevance: 57.6% of cross-community connectors were essential in yeast, compared with 30.9% of all proteins, while the connector set was comparable to or slightly above equally sized centrality-ranked sets.The corresponding essentiality rates were 56.5% for highest betweenness, 48.2% for highest PageRank, and 24.7% for highest degree.
  • Dense-network evaluation: 77.7% accuracy on the dense ChCh-Miner network exceeded 50.8% without network information, 51.4% for GraphSAGE roles, 41.7% for edge lists, 38.3% for sentences, and 25.3% for raw measurements.Role names alone reached 65.4% on the common ablation subset, while adding structural consequences increased accuracy to 74.6%.

Discussion

BioGlyph makes network topology more accessible to language models by encoding structural roles, supporting evidence, and consequences in compact, interpretable descriptions. Its benefits are strongest for dense or community-structured networks, while biological findings and exact graph algorithms define its practical scope.

  • Discussion: BioGlyph improved structural reasoning by compiling topology into role-based descriptions without changing the network or language model.The descriptions identify roles, provide supporting measurements, and state structural consequences.
  • Discussion: Selective compression and semantic interpretation were the two features associated with BioGlyph’s performance gains.Shorter descriptions fit model context windows more often, while explicit consequence statements and meaningful role names improved accuracy.
  • Discussion: A single BioGlyph description supported multi-turn reasoning, including follow-up questions and challenges to earlier answers.Models still sometimes abandoned correct answers when challenged, showing that the representation does not eliminate conversational instability.
  • Discussion: Learned graph roles and continuous embeddings did not match the benefit of explicit structural meaning for small frozen language models.Unsupervised roles performed close to the no-network baseline, and mismatched-node embeddings performed comparably to node-specific embeddings.
  • Discussion: In biological networks, structural roles revealed nonuniform associations with essentiality and other biological outcomes.Cross-community connectors and community cores were enriched for essential proteins, peripheral proteins were depleted, and Reactome cut nodes included more selective cancer dependencies.
  • Discussion: BioGlyph is most useful where direct representations are difficult to interpret or exceed context limits, but it does not replace exact graph algorithms.Sparse networks can make direct representations competitive, descriptions sometimes exceed prompt budgets, and graph algorithms remain appropriate for defined operations.

Conclusion

BioGlyph serves as an interpretable interface between exact network analysis and natural-language reasoning, helping frozen language models use structural evidence without changing the model or network.

  • BioGlyph converts network topology into concise descriptions of structural roles, evidence, and consequences that small, frozen language models can use unchanged.

Methods

BioGlyph retrieves a question-specific network region, computes classical structural signals, assigns fixed-rule roles, and renders them as evidence-backed text for language-model reasoning. Evaluation uses exact graph-analysis oracles, role-specific checks, and paired comparisons across representations.

  • The BioGlyph compiler: BioGlyph converts each retrieved network region into named structural roles with evidence and structural consequences for a frozen language model.
  • Structural signals: Structural signals include connectivity decompositions, k-core number, community partitions, degree, PageRank, and node and edge betweenness.Directed inputs preserve direction for degree, PageRank, and betweenness; reported experiments treat networks as undirected.
  • The BioGlyph compiler: The compiler assigns eleven roles across nodes, edges, and communities using exact connectivity tests, fixed criteria, or threshold-based structural values.Roles include CUT_NODE, BRIDGE_EDGE, ISOLATE, PERIPHERAL, FRAGILE_REGION, HUB, AUTHORITY, BOTTLENECK_LINK, CROSS_COMMUNITY_CONNECTOR, and BOUNDARY_NODE, with COMMUNITY_CORE defined separately.
  • Structural signals: Threshold-based roles compare structural quantities against fixed standard-deviation thresholds, usually within the retrieved region.The comparison set and thresholds vary by role, and threshold-based roles are not assigned when the comparison standard deviation is zero.
  • The BioGlyph compiler: The CUT_NODE role remains exact at every network size, although its attached evidence may use a whole-region count or a lower bound rather than the component-local grading count.
  • Evaluation: Evaluation compares retrieved-region answers with full-network oracle answers and checks individual structural claims against exact graph analysis.Correct assertions and denials receive equal credit, while paired contrasts exclude units attempted by only one representation.

Data availability

The study uses public networks and validates structural roles against external biological and gene-annotation labels that are withheld from the models.

  • All analyzed networks are public, and the study releases compiled descriptions, question sets, exact answers, model replies, scored tables, and analysis code.

Code availability

The complete BioGlyph implementation and analysis pipelines are available to editors and reviewers, with an interactive browser demonstration for reproduction and application.

  • A private Code Ocean capsule contains the complete BioGlyph implementation, preprocessing, representation construction, training, and evaluation pipelines.
  • An interactive browser-based demonstration lets users reproduce worked examples and apply BioGlyph to their own networks without local installation.

S1 The networks of the main text, and the result pooled over the whole benchmark

The benchmark spans twenty networks across domains, while the main pooled evaluation covers eight networks and compares BioGlyph with connection-, measurement-, and learned representations. BioGlyph leads broadly, with performance gains also observed across model families and network settings.

  • Benchmark scope: Twenty benchmark networks are listed, while the pooled headline evaluates eight main networks across five domains and two frozen 8B models.The main networks are characterized by node and edge meanings, network sizes, and exact cut-node and bridge counts.
  • Pooled benchmark result: 70.6% system accuracy was achieved by BioGlyph across 1,239 questions, versus 50.7% with no network and 48.7–50.0% for edge, adjacency, and learned-role representations.The raw-metric table reached 39.5%, with 41.7% of prompts exceeding the token budget.
  • Comparative performance: BioGlyph beat every encoder and the raw-metric table in all 24 network-by-model combinations, and beat adjacency text in 18 of 24.It reached 74.1% against 74.8% for a benchmark-trained graph neural network and remained below the 82.6% retrieval ceiling.
  • Representation comparison: The consequence-aware BioGlyph rendering reached 68.4%, compared with 61.9% for role names alone and 62.0% for names plus supporting measurements.The consequence step improved performance on 19 of 20 networks carrying the ladder.
  • Network setting: In the three-model environment sweep, BioGlyph led individually, while the six losses to adjacency text occurred on sparse citation and co-authorship graphs.The result suggests that representation advantages vary with network setting.
  • Model-family robustness: Across four model families on a drug-interaction network, BioGlyph outperformed raw measurements, with accuracies ranging from 57.7% to 87.5% versus 15.1% to 32.1%.These results extend the comparison beyond the two pooled 8B models.

S2 Talking to BIOGLYPH: the exchanges behind the numbers

The paper supplements aggregate accuracies with stored model exchanges across network types and interaction formats. These examples show how BioGlyph supports multi-turn structural reasoning while also documenting lookup errors, push-back failures, and context-length effects.

  • Exchange design: The supplementary figures present stored replies from scored runs, with verdicts assigned by the evaluator and cards selected deterministically.The figures include six single-turn questions, sustained threads, and comparisons across representations.
  • What models received: Models saw compiled descriptions and questions, but node identities and external labels such as essentiality or dependency were kept outside the exchanges.The compiled description was sent once with the first question and not repeated.
  • Representation panels: Figure S1 compares system accuracy for six representations, BioGlyph rendering steps, and an opaque-name control across four main-text networks.Its difference panel restricts comparisons to questions both arms fit into the context window and completed.
  • Pooled panels: Figure S2 reports pooled accuracy with confidence intervals, feasible-question comparisons, per-network results, and which arm wins for each network-by-model cell.Reference lines mark the no-network floor, trained graph neural network, and retrieval ceiling.
  • Turn types: The evaluation separates graded, lookup, opinion, and push-back turns, scoring exact answers and responses to asserted wrong answers but not opinions.This separation is maintained both in example cards and in reported numbers.
  • Example exchanges: The conversation examples cover main networks, a yeast link, a disease-network node, an email link, and a knockout table with biological cross-checks.They include a wrong email lookup, capitulation under push-back, and model answers compared with oracle and DepMap information.
  • Matched conversations: Figure S3e contrasts BioGlyph with raw metrics in matched three-turn conversations and also tests benchmark-style versus ordinary prose questions.One raw-metric conversation exceeds the context window; another fits and produces answers from both formats.

S3 The raw-measurement table: where it fails, why, and what recovers it

The raw-measurement table contains at least as much information as BioGlyph but is substantially longer and often exceeds the model’s context window. Larger models and wider windows recover much of its performance, showing that both representation and fit matter.

  • Length and overflow: 41.7% of raw-measurement prompts exceeded the 24,576-token budget, compared with 0.3% for BioGlyph descriptions.Median lengths were 19,723 tokens for the table and 6,229 for BioGlyph; at the wider setting, the yeast ratio reached 7.0 times.
  • Controlled comparison: Among questions both arms fitted and completed, raw measurements reached 61.4% versus 66.8% for BioGlyph, while restricting scores to feasible questions raised the table from 39.5% to 60.7%.The pooled two-stage interval for the shared-question difference excluded zero.
  • Recovery by scale and context: Qwen3-32B scored 41.7% from the table and 82.3% from BioGlyph overall, but 77.0% and 76.9% respectively on questions both arms fitted.A wider 131,072-token window reduced PP-Pathways table overflow from 40.7% to 1.6% and raised accuracy from 32.1% to 52.2%, near BioGlyph’s 52.6%.
  • Interpretation: The table carries every fact in the compiled description and more, but an 8B model reasons better over named roles and over descriptions that fit than over numerical facts that do not.The passage reports this as the paper’s qualified interpretation of the raw-metric comparisons.

S4 Which layer of the description does the work: the ladder on every network

The ladder tests whether BioGlyph’s gains arise from role names, supporting measurements, or stated structural consequences. Across networks, the consequence layer provides the strongest and most consistent improvement, while opaque names reveal the importance of meaningful language.

  • Ladder design: The ladder compares raw measurements, role names, names with evidence, and full descriptions with consequences on twenty networks and common feasible question sets.The four rungs preserve the same computed information while changing its rendering.
  • Main ladder result: On eight main networks and 2,333 units, accuracy moved from 63.6% for raw metrics to 61.9% for names, 62.0% for names plus evidence, and 68.4% for consequences.Only the final step’s interval excluded zero.
  • Across-network consistency: The consequence step was positive on 19 of 20 networks and largest on 17, while naming or evidence was as large or larger on PP-Pathways, HuDiNe, and the single-cell graph.Across eight supporting studies, the consequence step was positive in seven and its interval excluded zero in five.
  • Opaque-name control: The opaque control replaces role names with meaningless tokens, and meaningful names led on all eleven networks carrying the control.Names lifted system accuracy from 55.1% to 58.0% on PP-Pathways and from 51.1% to 66.8% on the yeast interactome.
  • Statistical reporting: The ladder’s rendering steps are reported with reasoning-controlled differences, bootstrap intervals, and totals for BioGlyph versus raw metrics.The accompanying table marks intervals that exclude zero.

S5 The sixteen further networks

Across sixteen further networks, BioGlyph’s advantage is strongest on dense, community-structured graphs and weakens or reverses on sparse or poorly matched networks. Overflow and network structure both shape the observed comparisons.

  • Biological networks: On STRING-Ecoli, BioGlyph reaches 79.3% versus 39.3% for the raw table, with a controlled comparison of 84.7% versus 76.6%.The raw table loses 54.2% of prompts to the budget, and the controlled interval excludes zero.
  • Human interaction networks: On PP-Pathways, BioGlyph reaches 57.4% versus 32.4% for the table, but the fitted comparison narrows to 51.1% versus 49.3%.The table overflows on 45.8% of prompts.
  • Human interaction networks: On HuRI, the description trails the table and names-only rung, while on DG-AssocMiner it reaches 48.9% versus 59.5% for the table and 64.3% for adjacency text.HuRI description prompts overflow at 25.6%; DG-AssocMiner remains lower after both arms fit, at 63.6% versus 65.3%.
  • Patient-derived networks: Patient-similarity graphs show near-parity when regions fit, whereas BioGlyph leads on the disease and patient-similarity networks at 17.1% versus 7.1% and 35.2% versus 18.3%.Retrieved regions are small and regular on the two patient-similarity graphs, and every arm fits.
  • Dense social and information networks: On Amazon-Photo, Wiki-Vote and email-Enron, BioGlyph remains ahead after both arms fit, with controlled accuracies of 70.2%, 69.9% and 77.1%.The corresponding raw-table values are 53.2%, 58.5% and 71.7%.
  • Sparse graphs: On sparse graphs, the raw table leads BioGlyph on Cora, CiteSeer and Power-Grid, while adjacency text leads on ogbn-arxiv.The description’s consequence step stays positive on all five sparse graphs, but the naming step is negative on four.

S6 Model families and scale, and the frontier ceiling

BioGlyph’s benefit persists across additional open model families but is absent for hosted frontier readers with larger context windows. The latter comparison marks a ceiling for the paper’s claim.

  • Open model families: Gemma-3-12B improves from 35.2% on the raw table and 49.2% on adjacency text to 68.0% on BioGlyph.Mistral-Nemo-12B similarly rises from 30.1% and 41.5% to 60.3%.
  • Open model families: Across the eight main networks, both additional open model families preserve the ordering favoring BioGlyph over the raw table and adjacency text.The controlled comparisons are reported in Table S8.
  • Frontier ceiling: For four hosted frontier readers on three networks, every arm scores between 73.9% and 87.8%, and the description leads the table in only one of twelve cells.It matches the table in four cells and trails it in seven, with gaps from 6.6 points below to 0.6 above.
  • Frontier ceiling: The frontier-reader comparison has no overflow and shows the description matching the table in four cells, so the benefit is not universal across model scales.The paper frames this as the honest ceiling of the claim.

S7 Conversation statistics: threads, the three-turn study and prose

Conversation studies test whether models can sustain and revise reasoning from a single BioGlyph description. Accuracy remains stable across turns, while challenge responses and answer normalization affect measured performance.

  • Eight-turn threads: Across 719 eight-turn threads and 11,504 turns, Qwen3-8B answers 87.8% of graded turns and Llama-3.1-8B answers 61.4%.Graded accuracy does not fall along a thread for either model.
  • Eight-turn threads: On challenged answers, Qwen retains correct answers 59.1% of the time and corrects wrong ones 6.8%, while Llama retains 25.1% and corrects 18.4%.The corresponding answer-loss rates are 27.7% for Qwen and 30.6% for Llama.
  • Three-turn study: In the three-turn study, BioGlyph starts all conversations, whereas the raw-measurement table fails to start 17.8% because the region does not fit at turn 1.The study includes 736 model-conversations per arm.
  • Three-turn study: BioGlyph preserves 49.2% of initially correct answers after a challenge, compared with 33.5% for raw measurements.This comparison uses conversations that fitted and finished.
  • Answer evaluation: Canonicalizing answers raises measured accuracy from 42.6% to 65.6%, motivating the extractor used in the conversation analysis.The result is reported in Table S9.

S10 Retrieval

Retrieval determines which network regions any representation can expose, while controlled retrieval comparisons preserve BioGlyph’s advantage. The paper also evaluates biological picks separately from graph-oracle reasoning.

  • Retrieval settings: The wider retrieval setting raises the answerability ceiling from 78.7% to 86.6% while increasing overflow for every long representation.Retrieval is treated as a shared component rather than part of the representation comparison.
  • Oracle-free retrieval: Under oracle-free personalized-PageRank retrieval, BioGlyph reaches 49.9% versus 33.0% for the raw table, which loses 27.2% of prompts.This setting avoids question-family routing, counterfactual paths and verification loops.
  • Retrieval settings: Figure S11 compares retrieval-only answerability, two pooled retrieval settings, and an oracle-free alternative across networks and question families.The figure’s dotted lines mark pooled shares or the no-network floor, depending on panel.
  • Biology-graded evaluation: The candidate-knockout screen presents five anonymous proteins with one biological label and grades the model’s pick against SGD or DepMap rather than the graph oracle.Three experiments are analyzed separately with pre-registered deterministic policy references.
  • Biology-graded evaluation: A deterministic knockout analysis ranks proteins by how much network structure detaches after removal, while the language-model leg estimates detachment counts from compiled descriptions.The Reactome-FI table reports both legs and their comparison with oracle values.
Loading 2609.03229v1…