Source-linked AI summary
HARP-LoRA: Hierarchy-Conditioned Rank Projection for Low-Rank Adaptation in Scientific Table Understanding
Hossein Sholehrasa, Mohammadhossein Amini, Doina Caragea, Remya Ampadi Ramachandran, Dan Wu, Zihang Shi, Zelin Li, Lindsey Hulbert, Eder Martinez, Xuan Xu, Majid Jaberi-Douraki
TL;DR
Complex table layouts make it difficult to associate scientific and PK values with their surrounding header context, which standard low-rank adaptation does not explicitly represent. HARP-LoRA conditions projected low-rank adaptation on each token’s row- and column-header hierarchies. It reports strong results across PK tables and WikiTQ while using a small trainable parameter budget.
Problem
LoRA does not explicitly incorporate table row, column, or header relationships, although PK values must be associated with complex surrounding context.
Source: §I · p. 1, B. Table-Specific PEFT · p. 2Method
HARP-LoRA forms token-specific intrinsic representations from row- and column-header hierarchies to condition projected low-rank adaptation while keeping the pretrained model frozen.
Source: §I · p. 2, A. Preliminaries · p. 3, B. Hierarchy-Conditioned Projected LoRA · p. 4Results
Gemma 3 27B with HARP-LoRA achieves a PK F1-score of 70.63% and WikiTQ accuracy of 56.87%, the strongest overall performance among evaluated model–method combinations.
Source: A. Main Results · p. 6Takeaways & Limitations
HARP-LoRA reports comparable or improved performance across evaluated settings while using substantially fewer trainable parameters than LoRA and TableLoRA.
Source: §I · p. 2The method assumes cell coordinates and header relationships are available from preprocessing; table detection and structural reconstruction from raw documents are outside its scope.
Source: §VII · p. 7
Abstract
from arXiv · showhide
Tabular data is widely used in scientific applications, where complex layouts, including multi-level headers, merged cells, captions, and footnotes, make information extraction difficult. This is particularly important in pharmacokinetics (PK), where values must be correctly associated with the drug, species, dose, administration route, unit, and experimental condition. Parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA) reduce adaptation cost but do not explicitly incorporate relationships between table values and surrounding headers. We introduce HARP-LoRA, a hierarchy-conditioned rank projection method for low-rank adaptation in scientific table understanding. HARP-LoRA extends projected LoRA by constructing token-specific intrinsic representations from the row- and column-header hierarchies of each data token. These representations condition the low-rank down-projection, incorporating hierarchical context while keeping the pretrained model and random projection tensors frozen. We evaluate HARP-LoRA on 950 manually annotated PK tables from 587 scientific articles and on WikiTableQuestions (WikiTQ) using Llama~3.2 3B, Llama~3.1 8B, and Gemma~3 27B. With Llama~3.1 8B, HARP-LoRA achieves a PK F1-score of 57.64\% and WikiTQ accuracy of 53.15\%, improving over LoRA by 1.42 and 6.56 points, respectively. With Gemma~3 27B, it achieves the best PK F1-score of 70.63\% and WikiTQ accuracy of 56.87\%. These results use only 172K, 197K, and 547K trainable parameters, far fewer than LoRA and TableLoRA. Overall, conditioning lightweight adapters on table-header hierarchies improves extraction from scientific tables and reasoning over flatter general-domain tables while maintaining parameter efficiency. Code is available at: https://github.com/hosseinsholehrasa/HARP-LoRA
I. INTRODUCTION
Scientific table understanding is difficult because values depend on complex layouts and header relationships, with PK extraction especially reliant on correctly connecting values to experimental context. HARP-LoRA conditions parameter-efficient adaptation on those header hierarchies and reports comparable or improved performance in its evaluations.
I. INTRODUCTION
- Complex scientific tables make reliable extraction difficult because PK values must be associated with drug, species, dose, route, units, and experimental conditions.PK information also supports dosing, safety, regulatory decisions, and drug-use recommendations.Source: §I · p. 1
- LoRA adapts serialized table tokens without explicitly conditioning on row, column, or header relationships, while TableLoRA uses coordinates rather than multi-level header semantics.The introduction identifies parameter-efficient adaptation that directly incorporates hierarchical structure as underexplored.Source: §I · p. 2
- HARP-LoRA generates token-specific intrinsic representations from row- and column-header hierarchies to condition low-rank adaptation.The pretrained model remains frozen, and the method is designed to keep its trainable parameter budget small.Source: §I · p. 2
- Header-cell mean pooling, level-preserving adaptive pooling, and RMS normalization provide a parameter-efficient representation of multi-level headers for the adapter.These operations incorporate hierarchical structural information directly into adaptation.Source: §I · p. 2
- HARP-LoRA was evaluated on 950 manually annotated PK tables and WikiTQ with three language-model backbones, and it outperformed baselines in several settings using substantially fewer trainable parameters than LoRA and TableLoRA.The reported performance is comparable to or better than existing baselines across the evaluated settings.Source: §I · p. 2
II. RELATED WORK
Prior work models table structure through specialized representations, pretraining, fine-tuning, serialization, or external processing. The related work motivates lightweight adaptation that can incorporate table structure directly without modifying the base LLM architecture.
A. Table Structure Modeling and LLM Reasoning
- Prior approaches span specialized table representations and pretraining as well as LLM fine-tuning, serialization studies, and PK-specific preprocessing and extraction.Examples include TAPAS, TURL, OmniTab, TableGPT, TableLLM, and AutoPK.Source: A. Table Structure Modeling and LLM… · p. 2
- Earlier structure-aware methods can require transformer changes or dedicated pretraining, while LLM-based approaches often leave the model to recover table organization from serialized inputs or preprocessing.This motivates lightweight adaptation that incorporates structure during fine-tuning without modifying the base architecture.Source: A. Table Structure Modeling and LLM… · p. 2
B. Table-Specific PEFT
- PEFT keeps base weights frozen and trains a small number of parameters, while LoRA extensions target rank allocation, initialization, compression, or fixed random projections.The passage names AdaLoRA, PiSSA, LoRA-XS, and projected LoRA as examples.Source: B. Table-Specific PEFT · p. 2
- TableLoRA adds special table tokens and row- and column-position information, but coordinates alone do not capture dependencies induced by multi-level headers.A cell’s meaning can depend on several levels of surrounding headers rather than only its row and column position.Source: B. Table-Specific PEFT · pp. 2–3
C. Scientific Table Understanding in Pharmacokinetics
PK values are distributed across complex table contexts, and their meaning depends on surrounding header hierarchies. Existing extraction work addresses retrieval, classification, preprocessing, or serialized inputs, while the proposed framework conditions low-rank adaptation on row- and column-header hierarchies.
C. Scientific Table Understanding in Pharmacokinetics
- PK values must be interpreted with drug, species, dose, route, units, and experimental conditions, often distributed across multi-level headers.Earlier work includes manual curation, literature collection, table classification, and extraction pipelines such as AutoPK.Source: C. Scientific Table Understanding in… · p. 3
III. PROBLEM DEFINITION AND MOTIVATION
- Serialization preserves table-token text but does not directly preserve the two-dimensional row–column relationships and higher-level header links.Adding structure to the LLM architecture can require architectural changes and additional pretraining or substantial retraining.Source: §III · p. 3
- PK extraction requires preserving each value’s association with its parameter, drug, species, dose, route, unit, and experimental condition.Incorrect associations can mislead resources supporting veterinary drug-use and withdrawal-interval recommendations.Source: §III · p. 3
- Existing approaches represent table structure through serialization, positional information, preprocessing, or coordinate-based PEFT, but do not directly model multi-level header hierarchies during adaptation.This gap is especially consequential when cell meaning depends on surrounding headers rather than the cell alone.Source: §III · p. 3
- The proposed framework conditions low-rank adaptation on each token’s row- and column-header hierarchies while keeping the pretrained model frozen.It is evaluated on a hierarchy-rich PK dataset and WikiTQ, which primarily contains flatter table structures.Source: §III · p. 3
IV. METHODOLOGY
HARP-LoRA encodes each table token’s row- and column-header paths into a conditioning vector for its token-specific down-projection, then uses a shared up-projection. Its projected-LoRA preliminaries construct LoRA factors from fixed random projection tensors and trainable low-dimensional variables.
IV. METHODOLOGY
- For each table token, HARP-LoRA encodes its associated row- and column-header paths separately, combines them into a conditioning vector, and uses that vector to construct a token-specific down-projection.A shared LoRA up-projection maps the resulting low-dimensional representation back to the model hidden space.Source: §IV · p. 3, Figure 1 · p. 4
A. Preliminaries
- Projected LoRA constructs LoRA factors by projecting low-dimensional trainable variables through fixed Gaussian random tensors that are sampled once and kept frozen.Only the low-dimensional latent variables are optimized during gradient-based training.Source: A. Preliminaries · p. 3
- The fixed Gaussian tensors used for the projected factors have dimensions r×din×k and dout×r×k.The passage introduces these tensors as frozen components of the projected parameterization.Source: A. Preliminaries · p. 3
- The mode-3 tensor–vector product contracts a tensor’s third dimension with the corresponding intrinsic vector.The projected-LoRA formulation uses trainable intrinsic vectors for the down- and up-projection factors.Source: A. Preliminaries · pp. 3–4
B. Hierarchy-Conditioned Projected LoRA
HARP-LoRA replaces the shared intrinsic vector of projected LoRA with a token-specific vector derived from the token’s row- and column-header hierarchies. Separate trainable vectors preserve the two branches, while the projection tensors remain frozen and the up-projection is shared.
B. Hierarchy-Conditioned Projected LoRA
- HARP-LoRA generates a token-specific intrinsic vector from the hierarchical row and column headers associated with each table token.The original projected formulation shares mA across tokens; HARP-LoRA instead conditions the down-projection on each token’s header context.Source: B. Hierarchy-Conditioned Projected LoRA · p. 4
- Separate trainable row- and column-branch vectors rescale their respective encoded hierarchies, whose gated representations are added to form mt.The element-wise operation is the Hadamard product.Source: B. Hierarchy-Conditioned Projected LoRA · p. 4
- A single frozen GA tensor is shared across the row and column branches, while the up-projection remains shared across tokens.Source: B. Hierarchy-Conditioned Projected LoRA · p. 4
C. Hierarchical Header Representation
For each table token, the method represents its associated row- and column-header paths across hierarchy levels. It pools header-cell tokens, preserves hierarchy levels while compressing the hidden dimension, then vectorizes and RMS-normalizes the result.
C. Hierarchical Header Representation
- Each table token is linked to corresponding row and column headers across up to Lc column-header levels and Lr row-header levels.Ct,ℓc and Rt,ℓr denote the token sets in the associated column- and row-header cells at each level.Source: C. Hierarchical Header Representation · p. 4
- Header-cell representations are obtained by mean-pooling the hidden representations of all tokens in each cell.Source: C. Hierarchical Header Representation · p. 4
- Pooled representations from all hierarchy levels are stacked, then adaptive average pooling compresses only the hidden dimension while preserving separate hierarchy levels.The output size is qb × Lb, where qb = k Lb and k is chosen to be divisible by both Lr and Lc; the pooled matrix is then vectorized.Source: C. Hierarchical Header Representation · p. 4
- The resulting vector is normalized by its root-mean-square magnitude.Source: C. Hierarchical Header Representation · p. 5
A. Datasets and Tasks
The study evaluates PK information extraction from manually annotated scientific tables and question answering over WikiTQ. Experiments compare four adaptation methods across three pretrained models under shared data, input, and output settings.
A. Datasets and Tasks
- The PK dataset contains 950 manually labeled tables from 587 articles, split into 570 training, 95 validation, and 285 test tables.The extraction task uses tables and article context, and asks models to return prompt-relevant PK information in a fixed schema.Source: A. Datasets and Tasks · p. 5
- WikiTQ comprises 22,033 question–answer pairs over 2,108 Wikipedia tables and tests lookup, comparison, aggregation, and arithmetic reasoning.Its tables generally have flat, single-level headers, and test tables are disjoint from training tables.Source: A. Datasets and Tasks · p. 5
B. Experiment Setup
- Experiments compare LoRA, TableLoRA, projected LoRA, and HARP-LoRA on Llama 3.2 3B, Llama 3.1 8B, and Gemma 3 27B using the same splits, serialization, prompts, and output format.All pretrained model parameters and random projection tensors were frozen; adapters were applied to key and value projections unless otherwise stated.Source: B. Experiment Setup · p. 5
C. Evaluation Metric
PK evaluation compares aligned generated and reference cells, while WikiTQ uses answer accuracy. HARP-LoRA performs strongly across model scales, with the captioned results highlighting its 8B and 27B comparisons and a separate study identifying k=1024 as the best tested intrinsic dimension.
A. Main Results
- 70.63% PK F1-score and 56.87% WikiTQ accuracy make HARP-LoRA the strongest overall model–method combination with Gemma 3 27B.It also achieves 79.11% precision on PK; Table I compares the methods across both datasets and three backbones.Source: Table I · p. 6, A. Main Results · p. 6
- 57.64% PK F1-score and 53.15% WikiTQ accuracy are HARP-LoRA’s best results with Llama 3.1 8B, exceeding projected LoRA by 0.86 and 4.18 points, respectively.These results use 197K trainable parameters, compared with 2.62M for LoRA and 3.48M for TableLoRA 2D.Source: Table I · p. 6, A. Main Results · p. 6
C. Evaluation Metric
- PK F1 pools cell-level matches after column and row alignment; WikiTQ reports answer accuracy against ground-truth answers.PK evaluation uses similarity thresholds for alignment and cell matching, and averages precision, recall, and F1 separately across three runs.Source: C. Evaluation Metric · p. 6
A. Main Results
- With Llama 3.2 3B, HARP-LoRA ranks second on both datasets and exceeds TableLoRA 2D and projected LoRA on PK F1.The passage describes the benefit as particularly clear on the hierarchy-rich PK task.Source: A. Main Results · p. 6
- The intrinsic-dimension study reports improving PK F1 as k rises from 256 to 1024, with the best score at k=1024.The caption describes HARP-LoRA with Llama 3.1 8B; it does not provide the score values.Source: Table II · p. 7
B. Effect of Intrinsic Dimension
HARP-LoRA performs best at an intrinsic dimension of 1024 in the tested settings, and its ablations show contributions from both header hierarchies and the other components. Its scope assumes preprocessed table structure, and its token-specific projection adds computation that can grow substantially for long sequences or large intrinsic dimensions.
B. Effect of Intrinsic Dimension
- F1-score rises from 30.68 at k = 256 to 57.64 at k = 1024, then falls slightly to 56.79 at k = 2048.The best result among the evaluated settings is at k = 1024; the reported gains suggest capacity benefits saturate beyond that point.Source: B. Effect of Intrinsic Dimension · p. 7
C. Ablation Study
- The full model scores 57.64 F1 on PK, while removing column-header hierarchy causes a larger drop than removing row-header hierarchy.Removing column-header hierarchy yields 56.36, compared with 56.91 without row-header hierarchy; the full model exceeds the shared-vector projected-LoRA variant by 0.86 points.Source: C. Ablation Study · p. 7, Table III · p. 7
VII. LIMITATIONS AND FUTURE WORK
- Token-specific down-projection costs O(Nrdink), versus O(rdink) for projected LoRA, and overhead can become substantial for long sequences or large intrinsic dimensions.Future work will investigate caching and reusing projections for tokens with shared structural contexts.Source: §VII · p. 7
- The method assumes cell coordinates and header relationships are available from preprocessing; automatic table-structure reconstruction from raw documents is outside its scope.The intrinsic dimension k must also be divisible by the selected row- and column-hierarchy depths.Source: §VII · p. 7
- The current framework is evaluated on text-based tables and does not directly process tables embedded in PDFs or images.Future studies will extend it to multimodal document settings and broader scientific and industrial domains.Source: §VII · p. 7
VIII. CONCLUSION
HARP-LoRA conditions low-rank adaptation on token-specific row- and column-header hierarchies while keeping pretrained weights and random projection tensors unchanged. Experiments on PK tables and WikiTQ show strong results with substantially fewer trainable parameters than LoRA and TableLoRA.
VIII. CONCLUSION
- HARP-LoRA generates token-specific intrinsic vectors from row- and column-header hierarchies to incorporate table structure into adaptation.The method keeps the pretrained model and frozen random projection tensors unchanged.Source: §VIII · p. 7
- HARP-LoRA achieves the best overall results with Llama 3.1 8B and Gemma 3 27B using only 172K–547K trainable parameters.The conclusion reports experiments on 950 manually annotated PK tables and WikiTQ, with substantially fewer trainable parameters than LoRA and TableLoRA.Source: §VIII · p. 7
- The conclusion identifies reducing token-specific projection cost, recovering table structure from raw documents, and extending to multimodal PDF and image tables as future work.The current method does not directly process tables embedded in PDFs or images.Source: §VIII · pp. 7–8