Source-linked AI summary
Improving LLM Group Fairness on Tabular Data via In-Context Learning
Valeriia Cherepanova, Chia-Jung Lee, Nil-Jana Akpinar, Riccardo Fogliato, Martin Andres Bertran, Michael Kearns, James Zou
TL;DR
LLMs can make tabular predictions in low-data settings, but NLP fairness approaches do not ensure group-fair tabular outcomes. This paper evaluates four fairness approaches across tabular datasets and models. Its conclusions distinguish practical trade-offs and recommend approaches according to model and resource constraints.
Problem
Fairness in tabular prediction depends on class labels and demographic representation, and fairness achieved in NLP does not ensure group-fair tabular predictions.
Source: §1 · p. 1Method
Results
Fair prompt optimization can improve fairness and classification performance depending on model creativity, while fair few-shot examples yield optimal results across models and datasets when positive-example ratios are selected optimally.
Source: §6 · p. 8Takeaways & Limitations
The authors recommend fair few-shot examples and fair prompt optimization for optimal accuracy trade-offs, soft prompt tuning as a potential option for smaller models, and self-refinement for larger models with limited budgets.
Source: §6 · pp. 8–9The study examines in-context approaches, excludes data preprocessing and most training or fine-tuning strategies, and focuses on demographic parity because it suits little-to-no-training-data settings.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Large language models (LLMs) have been shown to be effective on tabular prediction tasks in the low-data regime, leveraging their internal knowledge and ability to learn from instructions and examples. However, LLMs can fail to generate predictions that satisfy group fairness, that is, produce equitable outcomes across groups. Critically, conventional debiasing approaches for natural language tasks do not directly translate to mitigating group unfairness in tabular settings. In this work, we systematically investigate four empirical approaches to improve group fairness of LLM predictions on tabular datasets, including fair prompt optimization, soft prompt tuning, strategic selection of few-shot examples, and self-refining predictions via chain-of-thought reasoning. Through experiments on four tabular datasets using both open-source and proprietary LLMs, we show the effectiveness of these methods in enhancing demographic parity while maintaining high overall performance. Our analysis provides actionable insights for practitioners in selecting the most suitable approach based on their specific requirements and constraints.
1 Introduction
Tabular group fairness is distinct from stereotypical fairness in NLP: fair language-model representations do not ensure fair tabular predictions when group class distributions differ. This work evaluates four approaches to improve demographic parity across tabular datasets and models, then offers recommendations for choosing among them.
1 Introduction
- Tabular group fairness depends on class labels and demographic representation within them, so NLP-style fairness does not ensure equitable tabular predictions when group class distributions differ.Prior tabular studies found fairness discrepancies across groups and limited effectiveness from resampled fine-tuning and few-shot label flipping.Source: §1 · p. 1
- The paper empirically examines prompt optimization, soft prompt tuning, fair few-shot learning, and self-refining predictions to improve demographic parity in tabular tasks.The evaluation uses open-source and proprietary models across four datasets and provides recommendations for selecting methods in different scenarios.Source: §1 · pp. 1–2
2 Related Work
Related work covers LLM prediction on tabular data, bias mitigation in language models, and group-fairness methods for tabular prediction. The paper positions its contribution at the intersection of these areas.
2.1 Large Language Models on Tabular Data
- LLMs have been studied for serialized-tabular classification and can outperform traditional algorithms in low-data settings, but prior work reports fairness disparities and limited success closing the gap.Label flipping in in-context examples reduces bias but can lower prediction performance (Liu et al., 2023).Source: §2.1 · p. 2
2.2 Bias and Stereotypes in LLMs
- LLM biases have been linked to training data and reported across demographic groups, with potential harms in high-stakes deployment; mitigation and detection strategies remain challenging at pretraining scale.Reported biases include gender, race, ethnicity, and socioeconomic status (Wan et al., 2023; Haim et al., 2024).Source: §2.2 · p. 2
2.3 Fairness on Tabular Data
- Tabular fairness methods include data debiasing, optimization constraints, and prediction post-processing, while group-fairness metrics compare target measures across demographic groups.Demographic parity is a widely adopted measure requiring approximately equal positive-prediction frequencies across groups (p56).Source: §2.3 · pp. 2–3
3 Experimental Details
Experiments target low- or no-training-data prediction using serialized tabular inputs and task instructions, with fairness prompts or demonstrations varying by method. They evaluate accuracy alongside demographic parity and equalized odds across several models and datasets.
3 Experimental Details
- Experiments focus on little-to-no training data; predictions use task instructions and sample features, optionally augmented with fairness instructions or few-shot examples.A 50-example labeled validation set assesses accuracy for selecting prompts, while demographic parity is assessed directly on the test set.Source: §3 · p. 3
3.1 Language Models
- The experiments include Llama 3 8B and 70B, Mistral 7B, Mixtral 8x7B, and Claude Sonnet; computationally intensive methods use smaller models, while methods requiring advanced reasoning use larger models.The CatBoost comparison trains gradient-boosted decision trees on 50 examples and enforces a fairness constraint via GridSearch1.Source: §3.1 · p. 3, §3 · p. 3
3.2 Datasets
- The evaluation uses public fairness datasets, focuses on gender as the protected attribute, and includes Adult income, German credit, and ACS income and coverage prediction tasks.Evaluation samples include 1,000 examples for Adult and each ACS task; German credit is split evenly between evaluation and instruction generation.Source: §3.2 · p. 3
3.3 Serialization and prompts
- Categorical values are mapped to text and column names and entries are combined into row strings; task instructions are constructed from training-fold prototype clusters and revised into a summary paragraph.Figure 1 distinguishes prompt components optimized by each approach from gray components that remain unchanged.Source: §3.3 · pp. 3–4, Figure 1 · p. 4
3.4 Metrics
- Demographic parity measures equal positive-label selection rates across groups and is reported as the smallest-to-largest group-rate ratio, with values nearer 1 indicating better parity.Equalized odds, which balances false-positive and false-negative rates across groups, is also evaluated; demographic parity can be measured on an unlabeled test set without labeled training data.Source: §3.4 · p. 4
4 Methods
The methods alter fairness-specific instructions, embedding-space prompts, in-context demonstrations, or model predictions. They differ in how fairness is introduced and in their data, computation, and reasoning requirements.
Fair Prompt Optimization
- Fair prompt optimization appends fairness-specific instructions to task instructions and uses a meta-LLM to refine them iteratively from prior instructions’ demographic-parity scores and group selection rates.The process runs for 100 iterations and uses the same model as the downstream predictor as its meta-LLM.Source: §4 · pp. 4–5
Soft Prompt Tuning
- Soft prompt tuning optimizes prompt embeddings rather than discrete tokens, applying a fairness penalty to equalize positive-label token likelihoods across groups.It tunes 50 task-initialized tokens for 20 epochs on 1,000 zero-shot pseudo-labeled examples and includes cross-entropy loss to keep predictions close to the original outputs.Source: §4 · p. 5
Fair Few-Shot Examples
- Fair few-shot examples are selected by nearest-neighbor search for each test instance, share its sensitive attribute, and use the model’s default zero-shot predictions as pseudo-labels.The approach varies positive-example ratios for female and male test samples across six values each, producing 36 ratio pairs; similarity uses Jaccard for mostly categorical features.Source: §4 · pp. 5–6
Self-Refinement
- Self-refinement uses chain-of-thought reasoning to identify near-boundary individuals and flip predictions as needed to reach a target demographic-parity ratio.It first predicts a batch, then assesses batch parity and adjusts predictions; experiments use Llama3 70B and Claude Sonnet.Source: §4 · p. 6
5 Experimental Results
Across four approaches, the experiments examine fairness–accuracy trade-offs. Optimized fair prompts often improve fairness without much accuracy loss, while the other methods have distinct trade-offs and scope conditions.
5.1 Fair Prompt Optimization
- Manually written fairness instructions can improve demographic parity in some models, but no universal fair instruction consistently works across models.Source: §5.1 · p. 6
- Optimized fair prompts significantly improve fairness, often without much accuracy loss or even with accuracy gains.Table 1 covers optimized prompts, tuned soft prompts, and few-shot contexts across three models and four datasets.Source: Table 1 · p. 7, §5.1 · p. 7
5.2 Soft Prompt Tuning
- Soft prompt tuning improves demographic parity across datasets but trades off accuracy versus hard prompt optimization; hyperparameter sensitivity or pseudo-label reliance may contribute.Its batch-level fairness regularizer aims to equalize positive-label likelihoods across groups.Source: §5.2 · p. 7
5.4 Self-Refinement
- Self-refinement of batches of 40 when group positive rates differ by over 15% improves demographic parity except on ACS coverage, sometimes with notable accuracy trade-offs.Source: Table 2 · p. 8, §5.4 · p. 8
5.3 Fair Few-shot Examples
- Increasing female positive-example ratios improves demographic parity, while random examples show lower parity and greater variance than nearest neighbors.The strategy selects nearest neighbors, uses zero-shot predictions as pseudo-labels, and adjusts positive-example ratios between groups.Source: §5.3 · pp. 7–8, Figure 5 · p. 8
5.4 Self-Refinement
- The correction step gives no guarantee that similar individuals receive similar predictions, which conflicts with individual fairness.Source: §5.4 · p. 8
6 Conclusion
The conclusion compares four fairness approaches and recommends fair few-shot examples and fair prompt optimization for their accuracy tradeoff, with other methods suited to narrower scenarios.
6 Conclusion
- Fair prompt optimization can improve fairness and classification performance, but depends on model creativity and repeated dataset evaluation.Its instructions are interpretable, though why particular instructions improve fairness is not always clear.Source: §6 · p. 8
- Soft prompt tuning is computationally expensive and hyperparameter-sensitive, but permits differentiable fairness regularizers and may suit smaller models.It optimizes prompts in embedding space rather than discrete token space.Source: §6 · p. 8
- Fair few-shot examples are the most interpretable and predictable method, with optimal results across models and datasets when positive-example ratios are selected optimally.They require longer contexts and can cost more on larger datasets because of the needed forward passes.Source: §6 · p. 8
- Self-refinement requires strong reasoning capabilities and does not guarantee similar predictions for similar individuals.Batching predictions and revisions offers a computational advantage for larger models by reducing overall processing time.Source: §6 · p. 8
- Fair few-shot examples and fair prompt optimization are recommended as universal approaches with the optimal accuracy tradeoff.Soft prompt tuning may adapt smaller models, while self-refinement may suit larger models with limited budgets.Source: §6 · pp. 8–9
7 Limitations and Potential Risks
The work is limited to in-context approaches and demographic parity, and it notes potential performance and fairness risks that warrant case-specific evaluation.
7 Limitations and Potential Risks
- The study examines in-context approaches, omitting bias-mitigating preprocessing, direct post-hoc output modification, and training or fine-tuning beyond soft prompt tuning.Its fairness focus is demographic parity, chosen for settings with little to no training data; other notions may be adapted when labeled data are available.Source: §7 · p. 9
- Fairness improvements may come at a predictive-performance cost, overlook other criteria, or introduce unmeasured biases, so practitioners should assess fit before high-stakes deployment.The stated risks include optimizing group fairness along one dimension while potentially amplifying or introducing biases not directly measured.Source: §7 · p. 9
A Prompt Details
The appendix illustrates prompt construction across four datasets and describes how optimized prompts and selected few-shot examples are placed relative to the test sample.
A Prompt Details
- Default prompt examples mark task instructions in red and the prediction sample in blue.The appendix identifies the colored portions of its templates.Source: A Prompt Details · p. 13
- Optimized fair prompts replace task instructions, while selected fair few-shot examples precede the test sample.The appendix describes both prompt-placement conventions.Source: A Prompt Details · p. 13
A.1 Prompt Template for Adult dataset
- The Adult income template gives typical income-associated traits, presents an individual profile, and asks whether income exceeds $50K per year.The answer choices are income at or below $50K and income above $50K.Source: §A.1 · p. 13
A.2 Prompt Template for German Credit dataset
- The German Credit template describes traits associated with good or bad credit risk, then asks for a classification from a customer profile.The profile includes demographic, employment, housing, savings, checking, amount, duration, and loan-purpose attributes; choices are good or bad.Source: §A.2 · p. 13
A.3 Prompt Template for ACS Income dataset
- The ACS Income template contrasts characteristics associated with income above or below $50K before asking for a prediction from an individual profile.The profile includes sex, race, education, relationship, age, work hours, marital status, birthplace, worker class, and occupation; the choices are at or below $50K and above $50K.Source: §A.3 · p. 13
A.4 Prompt Template for ACS Coverage dataset
- The ACS Coverage template describes traits associated with public insurance coverage, then asks whether a profiled individual is covered.The profile includes demographic, education, military-service, disability, income, marital, and employment information; the choices are covered or not covered.Source: §A.4 · pp. 13–14
B Additional Experimental Details and Hyperparameters
Soft prompt tuning uses 50 task-instruction-initialized tokens, pseudo-labeled training samples, and demographic-parity regularization under specified model-specific settings.
B Additional Experimental Details and Hyperparameters
- Soft prompt tuning fine-tunes 50 task-instruction-initialized tokens for 20 epochs, using 1,000 zero-shot pseudo-labeled samples and a demographic-parity penalty weight of 0.5.Learning rates are 1e−4 for Llama 8B and 1e−5 for Mistral, with three warm-up epochs; batch sizes are 50 for Llama and 60 for Mistral under computational constraints.Source: B Additional Experimental Details and… · p. 14
C Datasets
The appendix points to dataset and feature details, reports experimental hardware and compute costs, and lists Pareto-optimal prompts with the highest and lowest demographic parity ratios.
C Datasets
- Dataset and feature details are listed in Table 3.Source: C Datasets · p. 14
D Hardware
- Experiments used eight Tesla V100 32GB GPUs through AWS; soft-prompt tuning took about 120 GPU hours per model and dataset, totaling 950 hours.Prompt optimization took around 35 hours per model and dataset (420 total); fair few-shot experiments took about 60 hours per model and dataset for one seed (2,160 total).Source: D Hardware · p. 14
E.1 Additional Results for Fair Prompt Optimization
- Pareto-optimal prompts for each dataset and model achieve the highest and lowest demographic parity ratios.The prompts are provided in Tables 4 and 5.Source: §E.1 · p. 14
E.2 Additional Results for Fair Few-Shot Examples
Across datasets and models, changing the positive-example proportion for a demographic group raises its selection rate, while random examples do not significantly shift selection rates. The Pareto-frontier comparison depicts accuracy–fairness trade-offs across three methods.
E.2 Additional Results for Fair Few-Shot Examples
- Increasing a demographic group's proportion of positive few-shot examples raises its selection rate across all datasets and models.Figures 7 and 8 show accuracy and demographic-parity metrics and male/female selection rates for fair in-context examples.Source: §E.2 · p. 14, Figure 7 · p. 16, Figure 8 · p. 17
- Random examples do not significantly influence selection rates, unlike the nearest-neighbor selection strategy.The comparison uses randomly selected examples while preserving similar in-context label ratios.Source: §E.2 · p. 14
E.3 Comparing Pareto Frontiers
- Figure 9 compares accuracy–fairness Pareto trade-offs for optimized fair prompts, soft prompts, and optimized class ratios for few-shot examples.It also marks zero-shot model performance and the Catboost GridSearch baseline.Source: §E.3 · pp. 14–15, Figure 9 · p. 20