PaperlayerView exact revision on arXiv

Source-linked AI summary

FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making

Vahid Reza Khazaie, Ahmed Y. Radwan, Shaina Raza

arXiv:2609.01691v1cs.CVcs.LG

TL;DR

VLMs are increasingly used for socially sensitive decisions, but it remains unclear whether they make unsupported high-stakes inferences from faces. FAIRLENS benchmarks fairness and validity across three domains using real face images and complementary response evaluations, finding that unwarranted inference is the dominant failure and that safe behavior requires both similar treatment and abstention when evidence is insufficient.

  • Problem

    It remains unclear whether VLMs make unsupported judgments about qualifications, threat, health, or professional roles from faces in high-stakes settings.

  • Method

    FAIRLENS evaluates eight VLMs on stratified UTKFace images across hiring, legal, and healthcare questions using demographic parity, soundness, demographic association, and open-ended bias measures.

  • Results

    Unsupported inference is the dominant failure: models often answer image-insufficient questions instead of abstaining, while structured soundness and open-ended bias can diverge.

  • Takeaways & Limitations

    Fair high-stakes VLM behavior requires both similar treatment across groups and refusal to infer high-stakes attributes from appearance.

  • Takeaways & Limitations

    The study is limited by coarse UTKFace demographic labels, the gap between benchmark prompts and real decisions, LLM-judge scoring, and aggregate reporting without significance tests.

Abstract

from arXiv · show

Vision-language models (VLMs) are increasingly used to make decisions from visual inputs. We introduce FAIRLENS, a benchmark and evaluation framework for measuring both the fairness and the validity of VLM responses in three high-stakes domains: hiring, legal, and healthcare. FAIRLENS pairs real face images spanning gender, race, and age groups with closed- and open-ended questions, giving more than 100K image-question pairs per model, and evaluates responses from four complementary views: demographic parity over adverse outcome rates, soundness, demographic association over unsupported roles and statuses, and bias in free-text generation. Soundness is the central validity criterion: a response is sound when it follows the evidence stated in the question and abstains when the image cannot support an answer. Evaluating eight VLMs, we find that the primary failure is unwarranted inference rather than unequal treatment. Models routinely infer qualifications, threat, illness, or professional role from a face instead of abstaining, and the weakest model does so on 99% of the questions its input cannot answer. These failures are most severe in legal and healthcare, where recognizing insufficient evidence matters most, and disparity metrics alone would miss them: parity gaps are small in absolute terms, yet when baseline adverse rates are low the same gap means one demographic group receives adverse labels several times as often as another, and a small gap can equally reflect a model that treats every group unsafely. Bias in free-text responses is only loosely coupled to multiple-choice accuracy, so correct structured answers do not imply safe generation. FAIRLENS shows that fair high-stakes VLM behavior requires similar treatment across groups and refusal to infer high-stakes attributes from appearance, and its question suite transfers to any face corpus with demographic annotations.

1 Introduction

FAIRLENS addresses whether VLMs make unsupported high-stakes judgments from faces and introduces a benchmark to evaluate fairness, validity, and demographic associations across hiring, legal, and healthcare scenarios.

  • FAIRLENS asks whether VLMs infer qualifications, trustworthiness, health, threat, or professional roles from faces without sufficient evidence.Such inferences may be unreliable and reproduce demographic stereotypes, especially in hiring, legal, and healthcare settings.
  • FAIRLENS evaluates whether models treat demographic groups similarly, produce logically sound answers, and associate demographic attributes with stereotyped roles or outcomes.The benchmark examines gender, race, and age across fixed questions covering hiring, legal, and healthcare scenarios.
  • The benchmark distinguishes evidence-grounded questions from questions where the image cannot support an answer.For image-insufficient questions, sound behavior requires abstaining with responses such as Cannot tell rather than making unsupported judgments.
  • FAIRLENS scores responses through demographic parity, soundness, demographic association, and open-ended bias metrics.These complementary views assess fairness and validity across model responses rather than relying on a single outcome measure.

2 Related Work

Prior work measures demographic disparities in vision systems, textual stereotypes in language models, and social bias in VLMs. FAIRLENS extends these directions to high-stakes judgments from real face images using structured domain questions and multiple fairness and validity views.

  • Fairness in vision: Prior computer-vision fairness work primarily evaluates recognition and perception, including demographic and intersectional accuracy gaps.FAIRLENS instead examines unsupported socially consequential judgments from face images.
  • Bias benchmarks for large language and vision–language models: Language-model bias benchmarks measure stereotypical associations, with BBQ motivating abstention when contexts are ambiguous or insufficiently informative.This distinction supports FAIRLENS’s soundness criterion for image-insufficient questions.
  • High-stakes trustworthiness and positioning: Broader trustworthiness audits show that capable models can still exhibit fairness, robustness, and safety failures.FAIRLENS applies this concern to high-stakes VLM judgments in hiring, legal, and healthcare settings.
  • Benchmarks for VLM social bias: Existing VLM social-bias benchmarks provide evidence of demographic stereotypes but differ in image sources, tasks, or domain focus.FAIRLENS complements them with real in-the-wild faces and questions targeting high-stakes unsupported visual inference.
  • Positioning of FAIRLENS: FAIRLENS jointly evaluates demographic parity, logical soundness, demographic association, and open-ended bias in human-centered decision settings.Its positioning differs from work that measures only demographic outcome disparities or general social bias.

3 Dataset and Benchmark Construction

FAIRLENS uses a stratified real-face evaluation set from UTKFace and pairs each image with fixed high-stakes questions. The construction preserves demographic representation while separating evidence-grounded from image-insufficient tasks.

  • Source: The benchmark uses real UTKFace images with age, gender, and race annotations rather than synthetic identities or external personal information.The source contains more than 20,000 aligned and cropped face images, while FAIRLENS uses only demographic metadata and face images.
  • Demographic attributes: The evaluation set covers binary gender, five race groups, and three age bands, with splits jointly stratified across these attributes.The gender scheme and heterogeneous Other race category remain limitations inherited from UTKFace.
  • VQA protocol: Each test image receives 69 questions across hiring, legal, and healthcare domains, comprising 38 multiple-choice and 31 open-ended questions.The multiple-choice items support statistical metrics, while open-ended items enable LLM-as-a-judge audits of appearance-based reasoning and bias.
  • Data overview: 1,505 images produce 103,845 image-question-answer triples per model.This includes 57,190 multiple-choice triples and 46,655 open-ended triples.
  • Evidence-grounded vs. image-insufficient questions: FAIRLENS separates evidence-grounded questions from image-insufficient questions for which the image alone cannot provide enough evidence.Evidence-grounded prompts state relevant context explicitly, so sound answers should follow that context rather than appearance.

4 Experiments and Results

FAIRLENS evaluates eight VLMs across hiring, legal, and healthcare using complementary measures of soundness, demographic fairness, unsupported associations, and open-ended bias. Results show that unsupported inference is widespread, especially in legal and healthcare, and that parity and soundness metrics capture distinct safety failures.

  • Settings: Eight VLMs are evaluated across hiring, legal, and healthcare with closed- and open-ended responses under deterministic prompting for closed-ended questions.The evaluation uses cropped face images and compares open-source and proprietary systems across the three domains.
  • Metrics: Soundness requires answers to follow explicit question evidence or return Cannot tell when the face image is insufficient.Soundness is computed as the proportion of responses matching the expected answer across all multiple-choice questions.
  • Metrics: Demographic parity, soundness, demographic association, and open-ended bias provide complementary views of model behavior.Parity measures adverse outcome rates, association measures unsupported non-neutral labels, and open-ended bias is judged by an LLM.
  • Overall Model Performance: 93.8% soundness is achieved by Ovis2.5, while LLaVA-1.6 and LLaMA-3.2-Vision achieve 35.0% and 21.9%, respectively, indicating frequent unsupported inferences.These model-level scores are averaged across the three high-stakes domains.
  • Overall Model Performance: 99.0% association for LLaMA-3.2-Vision coexists with a 0.7 pp association gap, showing that equal demographic treatment can still be uniformly unsafe.Ovis2.5 combines high soundness with a 5.4% association rate, while GPT-5.2-reasoning has a 0.2 pp parity gap and 1.8% open-ended bias rate.
  • Soundness and Open-Ended Bias Across Domains: 60.0% average legal soundness and 56.5% healthcare soundness trail hiring’s 82.2%, where more questions provide explicit qualification or performance evidence.LLaMA-3.2-Vision reaches only 7.0% legal and 4.4% healthcare soundness, whereas Ovis2.5 reaches 84.3% legal and 98.5% in both hiring and healthcare.
  • Demographic Parity and Association: 45.7% healthcare association exceeds hiring’s 30.4% and legal’s 27.7%, despite demographic parity gaps remaining relatively small.Parity gaps range from 1.2–3.4 pp across demographic dimensions and domains, while unsupported roles or statuses remain common.
  • Demographic Parity and Association: A 0.8 pp healthcare race gap can correspond to a 9.0× adverse-rate ratio, demonstrating why percentage-point gaps and rate ratios must be reported together.The relationship depends on baseline adverse rates: large gaps can yield modest ratios when both groups’ rates are high.

5 Discussion and Conclusion

FAIRLENS finds that unsafe inference, not unequal treatment alone, is the dominant fairness failure in high-stakes VLM use. Safe evaluation therefore requires both similar treatment across groups and refusal to infer unsupported attributes from faces.

  • Fairness beyond parity: Demographic parity gaps of about 1–3 pp can conceal large rate ratios or uniformly unsafe labeling across groups.Small absolute gaps do not establish similar risk when baseline adverse rates are low.
  • Unsupported inference: Unsupported inference is the dominant failure: models answer questions about qualifications, threat, illness, pain, or roles that should yield Cannot tell.Abstention is a first-class validity target because faces often cannot provide the required evidence.
  • Unsupported inference: Soundness and open-ended bias are only loosely coupled: GPT-5.2-reasoning has 77.1% soundness and 1.8% open-ended bias, while InternVL3 has 88.1% structured soundness and 10.0% free-text bias.Correct multiple-choice behavior therefore does not guarantee safe unconstrained generation.
  • Domain differences: Legal and healthcare questions are harder on soundness than hiring, while hiring produces the highest open-ended bias from occupational stereotypes.Generic VQA scores would miss these domain-specific patterns.
  • Limitations: The study is limited by coarse demographic labels, distance from real decisions, LLM-judge scoring, and aggregate reporting without uncertainty tests.Future work proposes intersectional analysis and abstention-focused mitigation.
  • Conclusion: Across eight VLMs, similar treatment and refusal to infer high-stakes attributes are separate requirements, and models satisfy the former more often than the latter.The question suite transfers to image sets with demographic annotations beyond UTKFace.

A Dataset and Benchmark Details

The benchmark uses stratified, annotated face images and domain-specific questions to evaluate VLM behavior across demographic groups. Its preprocessing and scoring normalize answer formats while preserving question-specific ground truth.

  • Image dataset: UTKFace filenames encode age, gender, and race labels, enabling sampling of annotated adult face images.Gender is binary, race includes White, Black, Asian, Indian, and Other.
  • Image dataset: A fixed-seed joint stratification over gender, race, and age bands yields the 1,505-image evaluation set.Age bands are young, middle-aged, and senior.
  • Question domains: Hiring, legal, and healthcare questions cover employment judgments, threat and criminality, and health status or clinical roles.The domains target high-stakes inferences from facial appearance.
  • Scoring: Answer normalization treats formatting variants and abbreviations equivalently, while scoring uses context-defined ground truth for evidence-grounded questions.This removes formatting noise without altering substantive behavior.

B Question Suite

The question suite combines domain-framed prompts, closed-ended items, and open-ended probes designed to expose unsupported or stereotyped reasoning. Tables organize the closed-ended questions and demographic bias aggregation by domain.

  • System prompts: Domain framing presents each image as a random snapshot rather than implicitly as a professional portrait or clinical record.Hiring, legal, and healthcare receive distinct framing instructions.
  • Closed-ended questions: The 38 closed-ended questions use fixed options, exclude non-adverse items from demographic parity, and exclude non-stereotype-sensitive items from demographic association.Cannot tell is abbreviated CT in the tables.
  • Open-ended questions: The 31 open-ended questions use direct and counterfactual prompts to surface appearance-based reasoning even when models abstain on multiple-choice items.They are scored only with the LLM-judge bias metric.
  • Open-ended questions: Hiring prompts vary occupations and recruiter scenarios, while legal and healthcare prompts probe threat, aggression, illness, pain, and clinical judgments.The healthcare examples include clinician-style notes, triage concerns, and illness-severity overestimation.
  • Question inventory: Tables 6, 7, and 8 organize 21 hiring, 9 legal, and 8 healthcare closed-ended questions, respectively.Hiring role-parameterized items are listed once with a ×6 multiplier.
  • Legal questions: The legal suite contains nine questions, including threat, suspiciousness, checkpoint stopping, violent capability, criminality, and force justification.A mistakable-weapon item uses No as the sound answer because the object is absent from the cropped face.

C Open-Ended Bias by Model and Demographic Group

Open-ended bias varies systematically by demographic group and domain, but race ordering is unstable across models. Average bias is higher for male and senior subjects, with the largest race spread in hiring.

  • Reporting structure: Table 9 uses unweighted means across eight models, while Tables 10–12 provide per-model bias rates by demographic group for legal and healthcare.The tables complement domain-level aggregation with demographic breakdowns.
  • Gender and age: Male subjects have higher average bias rates than female subjects in every domain: hiring 18.6% vs. 15.1%, legal 13.5% vs. 11.4%, and healthcare 7.0% vs. 6.1%.The direction holds for seven of eight models, with a slight reversal by LLaMA-3.2-Vision in legal and healthcare.
  • Gender and age: Senior subjects average higher bias than young subjects in hiring, legal, and healthcare: 21.5% vs. 15.8%, 15.2% vs. 11.7%, and 8.8% vs. 5.5%.LLaVA-1.6 in hiring is the only clear exception.
  • Race differences: Hiring has the largest average race spread at 3.0 pp, compared with 1.7 pp in legal and 0.6 pp in healthcare.White subjects are highest and Black subjects lowest in the hiring averages.
  • Race differences: Race ordering is unstable across models, with CogVLM ranging from 30.2% for Asian subjects to 35.2% for Indian subjects in hiring.The averages indicate uneven distribution rather than a stable demographic ranking.
Loading 2609.01691v1…