PaperlayerView exact revision on arXiv

Source-linked AI summary

FAIRGate: Agentic Runtime Policy Selection and Verified Repair for Fairer Recommendation Delivery

Navkar Jain, Enock Ayiku, Kenneth Fletcher

arXiv:2610.30359v1cs.IR

TL;DR

Fairness interventions often fix one objective, although the relevant action can vary across recommendation requests. FAIRGate selects and plans repairs from a predefined policy library, then uses deterministic validation to govern delivery. The reported results include improved long-tail exposure versus the base ranker and more evidence-grounded narration than direct LLM explanations.

  • Problem

    Existing fair-ranking and re-ranking approaches leave less studied whether final delivery can select a relevant policy per request, stay within a relevance budget, and explain verified actions.

    Source: §I · p. 1, §II · p. 2

  • Method

    FAIRGate has agents select predefined exposure and calibration policies and plan repairs, while a deterministic validator checks whether the repaired list is deliverable.

    Source: §I · p. 1, B. Problem Setup and Policy Semantics · p. 2, §VII · p. 6

  • Results

    LT-Cov@10 rises from 0.1055 to 0.1282 (21.52% relative gain) over base BPR-MF, while audit-grounded narration reduces unsupported claims to 0.0959.

    Source: A. RQ1: Runtime Policy Selection and… · p. 4, C. RQ4: Audit-Grounded Narration · p. 5, §VII · p. 6

  • Takeaways & Limitations

    The findings support adaptive policy selection with deterministic enforcement and audit-grounded communication for fairer recommendation delivery.

    Source: §VII · p. 6

    The predefined policy library bounds reachable fairness outcomes, so missing policies must be added by system designers before deployment.

    Source: §VI · p. 6

Abstract

from arXiv · show

Fairness-aware recommender systems often encode one fairness objective during training or apply a fixed re-ranking rule after scoring. In deployment, however, the appropriate fairness action can vary across requests because the candidate pool, item metadata, exposure debt, user preference history, and utility budget all change at runtime. We propose FAIRGate, a model-agnostic delivery controller that places an agentic policy-selection layer between a base ranker and the final top-K list. The agent does not define fairness objectives from scratch. It chooses from a predefined policy library covering long-tail exposure, item-group exposure balance, concentration control, calibration protection, and abstention. A deterministic executor then applies the selected policy through constrained repair, and a validator accepts the result only when relevance, feasibility, metadata, calibration, and audit constraints are satisfied. The final language-model component receives only the verified audit trace and produces evidence-linked narration after the ranking decision is fixed. This design gives the agent runtime responsibility while keeping fairness enforcement measurable, bounded, and replayable.

I. INTRODUCTION

Fairness interventions often rely on a fixed action, although the relevant action can differ by recommendation context. FAIRGate makes delivery a bounded runtime-control problem: agents select predefined policies while deterministic checks govern repair and delivery.

I. INTRODUCTION

  • A fixed fairness action may not fit every request: diverse candidate pools can support long-tail repair, while sparse histories may call for calibration protection or abstention.The paper frames fairness as deciding which policy applies, which repair is feasible, and whether the repaired list is ready to deliver.Source: §I · p. 1
  • FAIRGate separates relevance scoring, policy selection, repair planning, deterministic execution and validation, and narration of the verified trace.Its predefined policies cover exposure and calibration goals; the agent selects policies rather than inventing fairness criteria at inference time.Source: §I · p. 1

II. RELATED WORK AND DESIGN GAP

  • Prior work studies exposure fairness, popularity bias, provider and item-side visibility, and multi-stakeholder concerns including utility, representation, and calibration.FAIRGate treats exposure objectives as deployable policies selected and validated for each delivery context, with calibration protection both selectable and a validator condition.Source: §II · pp. 1–2
  • The identified gap is bounded, context-specific policy selection at final delivery, with relevance-budgeted repair and explanations grounded in verified actions.FAIRGate places adaptivity in policy selection while keeping enforcement deterministic.Source: §II · p. 2

III. FAIRGATE

The evaluation is organized around four questions about runtime policy selection, constrained repair, context-sensitive choices, and audit-grounded narration.

A. Research Questions

  • RQ1 asks whether runtime policy selection improves long-tail and item-group exposure satisfaction over base rankers, fixed re-rankers, and fixed-policy gates under the same utility budget.The comparison includes exposure satisfaction and the shared utility-budget condition.Source: A. Research Questions · p. 2
  • RQ2 asks whether validator-constrained repair preserves recommendation utility while improving selected exposure policies.Source: A. Research Questions · p. 2
  • RQ3 asks whether policy choices vary across runtime contexts and explain gains over fixed policy orderings.Source: A. Research Questions · p. 2
  • RQ4 asks whether audit-grounded narration reduces unsupported fairness claims relative to direct LLM explanations.Source: A. Research Questions · p. 2

B. Problem Setup and Policy Semantics

FAIRGate defines policies as measurable objectives with constraints, allowed repairs, and validators, then assigns agents to assess context, select policies, plan repairs, and verify delivery.

B. Problem Setup and Policy Semantics

  • A policy combines a measurable objective, constraints, allowed repair actions, and a validator; the library covers long-tail floors, group exposure balance, concentration caps, calibration protection, and abstention.The agent chooses a policy subset from runtime state, but the validator determines whether the repair is deliverable.Source: B. Problem Setup and Policy Semantics · p. 2
  • The framework evaluates group labels from available metadata such as genre or popularity tier, rather than claiming any one grouping exhausts deployment fairness.Source: B. Problem Setup and Policy Semantics · p. 2

C. System Architecture

  • Agents select policies and propose repairs, while validators check feasibility, relevance, calibration, metadata coverage, and traceability before delivery.This distinguishes agentic choice from deterministic authority.Source: C. System Architecture · p. 2

D. Runtime Agents

  • The fairness context agent summarizes concentration, exposure debt, calibration sensitivity, feasible replacements, and abstention risk from the supplied runtime metrics.Source: D. Runtime Agents · p. 2
  • The policy-selection agent chooses from the fixed library using metric triggers, and the repair planner proposes bounded swaps, relevance floors, priorities, and fallback behavior.Examples include pairing long-tail exposure with calibration protection when relevant tail candidates are available, or abstaining when metadata coverage is too low.Source: D. Runtime Agents · p. 2
  • The validator checks policy satisfaction, utility loss, calibration, metadata coverage, and item-movement records; narration uses only the resulting trace and evidence keys.Source: D. Runtime Agents · p. 2

E. Agent Input and Output Contract

The input-output contract narrows agent decisions to structured choices from predefined policies. Agents can adapt policy selection and plan repair, while deterministic execution and validation bound delivery.

E. Agent Input and Output Contract

  • Each agent receives precomputed metrics, candidate summaries, and policy descriptors and returns a structured object that can be parsed and checked.Source: E. Agent Input and Output Contract · pp. 2–3
  • The architecture assigns agents to interpret runtime context, select policies, and plan repair, with deterministic execution and validation constraining delivery.Source: Figure 1 · p. 3
  • Policy selection asks the agent to choose the smallest policy subset addressing the dominant risk while protecting utility and calibration budgets, and to name metric triggers and validators.The prompt asks which listed policy applies, not whether the list is fair.Source: E. Agent Input and Output Contract · p. 3

F. Verified Repair Objective

FAIRGate scores feasible replacement items by balancing relevance, exposure gains, and predicted calibration drift, then delivers, relaxes, or abstains based on validation. Its MovieLens 1M evaluation covers long-tail exposure, genre-group exposure, and calibration.

F. Verified Repair Objective

  • A repair is accepted only when it improves the selected policy objective and keeps utility loss below ε; validation can instead trigger relaxation or abstention.The replacement score combines relevance, long-tail gain, group exposure-debt reduction, and predicted calibration drift using non-negative weights selected on validation data.Source: F. Verified Repair Objective · p. 3

A. Datasets and Base Rankers

  • MovieLens 1M supports controlled evaluation of long-tail movie exposure, genre-group exposure, and calibration.The evaluation uses timestamp-based leave-last-one-out splitting and computes popularity tiers, genre-group counts, and calibration targets from training data only.Source: A. Datasets and Base Rankers · p. 3

F. Verified Repair Objective

  • The delivery procedure computes runtime state, selects policies using metric triggers, proposes bounded repairs, and validates relevance, policy, calibration, metadata, and trace constraints before delivery.Insufficient feasibility or failed validation can lead to a typed abstention or a lower-priority relaxation followed by revalidation.Source: A. Datasets and Base Rankers · p. 3, F. Verified Repair Objective · p. 3

B. Baselines, Metrics, and Ablations

The evaluation compares fixed and adaptive FAIRGate controls alongside ranking baselines, using utility, exposure, calibration, and agent metrics. The policy library is predefined, while the agent selects among its policies at runtime.

B. Baselines, Metrics, and Ablations

  • The key comparison is fixed-order versus full agentic FAIRGate, isolating runtime policy selection under the same policy library and utility budget.Other baselines include BPR-MF, a static long-tail re-ranker, and an agentic variant without validation.Source: B. Baselines, Metrics, and Ablations · p. 3
  • Evaluation metrics cover ranking utility, long-tail and group exposure, policy-pass rate, calibration divergence, validator rejection, policy agreement, and narration faithfulness.Narration measures include unsupported-claim rate, evidence coverage, audit-faithfulness rate, and sensitive-attribute leakage rate.Source: B. Baselines, Metrics, and Ablations · pp. 3–4
  • The agent input-output contract makes agent steps inspectable and gives the validator a fixed schema to check.Policies are fixed before evaluation, with runtime selection among the policy library.Source: Table I · p. 4, Table II · p. 4

C. Reproducibility Protocol

Thresholds and the utility budget are selected on validation data and frozen before testing. The implementation precomputes runtime metrics, constrains agent inputs and repair actions, and records the resulting audit artifacts.

C. Reproducibility Protocol

  • Thresholds and the utility budget are selected on validation data and frozen before test evaluation, with the same users and candidate pools used across methods.The oracle policy label is the validation policy with the largest exposure-policy gain under the utility budget and is used only as an RQ3 evaluation reference.Source: C. Reproducibility Protocol · p. 4
  • Narration is evaluated separately from ranking by checking whether each atomic explanation claim maps to a typed audit key.Supported keys include policy selection, rule triggers, item movement, validator rejection, relaxation, abstention, and post-gate metrics.Source: C. Reproducibility Protocol · p. 4

D. Implementation Details

  • A metric service computes runtime state before the agent is called, so the LLM does not calculate exposure or calibration from raw logs.The wrapper supplies candidate identifiers, relevance scores, popularity tiers, genre-group labels, and content tags.Source: D. Implementation Details · p. 4
  • The MovieLens policy library defines genre-group exposure balance and dominant-group concentration control, while structured generation uses schema validation and hard triggers.The selection agent is assessed on whether its choice improves the selected exposure objective under the validation-selected utility budget.Source: D. Implementation Details · p. 4
  • The executor chooses feasible near-score replacements that improve the selected policy objective while passing a calibration guard, and logs artifacts for diagnosing failed requests.Logged artifacts include the base list, selected policy object, validator report, and final explanation.Source: D. Implementation Details · p. 4

V. EXPERIMENTAL RESULTS

On MovieLens 1M, agentic FAIRGate raises long-tail coverage while keeping ranking quality close to the fixed-order gate. The comparison also shows a policy-pass advantage alongside a group-equality trade-off.

V. EXPERIMENTAL RESULTS

  • Experiments use a reproducible sample of 1000 MovieLens 1M users, with the same sampled users across methods to manage LLM-call cost and latency.Agentic components use GroqCloud API calls with Llama 3.1 8B.Source: §V · p. 4

A. RQ1: Runtime Policy Selection and Exposure

  • LT-Cov@10 rises from 0.1055 to 0.1282 versus base BPR-MF, a 21.52% relative gain, while agentic FAIRGate retains NDCG@10 close to fixed-order FAIRGate.It has the highest LT-Cov@10 among the static re-rank, fixed-order, and agentic variants.Source: A. RQ1: Runtime Policy Selection and… · pp. 4–5, Figure 2 · p. 5
  • Policy-pass rate is 0.6070 for agentic FAIRGate versus 0.0910 for fixed-order FAIRGate, while Group Gini is lower for fixed-order at 0.6581 versus 0.6672.The fixed-order gate's repeated balance and concentration repairs move genre-group exposure closer to equality, but its fixed priority sequence is less able to satisfy the policy selected for each runtime context.Source: A. RQ1: Runtime Policy Selection and… · pp. 4–5

B. RQ2 and RQ3: Validator and Agent Adaptation

The validator rejects some proposed repairs rather than delivering them when constraints fail, while policy selection adapts to runtime context. Audit-grounded narration is more evidence-faithful than direct LLM explanation on the reported measures.

B. RQ2 and RQ3: Validator and Agent Adaptation

C. RQ4: Audit-Grounded Narration

  • Audit-grounded narration reduces unsupported claims to 0.0959, raises evidence coverage to 0.9129, eliminates sensitive-attribute leakage, and reaches 0.5760 audit-faithfulness.Source: C. RQ4: Audit-Grounded Narration · p. 5

D. Why Agentic Selection Matters

Feasible fairness repairs vary across recommendation requests, so FAIRGate adapts policy choice while keeping delivery subject to deterministic validation. The evaluation is bounded by its movie-recommendation setting, predefined policies, and narration-scoring procedure.

D. Why Agentic Selection Matters

  • Feasible repairs vary by request: candidate pools, user calibration needs, concentration, and metadata gaps differ across contexts.Source: D. Why Agentic Selection Matters · p. 5
  • FAIRGate adapts at policy selection, not enforcement: the agent uses runtime evidence to select predefined policies, while deterministic validation authorizes delivery.Source: D. Why Agentic Selection Matters · p. 5
  • Static re-ranking tests one exposure correction, while fixed-order gating tests deterministic priority; agentic selection tests runtime policy choice among available actions.Source: D. Why Agentic Selection Matters · p. 5

VI. LIMITATIONS AND FUTURE WORK

  • MovieLens 1M results are evidence from a controlled movie-recommendation setting, not a general claim across domains; its metadata lacks rich provider-side marketplace structure.The passage identifies datasets with clearer creator, seller, or provider labels as a next evaluation step.Source: §VI · pp. 5–6
  • FAIRGate cannot create fairness objectives at runtime; missing policies must be added by designers, and domains may need different policies, metadata standards, and escalation rules.Source: §VI · p. 6
  • Narration results depend on claim parsing and audit-key matching, which may undercount partially supported natural-language claims.The passage proposes stronger claim decomposition and human review of borderline claims as future work.Source: §VI · p. 6

VII. DISCUSSION AND CONCLUSION

FAIRGate makes fairness delivery adaptive at policy selection while keeping enforcement deterministic, measurable, and replayable. Results show improved LT-Cov@10 and policy-pass rate over fixed alternatives while preserving most ranking utility, with validator-constrained repair and audit-grounded narration addressing deliverability and unsupported claims.

VII. DISCUSSION AND CONCLUSION

  • FAIRGate selects policies and proposes repair plans for the current recommendation context, then relies on deterministic validation to decide whether the repaired list is deliverable.Source: §VII · p. 6
  • The separation gives the agent adaptive responsibility while keeping enforcement measurable and replayable.Source: §VII · p. 6
  • FAIRGate improves LT-Cov@10 and policy-pass rate over fixed alternatives while preserving most ranking utility, and audit-grounded narration sharply reduces unsupported fairness claims versus direct LLM explanation.Validator-constrained repair prevents treating every exposure gain as deliverable.Source: §VII · p. 6

Related papers

Loading 2610.30359v1…