BiasMix-Finance: Post-Generation KYC Guardrails for LLM Portfolio Advice

Gaurav Kukreja; Parul Kukreja; Mohammed Abraar; Raj Dandekar; Rajat Dandekar; Sreedath Panat

Source-linked AI summary

BiasMix-Finance: Post-Generation KYC Guardrails for LLM Portfolio Advice

Gaurav Kukreja, Parul Kukreja, Mohammed Abraar, Raj Dandekar, Rajat Dandekar, Sreedath Panat

arXiv:2608.28646v1Aug 14, 2026cs.AI

TL;DR

LLMs can produce portfolio allocations that violate externally specified KYC-style caps, creating a need for auditable enforcement. The paper uses a model-agnostic verify-and-repair pipeline and BiasMix-Finance (Mini) to evaluate biased generations; projection eliminates final violations while keeping repairs bounded, within the study’s constrained scope.

Problem
LLM portfolio recommendations can violate customer-specific numeric constraints on risk, fees, and diversification, while prompting and reasoning modes cannot guarantee compliance on every run.
Method
The paper treats JSON allocations as auditable drafts, validates hard caps, and applies nearest-feasible convex projection within a reproducible BiasMix benchmark.
Results
Across models and prompting modes, deterministic projection returns feasible portfolios when the cap set is feasible while minimizing deviation from the model’s proposal.
Takeaways & Limitations
The verify-and-repair pattern is model-agnostic and asset-agnostic, extending structured LLM decision checks beyond portfolio allocation.
Takeaways & Limitations
The study evaluates constraint compliance rather than portfolio optimality and does not model expected returns, transaction costs, taxes, or utility-based objectives.

Abstract

from arXiv · show

Large language models (LLMs) can generate plausible-sounding ETF portfolios while silently violating basic KYC-style constraints on risk, fees, and diversification. This is especially problematic in agentic multi-turn advisory systems, where each draft recommendation can become an action unless guarded by an auditable enforcement layer. We study a model-agnostic, asset-agnostic post-generation guardrail pipeline: (i) enforce a strict JSON allocation schema, (ii) validate allocations against numeric caps, and (iii) when violations occur, deterministically project the output to the nearest feasible portfolio via a convex quadratic program (QCQP). We introduce BiasMix-Finance (Mini), a compact stress-test benchmark for constrained decision-making under biased LLM generations, with a 16-ETF universe, three investor profiles, and eight bias prompts. Across three models and three inference modes (direct, critique, self-consistency), first-pass generations violate at least one cap in 47.6-85.7% of test cases (67.2% pooled), but the convex projection layer reduces final feasibility violations to 0% while requiring only a small correction distance (test pooled median D=||w*-w0||_2=0.066), indicating that the guardrail typically preserves the intent of the original allocation. We report violation rates and correction distances with confidence intervals, and paired model comparisons with multiple-testing correction. To support reproducibility, we release the dataset, prompts, caps, and code in our public GitHub repository.

1 Introduction

The paper frames LLM portfolio allocations as auditable drafts that must satisfy mechanically checkable KYC-style caps, then evaluates deterministic repair under biased generations. Its methodology combines reproducible stress testing with convex projection and statistical reporting across models and inference modes.

Motivation and framework: The evaluation targets biased generations that can produce concentrated, high-volatility, or otherwise non-compliant allocations under preference-shaping contexts.The study operationalizes suitability as hard numeric caps on risk, fees, and diversification.
Motivation and framework: LLM-generated allocations are treated as auditable drafts subject to explicitly specified, mechanically checkable KYC-style numeric caps.The framework logs the draft, violations, and repaired output for governance and oversight.
BiasMix-Finance stress test: 72 scenario instances combine three risk profiles, eight bias recipes, and three random seeds over a fixed 16-ETF universe.This design isolates model and prompting responses to identical constraints and bias contexts.
Deterministic repair: The guardrail formalizes volatility, fee, HHI, single-asset, and sector caps and repairs violations by computing the nearest feasible portfolio with a constrained quadratic program.QCQP is used as a deterministic repair operator rather than introduced as a new optimization technique.
Evaluation: The study reports per-cap violation rates with Wilson 95% confidence intervals, bootstrap intervals for correction distances and metric deltas, and paired comparisons with Wilcoxon tests plus BH-FDR correction.These procedures support reproducible comparisons across splits, models, and inference modes.

2 Problem Setup

The problem setup represents an LLM allocation as JSON weights over an ETF universe, checks it against hard portfolio caps, and projects violations to the nearest feasible portfolio. Figure 1 summarizes this verify-and-repair flow from draft generation through acceptance or deterministic correction.

Universe: The controlled universe contains n = 16 ETFs, with portfolio weights represented by w ∈ R^n.The weights describe the allocation over the fixed ETF universe.
Scenario and caps: Each scenario combines a risk profile with a bias context and caps annualized volatility, weighted-average fees, HHI concentration, single-asset weight, and sector weight.The bias context is intended to nudge the model toward potentially non-compliant allocations.
Draft validation: The LLM emits a draft portfolio w0 as JSON with nonnegative weights summing to 1, after which portfolio summary metrics are computed.The supplied setup identifies risk as σ(w) and evaluates the draft against the scenario caps.
Feasibility: A draft is violating when w0 ∉ C, where C is the set of portfolios satisfying all specified caps.The feasible set provides the deterministic admissibility criterion.
Post-generation enforcement: When any cap fails, the system computes the nearest feasible portfolio by convex projection and otherwise accepts the draft directly.Figure 1 depicts validation, metric computation, conditional repair, and direct acceptance as the end-to-end pipeline.
Post-generation enforcement: Correction distance is defined as D = ∥w*−w0∥2, measuring the Euclidean adjustment from the draft to the repaired allocation.Projection is used to enforce feasibility under convex constraints.

3 Related Work

Prior work covers financial LLM capabilities, safety and alignment tools, and auditing or constrained-output methods. This paper differs by focusing on hard quantitative constraint compliance for portfolio weights rather than text accuracy or conversational policy enforcement.

LLMs for finance: Financial LLMs such as BloombergGPT and FinGPT demonstrate capability on financial NLP tasks, whereas this study evaluates numeric constraint compliance for portfolio-weight outputs.The distinction is between text-generation performance and allocation feasibility.
Guardrails and constrained outputs: Safety toolkits, classifiers, and alignment methods primarily target unsafe content or conversational policies rather than quantitative portfolio caps.Examples include NeMo Guardrails, Llama Guard, RLHF, and Constitutional AI.

4 Methodology

BiasMix-Finance evaluates LLM portfolio drafts under representative KYC-style caps and repairs violations through deterministic nearest-feasible projection. Its controlled benchmark spans risk profiles, bias recipes, models, and inference modes.

Universe and scenarios: The benchmark contains 72 scenarios across three risk profiles, eight bias types, and three random seeds in a fixed 16-ETF universe.The train/dev/test design supports controlled evaluation of bias-induced constraint violations.
Caps: Risk-profile caps constrain annualized volatility, weighted-average expense ratio, HHI concentration, and maximum single-asset and single-sector weights.The cap values are representative KYC-style suitability heuristics rather than jurisdiction-specific regulations.
Guardrail pipeline: When a strict-JSON draft violates any cap, the pipeline projects it to the nearest portfolio in the feasible set defined by those hard constraints.The projection is solved as a convex QCQP with a small variance regularizer λ = 2 × 10^-3.
Inference strategies: The evaluation compares direct generation, critique, and self-consistency across three LLM backends, with all modes sharing strict parsing, retries, validation, and projection.Self-consistency samples K = 5 candidates and selects the one with the lowest pre-projection violation.

5 Experimental protocol and statistical framework

The experimental protocol evaluates each scenario under a model-and-mode configuration, measuring draft compliance, repair magnitude, parsing, and metric changes with uncertainty estimates and corrected paired comparisons.

Endpoints: Each scenario is evaluated under a model-and-mode configuration using first-pass violation rate and correction distance D = ||w*-w0||_2 as primary endpoints.The protocol also records parse-failure rate and before/after changes in σ, WAER, and HHI.
Uncertainty estimation: Violation rates use Wilson 95% confidence intervals, while correction distances and metric deltas use bootstrap 95% confidence intervals.These procedures quantify uncertainty for both compliance frequencies and continuous repair outcomes.
Statistical comparisons: Paired model comparisons on correction distance use Wilcoxon signed-rank tests with Benjamini–Hochberg false-discovery-rate correction.The correction controls multiplicity across the paired comparisons.

6 Results

First-pass LLM portfolios frequently violate hard caps, especially in tight risk regimes, whereas deterministic projection achieves feasibility with corrections whose size tracks regime tightness.

First-pass violations: 1.0 is the Conservative first-pass violation rate across all test scenarios, while Aggressive falls from 0.476 to 0.048 with self-consistency.The Conservative feasible region is intentionally extremely tight because several caps intersect on the simplex.
Projection and feasibility: 0% final feasibility violations are achieved across models and modes after projection, with Wilson 95% CI [0.845, 1.0] for final pass.Strict JSON prompting with up to R = 3 retries yielded zero parse failures on the held-out test split.
Correction distance: D is smallest for Aggressive, moderate for Moderate at roughly 0.018–0.185, and largest for Conservative at roughly 0.220–0.462.Lower D indicates a draft was closer to feasibility before projection, while larger D signals stronger constraint-forced rebalancing.
Model comparisons: Model differences in correction distance are significant in direct and critique but not self-consistency for the reported held-out comparisons.For example, gpt-5-nano versus Llama 3.3 70B has pFDR = 0.0292 in direct and 0.0098 in critique.

7 Implications for Agentic Financial AI

The paper frames deterministic verification and repair as a composable governance layer for agentic financial advisors. It supports multi-turn correction, auditability, configurable stress testing, and feedback adaptation while preserving the LLM’s role as an intent generator.

Agentic integration: A deterministic verify-and-repair interface can compose with agent loops, ensuring externally visible actions satisfy hard KYC-style constraints.The pipeline separates probabilistic proposal generation from enforcement and can return a feasible portfolio at every turn.
Governance and auditing: Numeric caps and repair records provide an auditable, configurable governance layer for regulated recommendations.Recommendations can be logged with the draft, violations, repair, and correction distance D, while policies remain explicit and versionable.
Feedback adaptation: Corrected draft-to-feasible pairs support prompt adaptation and model adaptation while retaining repair as a backstop.These pairs encode how recommendations must change to satisfy policy and can be used for fine-tuning or preference learning.
Stress testing: BiasMix-Finance can serve as a regulatory sandbox for varying profiles, biased instructions, and cap configurations while measuring violations, correction distance D, and allocation stability.Its deterministic, policy-driven enforcement enables repeated stress tests as governance becomes tighter or looser.
Overall implication: The resulting guardrail is presented as a composable agent tool that turns probabilistic proposals into policy-compliant actions for multi-turn and autonomous workflows.This extends the contribution beyond post-processing static drafts.

8 Discussion, limitations, and future work

The discussion distinguishes reliable constraint compliance from portfolio optimality and positions the verify-and-repair pattern as transferable beyond portfolio allocation. It also identifies broader deployment boundaries and extensions involving richer constraints, objectives, estimates, and human evaluation.

Discussion: Prompting modes can reduce first-pass violations, but only post-generation projection deterministically enforces feasible allocations when the cap set is feasible.The projection is model-agnostic, auditable, and minimizes deviation from the model’s proposal.
Limitations: The study tests KYC feasibility and minimal adjustment rather than portfolio optimality, excluding expected returns, transaction costs, taxes, and utility objectives.It uses a fixed 16-ETF universe as a controlled stress test and does not include real-user or advisor-in-the-loop studies.
Generalization: The verify-and-repair pattern is presented as transferable to constrained financial actions and other structured decisions requiring explicit, externally specified rules.Examples include budget allocation, eligibility checks, policy-compliant trade lists, disclosure and fee compliance, and underwriting or regulatory recommendation drafts.
Future work: Future work includes larger universes, liquidity and turnover constraints, regime-aware covariance estimates, return-aware objectives, and human evaluation of intent preservation, trust, and usability.These extensions retain hard caps while broadening the modeled decision setting.

9 Conclusion

The study presents BiasMix-Finance as a reproducible stress-test and concludes that deterministic post-generation guardrails can enforce hard portfolio constraints while preserving allocation structure. It frames the contribution as an auditable methodology rather than a new optimization technique, with solver and evaluation robustness examined across controlled scenarios.

9 Conclusion: Prompting modes can reduce first-pass violations but do not guarantee compliance, motivating post-generation verification and repair.The test results compare direct, critique, and self-consistency modes using first-pass violation rates before projection.
9 Conclusion: BiasMix-Finance treats LLM allocations as auditable drafts evaluated against reproducible bias, profile, model, and prompting-mode conditions.The benchmark uses controlled test scenarios and visualizes first-pass violations and sector allocations before and after projection.
9 Conclusion: Sector comparisons use draft-versus-projected allocations to assess whether repair preserves model intent while enforcing hard caps.Points near the y = x line indicate closer sector-level preservation after projection.
9 Conclusion: A convex nearest-feasible projection provides deterministic repair for allocations violating externally specified portfolio caps.The paper positions the QCQP as a repair operator within a methodology and evaluation framework, not as a new optimization technique.
9 Conclusion: Solver robustness checks report stable feasibility and correction distance across settings, with retry logic improving robustness across models and scenarios.The implementation combines conic QCQP solving with optional QP fast paths and fallback behavior.

D.2 Solver parameter ablation and robustness

The QCQP projection remains robust across numerical tolerances and iteration budgets, with SCS reliably solving the conic formulation and OSQP serving as an optional fast path for pure QPs.

Across tolerance and iteration settings, QCQP projection remains robust, with SCS reliably solving the conic form induced by quadratic caps.OSQP primarily serves as an optional fast path when an instance reduces to a pure QP.

E.1 Implementation notes

The implementation combines strict machine-checkable outputs, deterministic validation and repair, reproducible logging, and statistical evaluation across models, modes, and splits. The controlled 16-ETF design supports interpretable diagnosis, while scaling to hundreds of ETFs and time-varying risk models remains future work.

Output and validation: Strict JSON parsing requires exactly weights and explanation keys, retries failed parses up to R=3 times, and renormalizes nonnegative weights to sum to 1.Singleton lists containing the required object are unwrapped.
Metrics: The pipeline reports first-pass violations, correction distance D, volatility, fee burden, concentration, parse failures, final feasibility, and end-to-end success.Final feasibility requires w*∈C, while end-to-end success requires both successful parsing and final feasibility.
Evaluation: Test metrics with confidence intervals provide the main evidence of generalization, while paired Wilcoxon tests compare models across modes and splits with Benjamini–Hochberg FDR control at q = 0.10.The analysis comprises 27 pairwise tests, or 36 including an all-splits aggregate view.
Experimental design: The fixed 16-ETF universe supports interpretable parsing and constraint-violation patterns, stable covariance and sector mappings, and systematic variation across prompts and risk profiles.The design is intended for controlled diagnosis rather than unrestricted universe scale.
Scope: The projection guardrail can extend to larger universes with fees, sector mappings, and covariance estimates, but hundreds of ETFs and time-varying risk models remain future work.The stated open issues include scaling behavior, universe-dependent concentration effects, and covariance estimation error.