Source-linked AI summary
Cloud and On-Premises Deployment of Uzbek Legal RAG via Targeted Retriever Fine-Tuning
Tatul Danielyan, Mariam Avetisyan, Hrant Davtyan
TL;DR
The paper addresses Uzbek legal QA under cloud cost limits and on-premises sovereignty, hardware, and latency constraints, where general benchmarks are insufficient. It builds domain-specific retrieval and end-to-end evaluations, validates an independent LLM judge, and compares deployment choices. The main result is that retriever fine-tuning is the higher-leverage investment, while generator fine-tuning is costly and unnecessary in this setting.
Problem
Uzbek legal RAG lacks task-specific evaluation, while deployment regimes impose conflicting cost, sovereignty, hardware, and latency constraints.
Method
The study builds two Uzbek legal benchmarks, validates an independent-family LLM judge, and evaluates retrievers and generators for cloud and on-premises deployment.
Results
Retriever fine-tuning cheaply closes roughly half the open/proprietary gap, making the retriever the higher-leverage investment under deployment constraints.
Takeaways & Limitations
Fine-tune the retriever once and rent or swap the generator rather than fine-tuning it for changing legal acts and long-context workloads.
Takeaways & Limitations
The study covers one language, domain, and jurisdiction, uses modest benchmarks, and has single-expert annotations without inter-annotator agreement.
Abstract
from arXiv · showhide
Deploying large language models for legal question answering raises challenges that general-purpose leaderboards do not capture, particularly for low-resource languages and under hard operational constraints. We report on building and operating a retrieval-augmented (RAG) legal assistant for Uzbek that must run in two regimes: a managed cloud service that maximizes answer quality within a per-token cost ceiling, and an on-premises deployment for clients whose legal data may not leave their infrastructure, restricting us to open-weight models on limited local hardware under latency constraints. Because no evaluation existed for this setting, we build two domain benchmarks: a retrieval benchmark of 178 expert-annotated legal queries with gold provision spans, and an end-to-end benchmark of 504 expert-curated question--answer pairs scored by an LLM judge whose ratings we validate against human judgments and against an independent-family judge. Applying these benchmarks under each regime, we find the open-versus-proprietary gap is small and cheaply closed by fine-tuning. Therefore, we train UTE-1, which is a state-of-the-art text embedder among open models for Uzbek. We also demonstrate that closing the performance gap via fine-tuning is both impractical due to the intensive hardware demands of long-context legal Q\&A and unnecessary, given that legal acts change frequently. We support this by reporting a negative result from a QLoRA experiment. We distill practical guidance for similar deployments, drawn from a system serving real users in production. We release our benchmarks, evaluation code and the fine-tuned embedder (UTE-1) \href{https://metric-ai-lab.github.io/Uzbek-Legal-RAG/}{at this https URL} to support future work on low-resource legal NLP.
1 Introduction
The paper presents a production Uzbek legal RAG assistant designed for cloud and on-premises deployment, where model choice must satisfy different cost, sovereignty, hardware, and latency constraints. It addresses the absence of Uzbek legal evaluation by introducing task-specific benchmarks and a retriever-first optimization approach.
- Motivation: The production assistant serves Uzbek legal users in a Turkic language that remains under-served by NLP resources.It is grounded in Uzbekistan’s national legal corpus, lex.uz.
- Deployment setting: Cloud deployment permits any model but imposes a per-token cost ceiling, whereas on-premises deployment requires open weights, local hardware, and low latency.These regimes create conflicting optimization objectives for the same assistant.
- Evaluation gap: General multilingual leaderboards do not reliably predict Uzbek legal retrieval or grounded answer quality.Existing Uzbek leaderboards measure generic language ability rather than grounded legal question answering.
- Contributions: The study contributes a transferable decision procedure that treats cloud and on-premises model selection as constrained optimization problems.One evaluation suite drives the selection of two deployed model stacks.
- Contributions: The authors release a 178-query retrieval benchmark, a 504-item end-to-end QA benchmark, evaluation code, and a human-validated judge protocol.The benchmarks include gold provision spans and cross-checking against an independent-family judge.
- Contributions: Fine-tuning the open retriever reaches a state-of-the-art open result for Uzbek, while generator fine-tuning is costly and unnecessary under the study’s constraints.The paper therefore recommends fine-tuning the retriever and renting or swapping the generator.
2 System and Deployment Setting
The assistant uses a provision-level hybrid RAG pipeline that combines dense and lexical retrieval over a large, structured legal corpus. Its long retrieved contexts drive both cost and latency, while retriever failures cannot be repaired by the generator.
- Pipeline: The pipeline segments the national legal corpus into provision-level chunks, prepends hierarchical headings, embeds them, and indexes them in Weaviate.At query time, questions are normalized or translated before retrieval.
- Pipeline: Hybrid search fuses dense cosine similarity with lexical matching because legal queries depend on exact terms such as article numbers and statute names.Pure semantic search can miss these exact legal cues.
- Model roles: The embedder and generator have asymmetric failure modes: missing the governing provision leaves the generator unable to recover a supported answer.A capable generator mainly reads, synthesizes, and attributes the text it receives.
- Constraints: Production retrieval uses k=50 provisions spanning roughly 55k–90k tokens because cross-references and expanded statutory tables require broad context.This long-context regime shapes both inference cost and latency.
3 Benchmarks and Evaluation Methodology
The evaluation framework combines complementary Uzbek retrieval and end-to-end legal QA benchmarks with an independently selected LLM judge validated against experts. It measures both model retrieval and the deployed pipeline under realistic legal-query conditions.
- Benchmark design: The study builds two complementary benchmarks because no public benchmark targets Uzbek legal RAG and evaluator reliability is a central concern.The benchmarks assess retrieval and complete question answering.
- Retrieval benchmark: The retrieval benchmark contains 178 human-written, expert-annotated legal questions with exact gold provision targets.It reports top-5 accuracy, defined by whether at least one gold provision appears among the five highest-ranked chunks.
- Retrieval benchmark: The retrieval evaluation also uses 127 Uzbek MTEB queries over 1,263 candidates with nDCG@5, complementing the in-domain legal set.The two retrieval sets are independent in content and scoring and together cover 305 queries.
- End-to-end benchmark: The end-to-end benchmark contains 504 legal question–answer pairs spanning 22 legal domains, with most items composed or reviewed by legal experts.It includes expert-authored, production-sampled, and reviewed LLM-drafted answers.
- End-to-end benchmark: Each QA item runs through the full production pipeline and is scored for completeness, contextual accuracy, hallucination-free behavior, legal comprehension, and overall quality.This evaluates deployed-pipeline performance rather than models in isolation.
- Judge validation: The independent judge achieves Spearman ρ=0.7032 and mean absolute error 1.29 against expert ratings on 52 questions.Using a different judge family reduces the risk of self-preference bias.
4 Cloud Deployment: Quality Within a Cost Ceiling
Cloud model selection maximizes end-to-end quality under a cost ceiling, yielding a stack built from gemini-embedding-001 and gemini-3-flash. The selected generator lies on the cost–quality Pareto frontier and is corroborated by positive production feedback.
- Selection objective: Cloud selection maximizes quality subject to a per-token cost ceiling, with any model admissible in principle.The practical ceiling is $3 per million input tokens and $15 per million output tokens.
- Embedder: gemini-embedding-001 leads retrieval with 0.961 top-5 in-domain accuracy and 0.906 nDCG@5 on MTEB.Because embedding costs are a small fraction of total cloud cost, it is deployed for the cloud service.
- Generator: gemini-3-flash is the strongest in-budget generator at 0.850 overall and is paired with gemini-embedding-001 for cloud deployment.More expensive reference models do not surpass it on this task.
- Cost and quality: gemini-3-flash sits at the quality-maximizing vertex of the cost–quality Pareto frontier while scoring higher than the compared frontier APIs.It is 2.1×, 4.0×, and 6.6× cheaper than grok-4.3, gpt-5.4, and claude-sonnet-4-6 respectively.
- Production corroboration: The deployed cloud stack serves approximately 30k questions per month, with 78% positive in-product votes from 375 of 479 votes.This production signal corroborates the offline QA ranking used to select the stack.
5 On-Premises Deployment: Sovereignty, Open Weights, and Latency
On-premises deployment excludes API models and selects open-weight systems that satisfy hardware and latency constraints. The authors deploy UTE-1 with Qwen, favoring retriever adaptation while rejecting generator fine-tuning as impractical and unnecessary.
- Deployment constraints: On-premises selection maximizes quality subject to a latency budget among open-weight models that fit client hardware.
- Retriever adaptation: Fine-tuning arctic-embed-l-v2.0 with synthetic translations and in-domain questions produces UTE-1 to close the open retriever’s legal-retrieval gap.
- Generator selection: 0.724 vs. 0.671 with UTE-1 makes gemma-4-31b-it the strongest open generator overall, but Qwen is deployed because its MoE design activates approximately 3B rather than 31B parameters per token.
- Latency-constrained deployment: The 3B-active Qwen model produces first tokens and full responses faster than dense Gemma while generating more reasoning tokens.
- Latency-constrained deployment: Higher throughput and concurrency at lower cost justify deploying Qwen on-premises despite a modest quality concession.
- Why generator fine-tuning was rejected: Generator QLoRA fine-tuning is impractical because retrieved contexts span 55–90k tokens, making training, evaluation, and hyperparameter search slow and expensive.
- Retriever adaptation: 0.916 top-5 accuracy is UTE-1’s in-domain retrieval result, up from 0.876 for arctic-embed-l-v2.0 and closing roughly half the proprietary gap.
- Why generator fine-tuning was rejected: Fine-tuning is also unnecessary because changing legal content requires retrieval, while weights-baked legal knowledge becomes stale as laws are amended.
6 Discussion: Lessons Learned
The paper argues that low-resource legal RAG deployments should prioritize retriever quality, domain-specific evaluation, and operational safeguards. It also identifies script switching and sovereignty-related observability gaps as concrete deployment challenges.
- Retriever fine-tuning is a higher-leverage investment than generator fine-tuning in low-resource legal RAG.The open/proprietary retrieval gap was smaller and cheaper to close, while generator fine-tuning was costly and of doubtful value.
- Domain benchmarks and a human-validated judge were prerequisites for reliable model selection because general leaderboards did not predict legal QA quality.
- Script switching between Uzbek Latin and Cyrillic emerged as a deployment hazard that general benchmarks miss.
- 5.08% of law citations were affected by script switching before normalization, and deterministic Uzbek transliteration resolved all observed broken citations.
- On-premises sovereignty requirements restrict models and remove telemetry, increasing the importance of correct offline evaluation before deployment.
7 Conclusion
The conclusion presents a production Uzbek legal RAG system evaluated across cloud and on-premises regimes. Its central transferable lesson is to fine-tune and reuse the retriever while renting or replacing the generator.
- The system served production users across cloud and on-premises regimes using benchmarks and a validated judge for constrained model selection.
- In low-resource legal RAG, fine-tune the retriever once and reuse it, while renting or swapping generators instead of fine-tuning them.
Limitations
The study is limited to Uzbek statutory law in one jurisdiction, with modest benchmark sizes that constrain how confidently results and rankings can be generalized.
- The study covers one language, one domain, and one jurisdiction, so its specific model rankings may not transfer.
- The benchmarks contain 178 retrieval queries and 504 QA items, while the judge-validation set contains 52 items.
- The authors avoid over-interpreting small score differences and will release the data so others can extend the evaluation.
Ethical Considerations
The paper frames legal RAG as an information-retrieval aid rather than legal counsel and emphasizes grounded, current-law answers, confidentiality, and responsible data handling.
- The system is not a substitute for qualified legal counsel, and interfaces provide citations and disclaimers against treating outputs as definitive legal advice.
- RAG grounds answers in retrieved statutes and supports tracking current law, while dedicated evaluation measures hallucination risk.
- On-premises deployment keeps sensitive legal documents on client infrastructure to honor data-sovereignty and confidentiality requirements.
- UzLib is used only as a coarse Uzbek-language admissibility filter, with final selection based on the end-to-end legal QA benchmark.
B Cost–Quality Analysis
The analysis compares production inference cost with QA quality for cloud deployment, then distinguishes that hosted-cost view from on-premises serving-throughput constraints.
- Standard nonbatch API inference cost is measured across the full 504-query QA benchmark, excluding the separate evaluation-judge overhead.The judge costs approximately $22 per model and is excluded from deployment comparison.
- The cost–quality Pareto frontier contains gemma-4-26b-a4b, gemma-4-31b, gemini-3.1-flash-lite, and gemini-3-flash.
- Cloud regime: gemini-3-flash is the cloud quality-maximizing frontier vertex, scoring higher while costing 2.1×, 4.0×, and 6.6× less than grok-4.3, gpt-5.4, and claude-sonnet-4-6.
- On-premises regime: On-premises selection cannot use hosted per-token cost as its binding axis because owned-hardware economics depend on serving throughput and active parameters.This explains why qwen3.6-35b-a3b is deployed on-premises instead of gemma-4-31b.
C Reproducibility Details
The production system combines structured legal chunking and script-aware preprocessing with hybrid retrieval and answer postprocessing.
- Legal provisions are segmented with hierarchical headings, while oversized statutory tables are split for indexing and re-expanded when any chunk is retrieved.
- Questions undergo normalization and, where needed, translation before dense and lexical retrieval, with answers postprocessed to match Latin or Cyrillic script.
- Training and evaluation sets are de-duplicated using both semantic and fuzzy lexical matching.