Source-linked AI summary
From Prompt Rules to Golden Data Model: Production-Scale Generation of Personalized, Compliant Pharmaceutical Claims
Suyash Mishra, Qiang Li, Subhabrata Adak, Upender Phogat, Srikanth Patil, Satyanarayan Pati
TL;DR
Pharmaceutical claim generation must be accurate, compliant, and audience-specific, while rule-based prompting can yield mismatched or low-relevance outputs. The paper evaluates a structured, retrieval-time approach using a Golden Data Model. The context pipeline improves alignment across intended indication, content type, and audience, while the conclusion reports improved compliance and reduced weekly cost.
Problem
The paper asks whether moving compliance and personalization signals from prompt-level rules to structured retrieval-time data can improve pharmaceutical claim generation.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The authors conclude that retrieval-time claim metadata is a more scalable and effective architecture for regulated content generation than expanding prompt-level rules alone.
Source: §6 · p. 6The study uses approved claims from selected products and a specific market context, cannot release the full production dataset, and bases efficiency gains on production-log analysis.
Source: §7 · p. 7
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly used to generate domain-specific commercial & medical content (e.g., claims, statements, and promotional phrases) in regulated industries such as pharmaceuticals. However, ensuring its regulatory compliance, factual accuracy, and audience-specific personalization remains a major challenge. We report a production case study asking whether moving compliance and personalization signal from prompt-level rules into retrieval time structured context improves pharmaceutical claims generation, using Composer, a deployed internal tool built on a Golden Data Model (GDM) of 1,867 pharmaceutical claims spanning four products. The GDM integrates claim metadata, references, audience labels, material type, usage history, Match Text Variant (MTV) links, and a six dimension quality score (6C). Across six product-audience segments, the context pipeline improves intended indication (+29%), intended content type (+62%) and intended target audience (+67%), 6C quality score (+86%), and claim reuse (+34%). Production log analysis over the deployment period shows a 29.5% reduction in weekly API cost. We report this as descriptive deployment evidence. This work is a systems and data architecture contribution: it shows that in a regulated, high-stakes domain, restructuring claim data for retrieval-time filtering is a more scalable lever than expanding prompt-level rules, and documents the engineering path data cleaning, graph linking, and quality scoring.
1 Introduction
Pharmaceutical claims must be grounded in approved sources, compliant with regulations, and tailored to their audience, yet rule-based prompting often produces mismatched or low-relevance outputs. The paper asks whether moving compliance and personalization signals into structured retrieval-time data can improve generation.
1 Introduction
- Claims must be factually grounded in approved sources, regulatory-compliant, and tailored to audiences such as healthcare professionals or patients.These requirements make pharmaceutical content generation particularly challenging.Source: §1 · p. 1
- The rule-based baseline often produced mismatched audiences, low relevance, and overuse of inactive or general claims.It relied on prompting and strict templates.Source: §1 · p. 1
- The study asks whether shifting compliance and personalization signals from prompt-level rules to structured retrieval-time data improves pharmaceutical claim generation.The question includes both output quality and production efficiency in a deployed system.Source: §1 · p. 1
- The proposed Golden Data Model transforms fragmented claims into a structured, graph-based representation with quality-aware filtering.The context pipeline relocates claim-level guardrails and prohibited-word constraints from static product-level prompt text to retrieval-gated metadata.Source: §1 · pp. 1–2
2 Related Work
Prior work covers pharmaceutical content management, biomedical retrieval, and verification benchmarks, but the paper identifies less attention to controlled, personalized generation under strict regulatory requirements. Its contribution is an empirical comparison of rule-based and context-based generation in a real-world setting.
Pharmaceutical content management
- The paper compares rule-based and context-based generation in a real-world setting, focusing on a claim-centered schema and audience-conditioned generation.It distinguishes this contribution from product descriptions that do not establish platforms’ internal generation architectures.Source: §2 · p. 2
- Prior research examined prescription-drug advertising content and the distinction between promotion and scientific exchange.The cited work includes Frosch et al. (2007) and Kessler (1991).Source: §2 · p. 2
Retrieval-augmented generation in biomedical NLP
- Standard RAG systems lack domain-specific regulatory constraints and structured claim validation, while RAG alone does not enforce audience-specific stylistic transformation.The paper identifies controlled, personalized generation under strict regulatory requirements as receiving less attention in biomedical NLP.Source: §2 · p. 2
Verification and hallucination evaluation
- The cited verification and safety benchmarks address distinct tasks and do not directly validate pharmaceutical claim generation.Examples include scientific claim verification, visual question answering, proposition-level evaluation, and multimodal safety assessment.Source: §2 · p. 2
3 Methodology
The methodology organizes claims into a graph-based, quality-aware model and retrieves ranked claims conditioned on audience and other context. It combines claim cleaning, relational linking, quality scoring, alignment annotation, and audience-based filtering.
3 Methodology
- A periodic pipeline clusters claims using keywords and embeddings to identify duplicates, near-duplicates, and contradictions.It performs high-volume upserts from Veeva Vault and other sources into a central library.Source: §3 · p. 3
- Claims are linked to references, materials, and audience metadata in a relational graph that records derivation and utilization.The graph representation is stored in a Graph DB.Source: §3 · p. 3
- The 6C framework scores claims along Clarity, Completeness, Consistency, Connectivity, Compliance, and Context.The framework translates relational evidence into a strength score for how firmly a claim is tied to supporting scientific text.Source: §3 · p. 3
- Internal US medical-content writers not involved in system development judged indication, content-type, and target-audience alignment against campaign briefs.Judgment was binary and blinded to pipeline.Source: §3 · p. 3
- Claims are filtered and ranked by target audience, region, and usage patterns, with 6C scores, status, and recency informing final reconciliation actions.The listed actions are Merge, Suppress, and Flag.Source: §3 · p. 3
- The context pipeline retrieves top-ranked claims, applies semantic filtering and ranking, and conditions generation on high-quality, audience-specific claims.The rule-based comparison relies on predefined instructions, strict citation and verbatim-use constraints, and templates.Source: §3 · p. 3
4 Dataset and Experimental Settings
The evaluation uses claims from four pharmaceutical products and assesses compliance, personalization, quality, and deployment metrics. The schema combines cleaned claim records with tagging and quality-score layers.
4 Dataset and Experimental Settings
- The dataset contains more than 1,800 claims from Hemlibra, Ocrevus, Ocrevus Zunovo, and Vabysmo, spanning 11 categories.Claims retrieved from Veeva carry metadata including references, usage history, and audience annotations; Claude Sonnet 4.5 is the production evaluation model.Source: §4 · p. 3
- The claim-level schema integrates cleaned raw data with LLM-derived tagging layers and 6C quality scores.It also extracts tone signals and constraint-aware key phrases for downstream generation.Source: Table 1 · p. 4
- Legacy claim data had ambiguous provenance and heterogeneous quality, including contradictory, duplicate, malformed, and low-value claims.These failure modes introduce retrieval ambiguity and reduce downstream claim personalization effectiveness.Source: Figure 3 · p. 4
- Evaluation covers approved-claim use, relevant-claim use, content-type alignment, target-audience alignment, and average 6C quality.Deployment metrics include success rate, latency, request tokens, total cost, and tokens per request.Source: §4 · p. 4
5 Main Results
The context-based pipeline achieves 100% alignment with intended indication, content type, and target audience, while rule-based results were lower. It also improves claim-variant use and quality, alongside lower weekly API costs.
5 Main Results
- Intended indication alignment rose from 71% to 100%, content-type alignment from 38% to 100%, and target-audience alignment from 33% to 100%.These alignment rates are guaranteed by the context pipeline’s retrieval filters; Figure 4 identifies audience and content alignment as the most pronounced gains.Source: §5 · pp. 4–5, Figure 4 · p. 6
- 34%: MTV claim utilization increased after the context pipeline, against 42.69% availability and 17.68% use in materials overall.The Pre/Post comparison reports MTV utilization within its evaluation sample.Source: §5 · p. 5, Table 3 · p. 5
- 29.5%: weekly API cost fell, while average cost per request increased from $0.0234 to $0.0316.Request volume fell from 5,485 to 2,871; the authors hypothesize fewer regeneration requests but did not measure regeneration rate directly.Source: Table 4 · p. 5, §5 · p. 6
- Average 6C scores were higher with the context-based approach, indicating improved clarity, consistency, and contextual relevance.Table 5 compares quality and usage before and after the Golden Data Model.Source: §5 · p. 5, Table 5 · p. 6
- Structured contextual data supports selecting more relevant claims, adapting to audience needs, and maintaining compliance through data filtering rather than prompt constraints.The results contrast this with rule-based constraints that enforce correctness but fail to capture contextual nuances needed for personalization.Source: §5 · p. 6
6 Conclusion
The paper presents a claim-centered Golden Data Model and context-based generation pipeline for regulated content. Its production evaluation reports improved compliance alongside reduced total weekly cost, while average cost per request rises.
6 Conclusion
- The context-based pipeline improves compliance while reducing total weekly cost, despite a higher average cost per request.Source: §6 · p. 6
- The work formalizes a 6C claim-quality algorithm and evaluates alignment, reuse, latency, token usage, request success, and API cost.Source: §6 · p. 6
- The paper argues that structured context provides a more effective LLM solution for regulated industries than explicit rule instructions.Source: §6 · pp. 6–7
7 Limitations and Ethic Statements
The study is limited to selected pharmaceutical products and market context, and its efficiency findings come from production logs. The authors describe safeguards and review processes while identifying other regulated domains for future evaluation.
7 Limitations and Ethic Statements
- The dataset covers approved claims from selected products and a specific market context, and the full production dataset cannot be released.Representative cleaned examples, schema-level records, scoring rubrics, system descriptions, logs, and tool videos are provided instead.Source: §7 · p. 7
- Efficiency gains are based on production-log analysis and should be read as deployment-level evidence, not a model-intrinsic efficiency claim.Source: §7 · p. 7
- Experiments currently focus on pharmaceutical compliance; generalization to financial services and manufacturing remains future work.Source: §7 · p. 7
- Generated content passes through existing MLR review, and guardrails are checked against approved claim text at generation time.Provenance and approval status are preserved during canonicalization.Source: §7 · p. 7
A Appendix
The appendix collects supplementary material on framework performance, the Golden Data Model, source-data governance issues, and the legacy prompting setup.
A Appendix
- Supplementary materials include an ablation study of token usage, latency, and monetary cost, plus 6C rationale, scoring, and loss functions.Source: A Appendix · p. 9
- The appendix provides a property dataset with an example claim record in Golden Data Model format and a comprehensive claims graph view.Source: A Appendix · p. 9
- A failure-mode matrix summarizes observed data-quality and governance pain points in Veeva claim records that motivate the Golden Data Model.Source: A Appendix · p. 9
- The legacy rule-based Composer pipeline's product-level rule JSON and prompt instruction lists are included.Source: A Appendix · p. 9
A.1 Ablation study: framework perform in terms of token count, latency, and monetary cost
The deployment comparison reports lower weekly API spend alongside fewer requests and substantially less response-token generation, despite higher average cost per request. The authors characterize the spend result as descriptive and note that efficiency depends on more than per-call pricing.
A.1 Ablation study: framework perform in terms of token count, latency, and monetary cost
- Logged per-request costs vary by provider and model tier, with Gemini 2.5 Flash examples costing 0.38–0.64¢ for 1.8–1.9K tokens.Anthropic endpoint examples range from 1.46–3.93¢ for 4–11.9K tokens; Table 8 identifies provider/model tier as a cost factor motivating cost-aware routing.Source: §A.1 · p. 9, Table 8 · p. 9
- Weekly API cost fell 29.5% from 128.70 to 90.69 as request volume dropped from 5,485 to 2,871, despite 35% higher average cost per request.The authors hypothesize fewer regeneration requests but did not directly measure regeneration rate, active-user counts, or campaign mix.Source: §A.1 · p. 9
- Response-token generation fell 65.4%, while request success rate rose from 99.2% to 100%.Source: §A.1 · p. 9
- Business-level efficiency depends jointly on API cost per request, request volume, success rate, and perceived output quality.Source: §A.1 · p. 9
A.2 Ablation study: the 6C Scoring Framework
The 6C framework scores claims across six quality dimensions, combining graph and metadata signals with LLM evaluations. Its scoring pipeline gates compliance and context availability on earlier checks.
A.2 Ablation study: the 6C Scoring Framework
- Connectivity, compliance, context availability, clarity, completeness, and consistency make up the six 6C scoring dimensions.Each dimension is scored between 0.0 and 1.0.Source: §A.2 · p. 9
- Compliance scoring follows connectivity, and context availability is assessed only after connectivity and compliance are established.The implementation grants full compliance credit when connectivity is at least 0.9 and the lifecycle state is Approved or Approved State.Source: §A.2 · pp. 9, 11
- Claims are retrieved with linked references and materials from the Neo4j graph database, alongside Match Text Variant data.The scoring pipeline uses those linked records as inputs to its graph-based checks.Source: §A.2 · pp. 9, 12
- Clarity, completeness, and consistency are evaluated through parallel LLM calls, while context availability is computed deterministically from metadata availability.Consistency evaluates alignment across claim text, Match Text Variant, and material text.Source: §A.2 · p. 10
- The final 6C score is the weighted arithmetic mean of the six dimension scores.The experiments assign equal weights to the six dimensions.Source: §A.2 · p. 10
A.3 Ablation Study: Post-Deployment Composer Usage Patterns
The post-deployment analysis asks whether Composer can better match audience, intent, indication, and content format while reducing API costs. It compares rule-based and Golden Data Model context-based pipelines and reports increased audience matching alongside observed production usage patterns.
A.3 Ablation Study: Post-Deployment Composer Usage Patterns
- The comparison contrasts prompt instructions and approved-claim rules with Golden Data Model context including references, audience labels, MTV links, 6C scores, and LLM-derived tags.The tags address hallucination prevention and prohibited wording.Source: §A.3 · p. 17
- The post-deployment logs cover 16,963 requests and 146 million tokens, with estimated monthly API cost of $543.39.The temporal pattern suggests human usage, with generally lower activity Wednesdays and higher activity Mondays, Tuesdays, and Fridays.Source: §A.3 · p. 17, Figure 8 · p. 17
- 67% higher target-audience match rate accompanied the context-based pipeline, which also required fewer repeated requests.The logs were analyzed for model-side measures and user-side activity and request behavior.Source: §A.3 · p. 17