AISPA: User-Centric System Prompt Auditing for Large Language Model Applications

1 Introduction

• System prompts are rarely disclosed or independently reviewed, leaving no concrete prompt-specific guidance on whether instructions serve users or meet protective standards.Existing governance standards and prompt-security work address AI governance and defenses against malicious inputs, but the paper says they do not scrutinize the instructions themselves.Source: §1 · p. 2
• AISPA evaluates individual prompt segments across eight user-centered dimensions, classifying each as protective (+1) or problematic (−1).The dimensions range from identity transparency and information truthfulness to harm prevention and fairness; they draw on safety guidelines, regulatory frameworks, expert review, and the Universal Declaration of Human Rights.Source: §1 · p. 2
• From 2024 to 2026, protective instructions more than doubled; 98.9% of products had at least one, roughly 40% had a problematic instruction, and 23.9% covered all eight dimensions.These findings show that protective instructions are widespread, but comprehensive coverage is rare and problematic instructions remain common.Source: §1 · p. 3
• The audit combined human-LLM collaboration with expert review of system prompts from 88 real-world AI products spanning general-purpose, specialized, and agent-based applications.The product types included chatbots, coding assistants, autonomous agents, search and research tools, and specialized applications.Source: §1 · p. 3
• Organizations differed substantially: Anthropic averaged 62.3 protective entries and near-zero problematic instructions per product, while some organizations showed the inverse pattern.The audit also surfaced gray-area directives, including parasocial dependency cues, identity concealment tactics, and politically motivated content-policy relaxations.Source: §1 · p. 3

Why prompts matter

• System prompts persist across conversations and define a product’s model behavior, yet remain invisible to users and can undermine user interests despite model-level guardrails.Developers may keep prompts private to protect against adversarial exploitation and proprietary design disclosure, but users cannot assess whose interests the instructions prioritize.Source: §2 · p. 3

Why auditing is needed

• Incidents expose missing safeguards in crisis response [2], factual accuracy [3], and behavioral constraints [4], while other prompts enabled harmful engagement [5][6] and ethical bypass for profit [7].The section also notes that a Shanghai court found legal responsibility flows to whoever controls the system prompt.Source: §2 · pp. 3–4

Auditing proposal

• The authors propose confidential pre-deployment third-party audits against shared user-protection criteria, with public certifications for compliant prompts and remediation reports for failures.The proposed criteria include manipulation, conflicts of interest, safety safeguards, and fairness; certification could strengthen accountability and give users and regulators a signal for oversight.Source: §2 · p. 4

User-centric normative foundation

• AISPA centers auditing on harms users may face from AI systems and grounds its taxonomy in fundamental rights articulated in the UDHR.This distinguishes the framework from conventional safety approaches focused primarily on system security and external adversarial attacks.Source: §3 · p. 4

Unified taxonomy

• The eight dimensions treat protective and problematic instructions as opposing evaluations along the same axis, rather than using separate taxonomies.For example, Identity Transparency covers both disclosing that the system is AI and concealing its AI nature; operationalization and annotation are described in Section 4.Source: §3 · p. 5, Figure 2 · p. 5

Auditing dimensions

• Identity Transparency and Truthfulness & Information Integrity address AI disclosure, non-impersonation, truthful communication, calibrated uncertainty, and information integrity.These dimensions are linked respectively to UDHR Articles 19 and 1, and Articles 19 and 27.Source: §3 · p. 5
• Privacy & Data Protection, Tool/Action Safety, and User Agency & Manipulation Prevention assess data practices, operational safeguards, and respect for user autonomy.Their normative foundations include UDHR Articles 12 and 3 for privacy, Articles 3 and 12 for tool safety, and Articles 18 and 19 for autonomy.Source: §3 · p. 6
• Unsafe Request Handling, Harm Prevention & User Safety, and Fairness, Inclusion & Neutrality cover refusal of harmful requests, risk mitigation, and non-discriminatory treatment.They are grounded respectively in UDHR Articles 29(2) and 30; Articles 3, 5, and 25; and Articles 1, 2, and 7.Source: §3 · p. 7

4 Human-in-the-loop Workflow for System Prompt Auditing

• AISPA formalizes a practical workflow for conducting third-party system-prompt audits in a traceable and operationally rigorous manner.Source: §4 · p. 7

4.1 Auditing Guidelines

• Auditors analyze continuous prompt spans, usually single sentences, and assign each evidence-bearing span a dimension and polarity.Consecutive sentences expressing the same intent can form one span; Figure 3 illustrates the process.Source: §4.1 · p. 7, Figure 3 · p. 7
• Auditing covers non-core instructions and ethically or safety-relevant clauses attached to core instructions, but excludes pure core functionality.A core instruction is excluded when its removal would fundamentally impair the product’s intended functionality.Source: §4.1 · p. 8
• Under the same eight dimensions, spans receive +1 when protective, −1 when problematic, or no label when irrelevant to a dimension.The guidelines are shared with the LLM pre-annotator and human annotators to promote consistency.Source: §4.1 · p. 8

4.2 Auditing Pipeline

• The LLM pre-annotator decomposes prompts into candidate spans, proposes dimension–polarity labels with rationales, and provides scalable, provisional coverage.A span may be relevant to multiple dimensions; the model bootstraps the audit, while its proposals remain subject to human review.Source: §4.2 · p. 8, Figure 4 · p. 9
• After calibration, trained annotators independently retain only text-grounded proposals and may add missed spans or assignments.They reject proposals that lack textual support or over-interpret the span.Source: §4.2 · p. 9
• Three domain experts review retained spans, resolve disagreements, and keep problematic labels only with unanimous agreement.This higher threshold reflects the greater reputational consequences of falsely labeling a product problematic.Source: §4.2 · p. 9, Figure 4 · p. 9

5 Auditing System Prompts in Commercial AI Systems

• The audit asks whether commercial system prompts comprehensively protect users and whether they include instructions that may work against user interests.It applies the AISPA framework to prompts from a diverse set of commercial AI systems.Source: §5 · p. 9

5.1 Dataset

• The dataset contained prompts from 88 real-world products across chatbots, coding assistants, autonomous agents, search and research tools, and specialized applications.Prompts came from six open-source GitHub repositories; maintainers confirmed curation procedures, and the researchers validated same-product prompt overlap across repositories.Source: §5.1 · pp. 9–10
• Six annotators independently screened spans after calibration, achieving pairwise inter-annotator agreement of 0.933; three experts adjudicated final labels.The audit produced 2,420 entries from 1,818 unique spans: 2,346 protective and 74 problematic; 44 entries across 29 spans were flagged as gray areas and analyzed separately.Source: §5.1 · p. 10

5.2.1 Overall Trends

• From 2024 to 2025, average prompt length rose from approximately 9K to over 30K characters and protective instructions more than doubled from 15.0 to 38.4.Problematic instructions peaked in 67% of products in 2025-Q1, fell to 19% by Q3, then rose to 29% in Q4; roughly one in three products was still flagged at year-end.Source: Figure 5 · p. 10, §5.2.1 · pp. 10–11
• Protective instructions appeared in 98.9% of products (87 of 88), but only 23.9% (21 of 88) covered all eight dimensions, and 38.6% (34 of 88) had a problematic entry.Among products covering all dimensions, 14 were general-purpose chatbots; protection and problematic instructions often coexisted in the same prompt.Source: Figure 6 · p. 11, §5.2.1 · p. 11
• Truthfulness and User Agency appeared in over 90% of products, while Unsafe Request Handling and Privacy appeared in only around 60%.User Agency had the highest problematic prevalence at 18.2%, followed by Truthfulness at 14.8%; autonomous agents and coding assistants often directed action without user confirmation.Source: Figure 6 · p. 11, §5.2.1 · p. 11

5.2.2 Organization-Level Trends

• Anthropic averaged 62.3 protective and 0.1 problematic entries per product; Venice averaged 2.0 protective and 3.0 problematic entries.Amazon and Cline averaged 42.0 and 39.5 protective entries, respectively; GitHub and Cursor had moderate protective counts alongside comparatively more problematic instructions.Source: §5.2.2 · pp. 11–12, Figure 7 · p. 12
• Across six generations, protective entries rose 3.1× for Anthropic, 3.3× for OpenAI, and 4.2× for xAI.Anthropic and OpenAI had only one isolated problematic instance each; xAI’s count reached zero by Grok-4 before rising to 2 in Grok-4.2.Source: Figure 8 · p. 12, §5.2.2 · pp. 12–13

The Gray Area: Borderline Cases in System Prompt Auditing

• 29 risky spans across 15 products (44 audit entries) were excluded from the main dataset because they raised user-protection concerns without being clearly problematic.The panel grouped these spans into four recurring patterns for separate analysis; the authors argue that such trade-offs are a feature of deployed AI systems, not an edge case.Source: §6 · pp. 13–14

Pattern 1: Human Mimicry and Identity Deception

• Human-mimicry instructions, including concealing the AI’s nature or deflecting identity questions, may make conversation natural while undermining D1 identity transparency.The section gives examples ranging from adopting a human persona to scripted evasive answers.Source: §6 · p. 13

Pattern 2: Parasocial Dependency Cues

• Parasocial cues cast the AI as a friend and good listener while discouraging users from ending the conversation, raising D5 user-agency concerns, especially for vulnerable users.One prompt explicitly says never to end or suggest ending the conversation.Source: §6 · p. 13

Pattern 3: User-Initiated Permission Override

• User-initiated permission overrides can support autonomy and customization, but without safeguards may let users bypass content protections (D6) and harm-prevention defaults (D7).The risk is that users can be given the ability to disable safety defaults, not that the system disobeys them.Source: §6 · p. 13

Pattern 4: Politically Charged or Unrestricted Content Policies

• Politically framed directives, blanket permissions for adult or offensive content, and redefined age terms can differentiate products while weakening D7 harm prevention and D8 fairness.The section characterizes these policies as deliberate product-differentiation choices that also weaken protections.Source: §6 · pp. 13–14

7 Related Work

• User experiences of confusion, perceived unfairness, and unexpected behavior can indicate harm, motivating user-engaged and human-integrated auditing approaches (Zhu et al., 2025; Deng et al., 2025; Shen et al., 2022; Delgado et al., 2023).WeAudit supports end users in surfacing experiential concerns throughout the audit workflow (Deng et al., 2025).Source: §7 · p. 14
• Black-box access can limit scrutiny, whereas greater transparency and white-box or beyond-black-box access enable deeper examination (Cen and Alur, 2024; Casper et al., 2024).Source: §7 · p. 14
• Audit design, methodology, institutional context, and legal, political, and economic structures shape whether audits can function as meaningful accountability mechanisms (Birhane et al., 2024; Terzis et al., 2024).Source: §7 · p. 14
• Because auditing often lacks clear, enforceable standards, prior work proposes standards to improve process reliability and transparency.Source: §7 · p. 14
• Outsider Oversight calls for institutionally designed third-party oversight beyond developer-controlled tools to represent users and affected parties (Costanza-Chock et al., 2022; Raji et al., 2022).Related work also identifies persistent power asymmetries (Raji et al., 2022, Urman et al., 2024).Source: §7 · p. 14

• AISPA applies an eight-dimension taxonomy and efficient human-in-the-loop workflow to the first audit of 3,249 instructions from 88 real-world AI product system prompts.Source: §8 · p. 14
• Roughly 40% of commercial systems contain at least one instruction that works against user interests, despite protective instructions becoming more common over time.Coverage of protective instructions remains uneven across products and organizations.Source: §8 · p. 14
• Gray-area instructions resist binary classification and reveal tensions between user autonomy and platform safety, and between organizational interests and the obligation to serve users.Source: §8 · pp. 14–15
• AISPA builds a foundation for greater transparency and accountability for commercial AI applications and could ultimately help ensure safe deployment of advanced AI systems.System prompts are a consequential but largely ungoverned layer of deployed AI behavior.Source: §8 · p. 15

9 Disclaimer

• The researchers used publicly available leaked or community-disclosed GitHub prompts for academic research and did not extract, reverse-engineer, or solicit proprietary prompts.Their stated purpose was to advance understanding of AI system transparency and user protection.Source: §9 · p. 15

10 Limitations

• Because prompts came from leaked or community-disclosed GitHub repositories rather than official channels, they may differ from production versions and reflect practices at the time of disclosure.Prompts may have been updated, modified, or replaced since disclosure.Source: §10 · p. 15
• Cross-repository comparisons found high overlap among prompts for the same product, partially mitigating concerns about dataset reliability.The authors note that this overlap supports reliability, but does not establish that prompts match current production versions.Source: §10 · p. 15
• The leaked-prompt corpus may over-represent products easier to extract or with technically engaged users and under-represent systems with stronger prompt protections.The authors therefore caution that the corpus may not represent all deployed AI systems.Source: §10 · p. 15
• Despite that representativeness concern, the dataset covers popular products from leading AI companies, so its findings may still reflect important system-prompt trends.Source: §10 · p. 15

A Product Prompt Collection

• The human audit covered 88 prompts across five categories: 37 general-purpose chatbot products, 25 coding assistants, 9 autonomous agents, 5 search and research products, and 12 specialized-application prompts.Source: A Product Prompt Collection · pp. 18–19
• This diverse collection enables comparison of how different types of AI systems encode safety practices and problematic instructions in their system prompts.Source: A Product Prompt Collection · pp. 18–19

B Data Validation

• Maintainers reported checking prompt validity through repeated extractions across independent chat sessions and verifying consistency and plausibility for the associated product.These maintainer interviews were one of two complementary strategies used to verify collected prompts.Source: B Data Validation · p. 19
• Cross-repository matches were found for 50 of 88 prompts; 22 exceeded 70% overlap, 12 exceeded 90%, and 8 were near-identical at ≥99%.Lower overlap could reflect differences in extraction scope, prompt version, or model generation; the 38 unmatched prompts came from products collected by only one repository and relied on maintainer verification. Figure 10 reports each matched product’s best overlap.Source: B Data Validation · pp. 19–20, Figure 10 · p. 20

C LLM Pre-Annotation Prompt

• The pre-annotator was run once per dimension, with instructions to favor recall, identify relevant spans, assign +1 or −1, and provide a rationale.The prompt required exact text copied from the document and asked annotators to omit spans that were not relevant to the dimension.Source: C LLM Pre-Annotation Prompt · p. 21

D Human Annotation Platform

• The web-based annotation platform let reviewers accept, reject, or edit proposed spans and add missed spans directly from the prompt.Its interface displayed prompts with color-coded dimension highlights, review cards with proposed polarity and rationale, dimension filters, and per-prompt progress tracking.Source: D Human Annotation Platform · p. 21

E Supplementary Analyses

• The supplementary analyses characterize the audit dataset’s structure and are not independent findings.The category- and prompt-level analyses provide context for the main findings.Source: E Supplementary Analyses · p. 22

E.1 Category-Level and Prompt-Level Patterns

• The lowest protection-balance products, with scores as low as −20%, all have prompts under 4 KB.Short prompts do not contain more problematic instructions per se; they lack enough protective instructions to offset even a small number of problematic directives.Source: §E.1 · p. 22, Figure 13 · p. 23

E.2 Dimension-Level Analyses

• 441 of 1,818 spans address two or more dimensions, with D6 (Unsafe Request Handling) and D7 (Harm Prevention) co-occurring most often (109 times).Source: §E.2 · p. 23
• D1 (Identity Transparency) and D3 (Privacy) rarely co-occur with other dimensions, indicating they are typically addressed in standalone instructions.Source: §E.2 · p. 23
• Figure 14’s word clouds show distinctive vocabularies: D1 foregrounds identity disclosure, D3 data protection, and D4 operational primitives, supporting the taxonomy’s distinct dimensions.Examples include person and knowledge for D1, data, sensitive, and secret for D3, and file, command, and code for D4.Source: Figure 14 · p. 23, §E.2 · pp. 23–24