PaperlayerView exact revision on arXiv

Source-linked AI summary

GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphs

Ahmed Sohair Khan, Estrid He, Chenglong Ma, Monica Wachowicz, Elham Naghizade

arXiv:2609.12448v1cs.CLcs.CR

TL;DR

Sensitive attributes can emerge from weak cues across ordinary posts, but those inferences are difficult to audit at the level of supporting records. GraphProfiler represents histories as source-linked personal knowledge graphs and resolves predictions to cited records and source texts; across the evaluated benchmarks, it remains competitive while providing compact provenance.

  • Problem

    Weak cues distributed across ordinary posts can collectively reveal sensitive attributes, making privacy risk difficult to locate within a user’s history.

    Source: §1 · p. 1

  • Method

    GraphProfiler organizes posts as a source-linked personal knowledge graph and resolves attribute predictions to cited graph records and original source texts.

    Source: §1 · pp. 1–2

  • Results

    Across SynthPAI and PANDORA, GraphProfiler remains competitive with strong baselines while resolving successful inferences to a median 2.6–3.1% of a user’s posts.

    Source: §1 · p. 2, §8 · p. 9

  • Takeaways & Limitations

    Source-linked graphs make latent privacy leakage auditable and identify a small set of posts for targeted inspection or rewriting.

    Source: §7 · p. 8, §8 · p. 9

    Removing cited evidence may not eliminate inference because long histories can contain redundant or substitute cues, so practical mitigation may require iterative auditing and reevaluation.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Sensitive attributes such as age, income, and occupation can be inferred from user-generated content by aggregating indirect cues across many ordinary posts. LLM-based profilers can perform this aggregation automatically and with high accuracy, which makes large-scale personal attribute inference a major privacy threat. Existing LLM-based profilers, however, offer limited insight into which specific posts, concepts, and relationships made an inference possible, which is key to targeted privacy mitigation, i.e., redacting or rewriting only the few posts that actually leak an attribute, rather than perturbing entire histories. We introduce GraphProfiler, an auditable LLM-based profiler that represents each user's post history as a source-linked personal knowledge graph where nodes and edges trace back to the originating post and resolves attribute predictions to cited graph records and source texts. GraphProfiler reaches 86.7% attack success rate on the eight-attribute SynthPAI benchmark, within two points of strong text-only baselines, and 84.6% on PANDORA, while citing supporting evidence for over 98% of predictions. Our controlled ablation experiments provide evidence that the cited posts contribute to attack success, as removing them reduces the attack success rate substantially more than removing an equal number of random posts.

1 Introduction

Sensitive attributes can emerge from weak cues scattered across ordinary posts, while existing profilers provide limited insight into which records support an inference. GraphProfiler uses a source-linked personal knowledge graph to trace predictions to posts and make targeted review possible.

1 Introduction

  • Weak cues in ordinary posts can collectively reveal sensitive attributes, leaving privacy risk distributed across a user’s history rather than concentrated in direct disclosures.Examples include work, rent, university, family roles, and budgeting habits.Source: §1 · p. 1
  • GraphProfiler represents posts as a personal knowledge graph and resolves predictions to cited graph records and their original source texts.The graph organizes dispersed cues into entities and relationships and provides a persistent evidence structure reusable across attribute queries and audits.Source: §1 · pp. 1–2
  • Because posts are processed individually, new content can be merged into the graph without re-indexing the full history, and only affected neighborhoods need re-querying.Each post’s graph contribution retains source-level provenance and can be added, replaced, or removed independently.Source: §1 · p. 2
  • A median 2.6–3.1% of a user’s posts support successful inferences, and removing cited posts lowers attack success more than matched random removal.The introduction also reports that GraphProfiler completes inference for every user in both benchmarks while providing source-linked provenance.Source: §1 · p. 2

2 Related Work

Prior work studies attribute inference, knowledge-graph-based reasoning, and provenance, but the paper distinguishes its focus on persistent, source-linked evidence for auditing latent attribute inference.

Author profiling and attribute inference

  • Prior author-profiling work recovers demographic, geographic, and personal attributes from text, while LLM profilers extend this threat at scale (Tigunova et al., 2020a; Gjurkovi´c et al., 2021; Harrigian, 2018; Tigunova et al., 2020b; Staab et al., 2024; Du et al., 2026).These systems can highlight snippets but do not build a persistent, source-linked evidence structure; TRACE-RPS rewrites histories without identifying the original posts contributing most to inference (Yan et al., 2026).Source: §2 · pp. 2–3

Knowledge graphs and GraphRAG

  • Unlike GraphRAG and privacy work on graph extraction, this paper uses a graph to retrieve indirect cues across texts and preserve source-linked provenance (Edge et al., 2024; Liu et al., 2025).Prior work also found structured knowledge representations can strengthen de-anonymization attacks in social networks.Source: §2 · p. 3

Faithfulness, attribution, and provenance

  • Rather than judging whether a rationale sounds plausible, GraphProfiler validates provenance through support audits, evidence-only inference, and cited-source removal.This frames provenance testing as functional intervention rather than plausibility assessment (Qi et al., 2024).Source: §2 · p. 3

3 Task and Threat Model

The task is to infer sensitive attributes from a user’s text history and return evidence alongside each prediction. The threat model is dual-use: structured cues support inference but can also identify records for privacy defense.

3 Task and Threat Model

  • The task infers sensitive attributes from a user’s posts or comments and returns both a prediction and an evidence set.The histories are denoted D_u = {t_1, . . . , t_n}, with each t_i a short post or comment.Source: §3 · p. 3
  • SynthPAI covers eight attributes, while PANDORA evaluates age and sex; the graph is both an attack substrate and a way to identify records for rewriting, generalization, or removal.The profiler should remain competitive with existing attacks while identifying evidence responsible for its inferences.Source: §3 · p. 3
  • Attack success rate counts exact, semantically equivalent, or attribute-granularity-compatible predictions as successful, using the same matching policy for all methods.Compatibility includes age ranges containing the labeled age and hierarchical location or place-of-birth matches at a correct coarser geography.Source: §3 · p. 3
  • Provenance analysis measures evidence compactness, support, sufficiency, functional importance, and semantic retention after source removal.Semantic retention uses SBERT cosine similarity as a coarse proxy.Source: §3 · p. 3

4 Methodology

GraphProfiler indexes posts as individually traceable source files, builds a privacy-oriented graph, retrieves attribute-specific evidence, and resolves cited records back to their original texts.

4.1 Input Representation

  • Posts are represented as separate source files with stable identifiers, preserving post-level provenance and separating whether an attribute is inferable from which posts support it.This allows a retrieved graph record to be traced to a particular post rather than an aggregated history.Source: §4.1 · pp. 3–4

4.2 Personal Knowledge Graph Construction

  • The personal knowledge graph retains privacy-relevant entities and relationships, then adds community reports to provide both local and summary-level evidence.The extraction stage preserves user-specific distinctions among related mentions instead of applying global ontology-level canonicalization.Source: §4.2 · p. 4

4.3 Attribute-Targeted Inference

  • Attribute-specific queries retrieve relevant graph context, and each profile entry records a predicted value, confidence, evidence mode, cited graph references, and rationale.The evidence mode is a diagnostic self-report, not a correctness filter.Source: §4.3 · p. 4

4.4 Evidence Resolution

  • The resolver maps cited table records through the graph index to graph entries and source files, rather than accepting model-generated filenames.The provenance object is retrieval-conditioned, not exhaustive proof of all evidence in the history; later evaluation separates correctness, evidence support, and the effect of removal.Source: §4.4 · pp. 4–5

5 Experimental Setup

GraphProfiler is evaluated on a synthetic benchmark covering eight personal attributes and on real-world Reddit histories labeled for age and sex. The experiments compare it with direct text-to-profile and agentic profiling baselines.

Datasets

  • SynthPAI evaluation uses 700 labeled attribute–user pairs from 269 users, covering eight personal attributes.The evaluation set is filtered to match prior baseline comparisons.Source: §5 · p. 5
  • PANDORA evaluation covers 208 users and 93,591 comments, with age and sex labels derived from Reddit flair metadata.The subset is filtered for label stability, English comments, duplicate removal, direct self-disclosure leakage, and comment-count bucketing.Source: §5 · p. 5

Baselines

  • GraphProfiler is compared with FTI, a direct text-to-profile LLM baseline, and AutoProfiler, an agentic profiling baseline.All methods use Azure OpenAI GPT-4o and the same prediction evaluation pipeline and matching policy.Source: §5 · p. 5

6 Results

GraphProfiler achieves competitive profiling performance while linking predictions to source evidence. Across audits and ablations, cited evidence is supportive and concentrated, and removing it reduces inference more than matched random removal.

6.1 Profiling Performance

  • 86.7% overall ASR on SynthPAI is close to FTI (88.7%) and AutoProfiler (88.3%); PANDORA overall ASR is 84.6% with 100.0% prediction coverage.On the shared PANDORA subset, GraphProfiler reaches 85.3% ASR versus FTI's 94.8% and AutoProfiler's 82.2%; full-evaluation coverage is 100.0%, 96.6%, and 69.2%, respectively.Source: §6.1 · pp. 5–6

6.3 Evidence Support and Sufficiency

  • 69.4% of cited PANDORA evidence versus 16.9% of random evidence is judged supportive; SynthPAI rates are 88.3% versus 17.5%.The PANDORA provenance diagnostics compare cited evidence with same-count random evidence.Source: Figure 3 · p. 6, §6.3 · p. 6
  • 85.0% of cited evidence annotations are rated supportive versus 33.0% for random evidence, with 72.0% human-human and 77.0% human-LLM agreement.The human validation uses 100 evidence sets; the LLM judge rates cited and random evidence 76.0% and 18.0% supportive on those cases.Source: Figure 4 · p. 6, §6.3 · p. 7

6.2 Evidence Resolution

  • 99.1% of SynthPAI and 98.8% of PANDORA predictions resolve to source-level evidence, with median cited-source ratios of 2.6% for age and 3.1% for sex.This concentration supports targeted mitigation by modifying a small subset of posts rather than an entire history.Source: §6.2 · p. 6

6.4 Functional Importance of Evidence

  • 73.3% PANDORA ASR remains after full cited-source removal versus 88.3% after matched random removal; SynthPAI rates are 66.4% versus 80.9%.The corresponding SBERT losses are 2.8 versus 0.2 points on PANDORA and 9.3 versus 6.3 points on SynthPAI; SBERT is only a coarse semantic-retention proxy.Source: Figure 5 · p. 7, §6.4 · p. 7

6.3 Evidence Support and Sufficiency

  • 51.9% PANDORA and 58.2% SynthPAI ASR from cited texts alone exceed random-evidence controls at 7.5% and 4.9%, respectively.Evidence-only inference need not match full-graph inference because some attributes require graph structure to organize distributed weak cues.Source: §6.3 · p. 7

6.5 Graph-Level Risk Structure

  • Cited graph records cluster around relevant evidence neighborhoods: age, education, occupation, and sex draw strongly on corresponding entity types, while other attributes use auxiliary cues.Reported auxiliary cues include financial habits, cultural heritage, place references, and personal experiences; graph-level themes show risk can arise without direct disclosure.Source: Figure 6 · p. 7, §6.5 · p. 8

6.6 Qualitative Case Study

  • A SynthPAI income inference links budgeting, part-time work, and student-status cues to graph entities and cited relationships; removing cited evidence changes LOW income to MEDIUM.The prediction remains LOW after random same-count removal, illustrating functional importance in this case.Source: Figure 7 · p. 8, §6.6 · p. 8

7 Discussion

The discussion connects compact source-linked evidence trails to targeted inspection or rewriting, and situates them in privacy leakage distributed across ordinary posts.

7 Discussion

  • On PANDORA, the median cited-to-total source-file ratio is below 3%, making cited posts candidates for targeted inspection or rewriting.The removal study does not estimate population-level mitigation effectiveness.Source: §7 · p. 8
  • The PKG makes it possible to inspect which posts contributed, how cues connect, and whether removing a source changes the inference.This shifts privacy auditing from plausibility assessment toward targeted intervention.Source: §7 · p. 8
  • Repeated ordinary posts leave organized semantic traces around life stage, finances, location, and identity; making them visible is a first step toward user and defender control.The discussion contrasts these traces with explicit disclosures.Source: §7 · pp. 8–9

8 Conclusion

GraphProfiler represents social-media histories as source-linked personal knowledge graphs and resolves sensitive-attribute predictions to graph records and source texts. Across SynthPAI and PANDORA, it remains competitive with strong baselines while offering compact, high-coverage provenance for auditing latent leakage.

8 Conclusion

  • GraphProfiler resolves sensitive-attribute predictions to graph records and source texts, making the supporting parts of a user’s history inspectable.It represents social-media histories as source-linked personal knowledge graphs.Source: §8 · p. 9
  • Across SynthPAI and PANDORA, GraphProfiler remains competitive with strong baselines while producing compact, semantically meaningful, high-coverage provenance connected to predictions.Source: §8 · p. 9
  • Source-linked personal knowledge graphs are dual-use: they can support sensitive-attribute inference while making latent privacy leakage auditable.The conclusion argues for tools that identify records enabling inference and support targeted, iterative mitigation.Source: §8 · p. 9

Limitations

The study is an audit-oriented privacy study rather than a complete mitigation system, and its results have important limits in evidence scope, evaluation, and data.

Limitations

  • Removing cited evidence may not eliminate inference because long histories can contain redundant or substitute cues; prediction coverage also does not establish complete privacy-risk detection.Incorrect predictions can miss genuine leakage, and failure to recover an attribute does not establish that it is non-inferable.Source: Limitations · p. 9
  • Retrieved graph context is selected rather than exhaustive, and extraction faithfulness of graph entities and relationships lacks independent large-scale annotation.Source: Limitations · p. 9
  • Sensitivity to alternative PKG construction methods remains untested, and without a matched chunk-level RAG baseline the marginal contribution of graph structure remains unsettled.Source: Limitations · p. 9
  • SynthPAI uses synthetic histories, PANDORA’s flair-derived proxy labels may be noisy or temporally unstable, and the implementation relies on GPT-4o throughout.Future work should test open-weight models, expand human validation, evaluate multilingual settings, and study privacy-preserving transformations over cited evidence.Source: Limitations · p. 9

Ethical Considerations

GraphProfiler is presented as a dual-use privacy-risk study intended for assessment and anonymization research, with use in adversarial contexts raising ethical concerns. Its evaluation and released artifacts follow stated access and data-handling restrictions.

Ethical Considerations

  • The same graph representation can make profiling auditable while also strengthening sensitive-attribute inference attacks.Source: Ethical Considerations · p. 9
  • The stated intended use is privacy risk assessment and anonymization research; adversarial deployment raises serious ethical concerns beyond the study’s scope.Source: Ethical Considerations · p. 9
  • The study does not identify or contact users, display usernames or sensitive messages, or redistribute raw comments, identifiers, or per-user records.Released code and prompts contain no PANDORA text or user-level data.Source: Ethical Considerations · p. 9

A Evaluation Details

The appendix specifies evaluation matching and coverage, describes the filtered PANDORA subset, and expands the evidence-resolution and audit results across both benchmarks.

A Evaluation Details

  • Evaluation accepts exact or normalized matches for sex, income, relationship status, education, occupation, location, place of birth, and age.Occupation permits compatible synonyms; location and place of birth permit compatible broader geographies, while age accepts close values and compatible ranges.Source: A Evaluation Details · p. 11
  • Prediction coverage counts parseable predictions, while evidence coverage counts predictions whose cited graph references resolve to source-level evidence.Source: A Evaluation Details · p. 11

B PANDORA Filtering and Composition

  • The filtered PANDORA evaluation uses stable age and sex labels, English comments, deduplication, direct-self-disclosure removal, and comment-count sampling.Users with 2,500 or more comments are excluded; all eligible users below 100 comments are retained, with 10% sampled from higher comment-count buckets.Source: B PANDORA Filtering and Composition · p. 11

C Evidence Resolution Details

  • Figure 9 reports resolved source files per prediction and the median cited-to-total source-file ratio across SynthPAI and PANDORA.The cited-source concentration pattern also holds across SynthPAI attributes.Source: C Evidence Resolution Details · p. 11

D Evidence Support and Sufficiency Details

  • Figure 10 reports evidence-support and evidence-only inference audits for both benchmarks, with the cited-versus-random separation also holding across SynthPAI attributes.The main paper presents the PANDORA view because it is the longer real-world setting.Source: D Evidence Support and Sufficiency… · p. 11

E Cited-source removal Ablation Details

The ablations test whether provenance-linked sources matter to inference and how their removal affects retained history meaning. The appendix also describes the human support-audit protocol and judgments.

E Cited-source removal Ablation Details

  • Full cited-source removal produces a larger SynthPAI ASR drop than full random removal while preserving substantial input similarity.This mirrors the PANDORA pattern, but SynthPAI has larger utility loss because cited sources are a larger share of its shorter histories.Source: E Cited-source removal Ablation Details · p. 11, Figure 11 · p. 13
  • Cited-source removal effects vary by attribute: location, place of birth, income, and sex show the largest SynthPAI drops, while education and relationship status are less affected.On PANDORA, age and sex degrade less than some SynthPAI attributes, consistent with substitute cues remaining in longer histories.Source: E Cited-source removal Ablation Details · p. 12, Figure 12 · p. 13
  • At matched removal doses, cited-source removal often exceeds random removal, especially at 100%; remaining performance is consistent with redundant cues across a user’s history.The authors frame mitigation as iterative audit-and-rewrite cycles; the removal experiment deletes source files before rebuilding the graph index.Source: E Cited-source removal Ablation Details · pp. 11–12

F Human Support-Audit Protocol

  • The human validation presents one case at a time and hides whether evidence is cited or random; annotators judge whether source texts support the prediction without outside information.Annotators were recruited internally from the research team, and Figure 13 shows the dashboard.Source: F Human Support-Audit Protocol · p. 12, Figure 13 · p. 14

Human evidence-support evaluation instructions

  • The audit uses Yes, Partial, or No judgments, with Yes and Partial counted as supportive; human-human binary agreement is 72.0% and human-LLM binary agreement is 77.0%.Partial denotes evidence supporting a broader or weaker version of the predicted attribute.Source: Human evidence-support evaluation… · pp. 13–14

G Implementation Settings and Runtime

Implementation uses a per-user GraphRAG index and reports source-history semantic retention and runtime measurements for the benchmark environments.

G Implementation Settings and Runtime

  • GraphProfiler uses GraphRAG v2.7.0 with per-user indexing, Azure OpenAI GPT-4o for graph and claim extraction and querying, and text-embedding-3-small for local search.Graph/node layout embeddings and UMAP are disabled, while local-search text embeddings remain enabled.Source: G Implementation Settings and Runtime · p. 14
  • Semantic retention compares original and ablated source-history embedding aggregates using all-MiniLM-L6-v2 and is treated as a coarse proxy for retained meaning.The measure does not imply that deletion is the preferred privacy intervention.Source: G Implementation Settings and Runtime · p. 14
  • SynthPAI graph indexing takes a median 170.4 seconds per user, while the initial all-attribute graph query takes a median 61.2 seconds per user.PANDORA wall-clock time depends on shard scheduling, VM availability, and endpoint throughput across five VMs.Source: G Implementation Settings and Runtime · p. 14

H Community Reports

This section presents community reports that summarize recurring personal structures across users, alongside the prompts used to turn graph context into source-grounded, attribute-specific inferences. It covers evidence and reference reporting, qualified guesses when evidence is weak or implicit, and procedures for judging predictions and auditing cited evidence.

H Community Reports

  • Community reports are an abstractive, coarse audit layer that can merge evidence from multiple source records, so the paper bases provenance claims on source-resolved evidence instead.Those sources include cited entities and relationships, support audits, evidence-only inference, and cited-source removal.Source: H Community Reports · p. 14
  • Theme prevalence counts a user once when at least one community report matches a theme, rather than counting reports, enabling comparison across datasets without over-weighting users with many communities.Figure 14 reports this per-user theme prevalence.Source: H Community Reports · pp. 14–15, Figure 14 · p. 15
  • The most common report themes describe habits, identity, culture, finances, work, education, relationships, and life stage, showing how ordinary comments can form stable graph neighborhoods that support sensitive inference.The reports are not primarily narrow identifiers such as an explicit age, city, or relationship label.Source: H Community Reports · p. 15
  • PANDORA shows high prevalence for several broad themes, while SynthPAI’s shorter, more controlled histories show the same themes with lower or more selective prevalence.The passage attributes PANDORA’s higher prevalence to users often having more comments, creating more opportunities for community summaries.Source: H Community Reports · p. 15

Community-report graph prompt

  • The graph-report prompt’s output schema includes a title, summary, privacy-risk rating and explanation, and findings with summaries and explanations.Its grounding rules require data references, cap listed record IDs at five per reference, and prohibit unsupported information.Source: Community-report graph prompt · p. 20

Community-report text prompt

  • The community-report prompt requests a privacy-risk float score from 0-10, a one-sentence explanation, 5-10 grounded findings, and a [START, END] YYYY-MM-DD date range preserving time-specific information.Reports should cover attribute connections and noteworthy inferences; a reference lists no more than five record IDs, using “+more” for additional records.Source: Community-report text prompt · pp. 20–21

Local-search system prompt

  • The local-search system prompt requires responses to summarize supplied table data, follow the requested format and length, and avoid unsupported claims.Data-supported statements use table-qualified references, with no more than five record IDs per reference before “+more.”Source: Local-search system prompt · p. 22

K Attribute-Targeted Query Prompts

AGE query prompt

  • The AGE prompt requires a specific, nonempty estimate, using direct age statements first and timeline or correlated life-stage cues when direct evidence is absent or weaker.It directs the model to distinguish direct, weak, and implicit evidence and use GraphRAG references when available.Source: AGE query prompt · p. 23

EDUCATION query prompt

  • The EDUCATION prompt asks for the highest completed or current level, maps degree terminology to standardized levels, and allows correlated cues when direct evidence is weak or absent.It says not to inflate professional training to a university degree without evidence and requests provenance references when available.Source: EDUCATION query prompt · pp. 23–24

INCOME query prompt

  • The INCOME prompt directs the model to return a nonempty plausible guess, using a low-confidence inference from weak, implicit, or correlated cues if direct evidence is absent.Source: INCOME query prompt · p. 24
  • Income is assigned to one of four buckets by aggregating financial, occupational, location, household, and lifestyle cues, with cost of living considered.The prompt asks the model to characterize the inference as direct, weak, or implicit.Source: INCOME query prompt · p. 24

LOCATION query prompt

  • Location inference must return a named place for current residence, favoring present-day routines and geographically diagnostic cues over travel or past locations.When direct evidence is absent, the prompt still requires a plausible low-confidence named place rather than a generic location category.Source: LOCATION query prompt · p. 25

OCCUPATION query prompt

  • Occupation inference favors a specific current role, using repeated job-related evidence first and education, income, life stage, routines, and interests as implicit cues.The prompt distinguishes occupation from past jobs, hobbies, education, and aspirational plans, and asks whether the inference is direct, weak, or implicit.Source: OCCUPATION query prompt · p. 26

POB query prompt

  • Place of birth inference prioritizes named places tied to birthplace, upbringing, or origin cues and keeps that past location distinct from current residence.When only a country-level clue is supported, the prompt requires a named country rather than an unknown answer.Source: POB query prompt · p. 27
  • POB inference uses origin-linked cues such as childhood, family, migration, and repeated cultural references, while requiring a named place and qualifying weak or implicit evidence.When no direct GraphRAG reference supports an inference, the prompt permits weak or implicit evidence and requires an evidence explanation; it also directs the model to return a plausible named country or region when a reliable city is unavailable.Source: POB query prompt · p. 27

RELATIONSHIP_STATUS query prompt

  • Relationship-status inference must return a non-empty guess, prioritize explicit current evidence, and may aggregate household, parenthood, dating, or family-event cues when direct evidence is weak.Allowed labels are Single, In a Relationship, Married, Divorced, Widowed, and Engaged; silence alone does not support Single.Source: RELATIONSHIP_STATUS query prompt · p. 28

SEX query prompt

  • Sex inference must choose male or female, prefer explicit self-positioning, and avoid stereotypes while allowing carefully aggregated family-role or lived-experience cues.If reliable evidence is absent, the prompt still requires a low-confidence implicit guess and disclosure that direct evidence is absent.Source: SEX query prompt · pp. 28–29

L Evaluation Prompts

  • The main evaluation prompt handles attribute matching, while specialized judges address education, geographic attributes, and occupation when semantic equivalence matters.Source: L Evaluation Prompts · p. 29

Main evaluation judge prompt

  • The main judge distinguishes correct, less-precise, and incorrect predictions, with attribute-specific rules for age, sex, income, relationship status, education, location, POB, and occupation.For example, a broader compatible prediction may be rated less precise, while an incompatible value is rated incorrect.Source: Main evaluation judge prompt · pp. 29–30

Education semantic judge prompt

  • The education judge compares canonical degree levels, accepts equivalent or more specific phrasing at the same level, and distinguishes compatible lower-level answers from overstatements or contradictions.Field of study need not match when degree level matches.Source: Education semantic judge prompt · p. 30

Geographic semantic judge prompt

  • The geographic judge accepts correct city-and-country matches or more specific compatible places, rates broader compatible places as less precise, and rejects wrong-country or vague answers.Source: Geographic semantic judge prompt · pp. 30–31

Occupation semantic judge prompt

  • The occupation judge accepts equivalent or slightly more specific role labels, but rates broader compatible occupation categories as less precise.Source: Occupation semantic judge prompt · p. 31
  • The occupation judge distinguishes exact or equivalent roles from less-precise same-domain predictions and incompatible roles, while rejecting broad labels without a compatible domain.It does not require exact wording or a field modifier when the role or domain is clearly equivalent.Source: Occupation semantic judge prompt · p. 31

M Evidence Audit Prompts

  • The evidence-audit prompts separately assess whether cited or random texts support a prediction and whether shown source texts alone enable direct attribute inference.Source: M Evidence Audit Prompts · p. 31

Evidence support audit prompt

  • The support audit uses only shown texts and labels support as yes, partial, or no, while classifying evidence as direct disclosure, indirect, implicit, or unsupported.It separately records whether the attribute and exact guess are supported.Source: Evidence support audit prompt · p. 31

Direct LLM inference over source texts prompt

  • The direct-inference prompt restricts guesses to shown texts, allows concrete attributable clues, rejects stereotypes, and requires Unknown when evidence is insufficient.It requests a concise guess, confidence score, and reason, with attribute-specific output guidance for age, education, income, residence, occupation, birthplace, relationship status, and sex.Source: Direct LLM inference over source texts… · p. 32
Loading 2609.12448v1…