Privacy Leakage Through AI-mediated Analysis of Smartphone Data

1 Introduction

• Because users may want personalized services while keeping some traits hidden, understanding software-generated inferences matters for assessing privacy risks.Such inferences may concern health, political beliefs, or physical location.Source: §1 · p. 1
• Prior approaches to user profiling relied on structured text, while LLMs can also analyze rich media such as images, videos, and sound recordings.Smartphones combine rich media with structured data such as contacts, calendar events, and GPS locations.Source: §1 · p. 2
• PRIVA-SEE uses granted phone permissions to collect user-specific data and prompt an LLM to infer sensitive characteristics and explain its reasoning.The app requests access to photos, calendars and reminders, location, and contacts.Source: §1 · p. 2
• With access to only a subset of smartphone data, PRIVA-SEE often made invasive, accurate inferences about participants and people in their social circles.The IRB-approved study included 465 participants.Source: §1 · p. 2
• Participants reported statistically significant changes in willingness to share data after viewing PRIVA-SEE’s inferences.The study also examined how inference sensitivity and accuracy related to participant comfort.Source: §1 · p. 2
• The authors recommend ongoing warnings about inferences enabled by shared data, along with changes to smartphone consent practices and LLM acceptable-use policies.For example, sharing photos may implicitly expose personal information inferable from them.Source: §1 · p. 2
• The study’s observed privacy risks may understate those of real-world data brokers, which can use larger computing resources than the prototype.The prototype’s LLMs ran on a handful of GPUs with limited VRAM.Source: §1 · p. 2

2.1 Online Data Sharing

• Primary sources, intermediaries, and buyers collect or aggregate user data, and infer traits that may help characterize users or target them.Examples include inferring pregnancy from purchases or luxury-goods interest from ZIP-code-linked records.Source: §2.1 · p. 2

2.2 Permissions Systems

• Android and iOS gate sensitive data behind consent prompts, but app-store privacy declarations depend on developers’ descriptions of backend data handling.Android’s prompt lists requested resources without requiring an explanation there, while iOS requires rationale strings for sensitive-data prompts.Source: §2.2 · p. 3

2.3 Related Work

• Prior work examined implicit model hypotheses, overt-sharing permissions, and coarse-grained ad profiles, but not whether users understand sensitive inferences enabled by sharing data.The paper contrasts its focus on fine-grained LLM-generated inferences with earlier studies of overt sharing and broad profiling.Source: §2.3 · p. 3

3 PRIVA-SEE System Design

• PRIVA-SEE sends smartphone data to server-side LLMs for profiling while centering user choice and transparency, including by showing users the inferences and their stated reasons.The system encrypts data in transit and at rest and deletes raw data and derived inferences after the user finishes.Source: §3 · pp. 3–4

3.1 User-Facing App

• The app requests access to contacts, location, photos, calendar events, and iPhone reminders, but limits uploaded bundles and heuristically selects 10 likely-sensitive photos.Photo selection favors screenshots, Favorites, folders labeled with terms such as “private” or “secret,” and images with GPS locations.Source: §3.1 · p. 4

3.2 Data Ingestion and Storage

• An HTTPS-facing proxy forwards client requests, while the application server validates and stores submitted data and assigns each batch a unique session ID for processing.The database accepts requests only from the application server and is isolated from the public internet.Source: §3.2 · p. 5

3.3 Secure Analysis Environment

• Secure-cluster workers reformat data, run permission-specific LLM analyses, and synthesize selected inferences before results are returned to the app.Raw user data is deleted from the backend after the inferences are stored, and the backend deletes the inferences after the client downloads them.Source: §3.3 · p. 5

3.4 Industry Representativeness

• The prototype’s inferences were more accurate than those possible through traditional adtech tracking, and the authors regard them as a lower bound on industrial capabilities.Open-source models, limited compute and data resources, and a three-minute per-batch time limit constrained the prototype.Source: §3.4 · pp. 5–6

4 Study Methodology

• An IRB-approved study deployed PRIVA-SEE on real phones to catalog LLM-generated inferences and measure users’ responses to them.The study asked whether participants found inferences accurate and whether seeing them changed their willingness to grant app permissions.Source: §4 · p. 6

4.1 Recruitment Procedure

• Prolific recruits were U.S. adults with smartphones, with Android and iOS owners recruited equally.Participants received information about the study’s privacy safeguards and were paid $15; completion took 33 minutes on average.Source: §4.1 · p. 6

4.2 User-facing Workflow

• Participants rated pre-study sharing comfort, granted minimum access to requested data, evaluated five generated inferences and their reasoning, then repeated permission-comfort questions.The app assessed comfort, surprise, and accuracy on five-point scales; the workflow also collected open-ended responses and an IUIPC privacy-concern measure.Source: §4.2 · pp. 6–7

4.3 Analysis Methodology

• The analysis used inferential and descriptive statistics for closed-ended responses, rank correlations, mixed-effects modeling, and iterative coding of open-ended answers.The correlation analysis examined accuracy and comfort relationships while avoiding mathematical coupling with the mixed-effects models.Source: §4.3 · p. 7, §4.3.1 · p. 7

4.3.1 Statistical Analysis

• Separate ordinal mixed-effects models assessed pre/post comfort for each permission type, accounting for participant differences and summed inference accuracy and comfort.The models treated comfort as ordered categories and included time, accuracy, comfort, and their interactions.Source: §4.3.1 · p. 7

4.3.2 Qualitative Analysis

• Two researchers iteratively coded open-ended responses, revising the codebook across rounds until Krippendorff’s α reached 0.8.After three rounds, the researchers divided and coded the remaining responses.Source: §4.3.2 · p. 7

4.4 Study Limitations

• Recruiting people willing to share data may have attracted less privacy-concerned participants; researchers emphasized privacy safeguards to encourage privacy-sensitive participation.The authors cite participants’ stated comfort sharing private data as an indication that this approach may have worked.Source: §4.4 · p. 8
• About half of eligible recruits did not consent, and social-desirability bias or the lower-income sample could affect reported privacy concerns or socioeconomic inferences.The final sample’s IUIPC score was comparable to prior work, while the authors note that lower income may affect socioeconomic-status inferences.Source: §4.4 · p. 8

5.1 Participant Demographics

• The 465 participants were generally representative, but overrepresented lower-income people; 28.7% reported annual income under $25,000 versus 8.2% of the general US population.The sample also overrepresented white and college-educated participants and underrepresented Hispanic or Latino participants and adults 65 or older.Source: §5.1 · p. 8

5.2.1 (RQ1): What types of inferences were made by PRIVA-SEE?

• PRIVA-SEE inferred sensitive topics, including medical conditions (8.5%) and mental health conditions (2.6%), and sometimes produced specific identifiers such as exact addresses (1.9%) or names (2.3%).Inferences could concern people close to the user as well as the user, so sharing a phone’s data may affect the social circle’s privacy.Source: §5.2.1 · pp. 8–9, “You are a homeowner with a moderate… · p. 9

5.2.2 (RQ2): What chain of thought did the model provide?

• Chains of thought referenced all four data types, with location most often (36.3%) and photos least often (15.1%); area-code mentions had the highest reported accuracy (62.3%).Other reported accuracy rates were 59.3% for calendar events/reminders, 53.9% for contacts, and 52.9% for screenshots.Source: §5.2.2 · p. 9

5.2.3 (RQ3): How accurate are inferences?

• 48.8% of inferences were rated “completely accurate” and 16.7% “somewhat accurate”; the authors characterize these rates as lower bounds for practice.For context, a cited study reported 24.4% accuracy for major data brokers’ inferences about gender and age buckets.Source: §5.2.3 · pp. 9–10, Table 2 · p. 10

5.2.4 (RQ4): What are the causes for inaccuracy?

• Inaccuracies included misattributing information about someone close to the user, generalizing from group-level facts, and errors in model reasoning or data preprocessing.Examples include inferring a user’s income from their location’s average income and misreading an acronym or mapping GPS coordinates to an incorrect address.Source: §5.2.4 · p. 10

5.3.1 (RQ5): What is the relationship between inference accuracy and participant comfort?

• Accuracy had a weak positive monotonic relationship with comfort (Spearman ρ=0.263): participants were somewhat or very comfortable with 66% of accurate inferences versus 27% of extremely inaccurate ones.For accurate inferences, participants with lower comfort often cited sensitivity or privacy (65.3%).Source: §5.3.1 · p. 10
• Among participants who saw accurate inferences, accuracy was cited more often to support comfort than discomfort (33.6% versus 15.5%); sensitivity could still contribute to discomfort.Participants also reported less willingness to share data after seeing examples of LLM-generated inferences.Source: §5.3.1 · p. 10, §5.3.2 · p. 13

5.3.2 (RQ6): How did the PRIVA-SEE app interaction impact participant data sharing?

• After PRIVA-SEE, participants reported less comfort granting app permissions; the largest shift was for camera roll and the smallest for contacts, and 52% said their future permissions behavior would change.Participants most often attributed change to better understanding permissions data content (21.5%); 14% cited better understanding of downstream inference capability.Source: §5.3.2 · p. 12

6 Recommendations

• Smartphone OSes should require request-time justifications that explain privacy consequences as well as benefits, since permission decisions are affected by users’ understanding of downstream data use.The paper gives inferring where a user works and lives from location data as an example.Source: §6 · p. 13
• OSes should use local models to flag newly created sensitive data and encourage users to protect it, such as by moving a medical-bill photo to a hidden album.The recommendation is motivated by the finding that reactions to inferences affect privacy behavior.Source: §6 · p. 13
• Transparency should identify the specific data items apps access, because permission-class logs do not show which items were accessed and item-level visibility could help identify misbehavior.Source: §6 · p. 13
• Model acceptable-use policies should explicitly prohibit using models to produce privacy-invasive inferences about users.The paper notes that existing policies do not specifically address these harms, even when apps have consent to access raw personal data.Source: §6 · p. 13

7 Conclusion

• Smartphones concentrate sensitive user-specific data, while sophisticated ML models can infer private aspects of users’ lives from it.Source: §7 · p. 13
• The paper cataloged inferences from real-life user data, documented that users may find them surprising or disturbing, and recommends changes to OSes and ML models to better respect privacy.Source: §7 · p. 13
• Many users became less willing to share data with apps in the future after seeing the inferences.Source: §7 · p. 13

Ethical Considerations

• Participants faced risks of privacy violations and psychological discomfort from sensitive inferences, while the techniques may apply to a much broader population with similar permissions.The study involved access to calendar, contacts, photos, and location data.Source: Ethical Considerations · p. 14
• The study’s potential benefits include making latent privacy risks visible, educating users, grounding recommendations in evidence, and providing a framework for future research.Source: Ethical Considerations · p. 14
• Publication may enable profiling or surveillance, make erroneous inferences seem acceptable for targeting, lower barriers to replication, and cause participant distress.Source: Ethical Considerations · p. 14
• The researchers limited data collection, used informed-consent materials, kept data and outputs in a controlled environment, and withheld a turnkey system.The study received IRB approval, including review of encryption, access restrictions, and retention practices.Source: Ethical Considerations · p. 14
• For real-world deployment, the authors recommend inference previews, separate consent for raw access and sensitive-attribute inference, model guardrails, and clearer enforceable policies.Source: Ethical Considerations · p. 14
• The authors justify studying these risks because permissions-based collection and LLM deployment are widespread, while empirical evidence can inform users and regulators.They say exposure to inferences can improve user understanding and reduce permissiveness.Source: Ethical Considerations · p. 14

Open Science

• The authors withheld the full prompt set because releasing it could facilitate systems that generate invasive inferences about individuals.Source: Open Science · p. 15
• Extended non-aggregated participant data are not shared; the appendix instead provides selectively redacted and aggregated materials to support transparency while reducing re-identification risk.Source: Open Science · p. 15

A Social Media App Permission Requests

• The social-media permission discussion lists calendar, contacts, location, and photos; its table covers data types collected by popular apps as listed on Google Play in early 2026.The listed applications are YouTube, Facebook, Instagram, TikTok, and WhatsApp.Source: A Social Media App Permission Requests · p. 18, Table 4 · p. 18

• The displayed sample consists of participant inferences, redacted to maintain participant privacy.Source: B Sample Inference by Type · p. 18

B.1 Mental Health

• Calendar entries and images prompted inferences about ADHD medication, trauma or C-PTSD, OCD, and ongoing therapy or psychiatric care.One inference linked Concerta-related calendar entries to ADHD treatment; another interpreted an image of an OCD and therapy conversation as indicating a need for support.Source: §B.1 · p. 18

B.2 Medical Conditions; B.3 About Children

• Calendar and contact data led to inferences about cancer-related care, regular Ozempic use, rheumatoid arthritis, and children with special medical needs.Examples also inferred that a family member had undergone oncology and cardiology appointments and tests, and that a child had a genetic condition.Source: §B.2 · pp. 18–19, §B.3 · p. 19

B.4 Intimate Content

• Images and messages prompted inferences about fashion or beauty interests, romantic attention, and dating-app use.The fashion-related inference was marked “Somewhat inaccurate”; the participant was a 25–34-year-old man, a context the model missed.Source: §B.4 · p. 19, §B.5 · p. 19

B.5 Reproductive Health

• A comment about birth control prompted an inference about a woman seeking alternatives, while calendar and image data prompted pregnancy and prenatal-care inferences.The pregnancy inference was summarized as concerning the participant’s sister-in-law and drew on an ultrasound event; another inference connected prenatal-care material and an ultrasound image with preparing for a new family member.Source: §B.5 · pp. 19–20

B.6 Gambling

• Sports-betting app images prompted inferences that users had placed bets, including on a recent or upcoming NFL game.One example identified a bet on a redacted sports team.Source: §B.6 · p. 20

B.7 Financial Status

• Phone data prompted financial inferences about cryptocurrency access, money transfers, benefits, Medicare, and possible loan-related issues.Other examples inferred installment-plan purchases, insurance coverage, and a past-due electricity balance.Source: §B.7 · p. 20

B.8 Legal Issues; B.9 Politics

• ID cards, calendar events, and contacts prompted inferences about correctional-facility residence, court proceedings, immigration issues, and political connections.The political examples included inferred links to both Democratic and conservative organizations.Source: §B.8 · pp. 20–21, §B.9 · p. 21

B.10 Sexual Orientation; B.11 Religion; B.12 Demographics; B.13 Interests

• Contacts, location data, calendar events, and a product image prompted inferences about LGBTQ+ identity, religious affiliations, demographics, and personal interests.Examples included a possible connection to the Jewish community, Unitarian Universalist involvement, possible LDS membership, ethnic or cultural inferences, interest in firearms or social justice, and a visible tattoo.Source: §B.10 · p. 21, §B.11 · p. 21, §B.12 · p. 21

D.2 Chain of Thought Questions

• The chain-of-thought questionnaire asked whether data could support an inference, whether an app could make it in code, and which categories described it.Listed categories include identity, location, and personally identifiable information.Source: §D.2 · pp. 25–26
• Other chain-of-thought questions asked whether an inference included other people’s data and whether participants were comfortable sharing its text.Source: §D.2 · p. 26

D.3 Pre/Post Survey Instrument

• The pre/post survey asked how comfortable participants would be granting a social media app access to their camera roll, calendar/reminders, contacts, or location.The item used a 5-point Likert scale.Source: §D.3 · p. 26
• Privacy-attitude items addressed consumer control and autonomy, company disclosure of data practices, and clear privacy-policy disclosure.Source: §D.3 · p. 26

10. It bothers me to give personal information to so many online companies. (7-point Likert scale)

• Other attitude items asked whether company requests for personal information bother participants, prompt second thoughts, or raise concern about excessive collection.Source: §D.3 · p. 26, §10 · p. 26

14. How would you describe your race? (Select all that apply)

• Demographic questions covered age, gender identity, race, Hispanic or Latino identity, education, taxed income, and reliance on platforms such as Prolific for earnings.Source: §12 · p. 27, §14 · p. 27
• The appendix identifies cumulative link mixed-effects models predicting comfort with calendar and contacts permission sharing.Source: Table 5 · p. 28, Table 6 · p. 28

E.1 Intervention-Level Ordinal Mixed-Effects Model

• Table 7 presents a cumulative link mixed-effects model predicting comfort with location permission sharing.Source: Table 7 · p. 29
• Table 8 presents a cumulative link mixed-effects model predicting comfort with camera-roll permission sharing.Source: Table 8 · p. 29
• The intervention-level ordinal model treats accuracies and comfort ratings as pooled summation scores, without weighting individual inferences by importance.Including individual inferences led to convergence issues; the final model converged without evidence of singularity.Source: §E.1 · p. 30

• Table 9 reports the correlation between inference accuracy and reasoning accuracy.Source: Table 9 · p. 30
• Tables 10 and 11 report correlations between inference accuracy and comfort, with Table 11 stratifying results by inference type.Source: Table 11 · p. 30, Table 10 · p. 30
• Table 12 categorizes PRIVA-SEE inferences, and category percentages may exceed 100% in total because an inference can receive multiple types.Source: Table 12 · p. 31

G Inference Category Results

• Table 13 presents participant demographic information.Source: Table 13 · p. 32

I Inaccuracy Types (Full)

• Table I provides an overview of identified inaccuracy categories.Source: I Inaccuracy Types (Full) · p. 33

J Additional Plots

• Figure 8 presents inference accuracy by reasoning data source.Source: Figure 8 · p. 34
• Figures 9–11 address user comfort with inferences, including comfort by inference type and with reasoning.Source: Figure 9 · p. 34, Figure 10 · p. 34, Figure 11 · p. 34