Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Risks

Research question and approach

• The paper addresses the gap in systematic understanding of how modern AI and ML change product and service privacy risks.The authors analyze 321 privacy-relevant incidents and compare them with Solove’s taxonomy as a pre-AI baseline.Source: §1 · pp. 2–3

Findings

• The taxonomy identifies AI capabilities and data requirements as sources of both newly created and exacerbated privacy risks.Examples include deepfake pornography exposure and surveillance risks associated with collecting face scans for facial-recognition training.Source: §1 · p. 3

New risks

• AI-generated human-like media creates new exposure risks, while arbitrary classification functions enable a new phrenology/physiognomy risk category.The latter includes attempts to infer sensitive attributes, such as sexual orientation, from physical attributes.Source: §1 · p. 3

Implications

• Existing approaches such as differential privacy and federated learning address only a subset of AI privacy risks.The authors call for tools and resources to help practitioners negotiate the utility-intrusiveness trade-off.Source: §1 · p. 3

2.1 Human-centered AI

• Prior work highlights AI’s potential for spurious predictions about individuals, including predictions from profile images about sexual orientation or “criminality.”The section also connects inferred-interest advertising with reported psychological distress and traumatization.Source: §2.1 · p. 3
• The paper presents its taxonomy as a tool and shared language for foregrounding end-user privacy in human-centered AI design.The background notes that practitioners and stakeholders have struggled with privacy and fairness concepts in implementing human-centered AI guidelines.Source: §2.1 · p. 4

2.2 Prior privacy taxonomies and concepts

• Existing AI privacy frameworks often categorize potential risks or strategies without accounting for documented harms or how AI changes established privacy risks.The paper positions its approach as bridging AI-focused work and Solove’s broader privacy taxonomy through analysis of realized risks.Source: §2.2 · pp. 4–5
• Solove’s taxonomy classifies privacy intrusions across collection, processing, dissemination, and invasion, but prior work had not applied or adapted it to emerging realized AI intrusions.The paper adopts an AI-centric approach to codifying realized risks across the AI system life cycle.Source: §2.2 · pp. 4–5

2.3 Creating a privacy taxonomy

• Privacy taxonomies are difficult to develop because privacy lacks a universally accepted definition and taxonomies must be operationalized within a domain.Existing taxonomies may target legal professionals or perceived risks rather than AI-specific functionality and documented harms.Source: §2.3 · p. 5
• The paper takes a first step toward codifying documented privacy-risk patterns resulting from AI capabilities and requirements.It addresses conceptual AI privacy intrusions that may be invisible to end users.Source: §2.3 · p. 5

3.1 Constructing the Taxonomy of AI Privacy Risks Based on AIAAIC

• The authors analyzed 321 documented incidents from AIAAIC, filtering for deployed AI technologies associated with end-user privacy risks.They also reviewed a random 10% sample of cases without the privacy label; the identified risks were already represented among tagged cases, which they took as saturation.Source: §3.1 · p. 6
• Using Solove’s taxonomy as a baseline, the authors classified AI’s role as creating a risk, exacerbating it, or not meaningfully changing it.They defined exacerbation as increasing a risk’s scale, scope, frequency, or intensity, and creation as AI capabilities or requirements fundamentally enabling the risk.Source: §3.1 · p. 7
• The final codebook contains 12 operationalizable privacy-risk labels, including phrenology/physiognomy, exposure, surveillance, and intrusion.The authors describe phrenology/physiognomy as estimating sensitive attributes from physical attributes.Source: §3.1 · p. 7

3.2 Qualitative Analysis Procedure

• The authors iteratively refined the codebook and double-coded cases to assess agreement, with Cohen’s Kappa above 0.8 for every risk type and averaging 0.94.After the reliability check on 65 cases, one coder coded the remaining 124 cases.Source: §3.2 · p. 7

• AI exacerbates secondary-use risks through new capabilities over collected data, exclusion risks through training without consent, and insecurity risks through pipeline vulnerabilities.Source: §4 · p. 9
• Generative AI creates exposure risks by reconstructing redacted content or exposing inferred sensitive information, and distortion risks through realistic fake images or audio.Source: §4 · p. 9
• AI creates disclosure risks by inferring beyond raw data and exacerbates them through training-data sharing; publicizing personal datasets expands accessibility risks.Source: §4 · p. 9
• Centralized or ubiquitous surveillance infrastructures exacerbate the scale and ubiquity of intrusion risks.Source: §4 · p. 9
• The taxonomy distinguishes AI-created instantiations of existing risks, a new risk category, and exacerbations of risks in Solove’s taxonomy.Source: §4 · pp. 9–10
• 298 of 321 incidents (92.8%) involved AI creating or exacerbating the described privacy risks.Source: §4 · p. 10

4.1 Data collection risks

• AI may create data-collection risks by enabling collection of previously inaccessible data, or exacerbate them through AI/ML development needs or increased collection scope.Source: §4.1 · p. 10

4.1.1 Surveillance (150/321).

• The analysis found AI exacerbating surveillance risks, but no incidents of AI creating new surveillance risks.Source: §4.1 · p. 10
• AI enhances surveillance scale by linking diverse sources and increasing the quantity of collected personal data.Source: §4.1.1 · pp. 10–11
• Documented surveillance spans physical and cyber settings: Xinjiang’s system combined CCTV, Wi-Fi, and sensitive records, while Gaggle monitored students’ digital footprints.Source: §4.1.1 · p. 11
• Because training-data volume can affect model performance, AI can increase demand for large-scale personal-data collection; without automated analysis, there may be little incentive to collect at that scale.Source: §4.1.1 · p. 11
• AI exacerbates surveillance ubiquity through physical sensors and devices, including employee geolocation tracking, CCTV, and virtual-assistant microphones.Source: §4.1.1 · p. 11

4.2 Data processing risks

• The incidents covered established identification, aggregation, secondary-use, exclusion, and insecurity risks, alongside new phrenology/physiognomy risks; AI creates new identification and aggregation risks and exacerbates several established risks.The reported mechanisms include re-purposing foundation models, training on datasets containing content obtained without consent, and introducing security vulnerabilities through AI use.Source: §4.2 · p. 11

4.2.3 Phrenology / Physiognomy (27/321).

• AI creates a new phrenology/physiognomy risk by inferring personality, social, or emotional attributes from physical traits through arbitrary input-output correlations.This differs from aggregation, which combines observable data to make deductive inferences; these risks produce new, unfounded inferences from physical characteristics users cannot control.Source: §4.2 · p. 11, §4.2.3 · p. 13

4.2.1 Identification (124/321).

• AI technologies enabled automated identification at scale (20/124), allowed identification risks to occur more quickly, in nigh real-time (24/124) once models are trained, enabled robust identification with low-quality data (7/124), and enabled ubiquity identification risks in situated physical environments (73/124).Source: §4.2.1 · pp. 11–12

4.2.2 Aggregation (49/321).

• AI creates aggregation risks by processing vast amounts of personal data to infer invasive things at scale (23/49), performing complicated inferencing nigh instantly (11/49), and creating predictive aggregation risk (10/49).AI technologies continuously monitor data streamed from physical sensors.Source: §4.2.2 · p. 12

4.2.3 Phrenology / Physiognomy (27/321).

• Models have been used to infer sexual orientation and criminality from facial images, personal attributes from faces, and deceptiveness from voice data.Examples include an engagement-estimation tool used without students’ consent and a voice-based system used by health insurance and money lending platforms to select low-risk clients.Source: §4.2.3 · p. 13

4.2.4 Secondary Use (39/321).

• AI does not always change secondary-use risks (6/39); Luca, a German COVID-19 contact-tracing app, was found to repurpose personal data such as location.Secondary use is using personal data collected for one purpose for a different purpose without end-user consent.Source: §4.2.4 · p. 13
• AI can exacerbate secondary-use risks through creating unanticipated additional features or capabilities with collected personal data (22/39), and reusing public datasets to train models for purposes other than their original collection purpose (11/39).Source: §4.2.4 · p. 14

4.2.5 Exclusion (149/321).

• AI does not meaningfully change exclusion risks when the issue is simply covert processing of personal data (76/149).A homestay company’s trustworthiness algorithm covertly assessed publicly available social-media posts.Source: §4.2.5 · p. 14
• AI requirements can exacerbate exclusion by incentivizing collection of large, rich personal datasets without consent (73/149).This creates a tension with inclusive AI approaches: capturing more people’s data can increase exclusion risk when consent and control are inadequate [12].Source: §4.2.5 · pp. 14–15

4.2.6 Insecurity (17/321).

• AI capabilities and/or data requirements directly exacerbated insecurity risks in 5/17 cases, while other cases involved the same risks arising from poor operational security unrelated to AI capabilities or data requirements.Some cases involved AI technologies unexpectedly revealing personal data on which they were trained. Converting raw data into training-ready labeled data can require exposing raw personal data.Source: §4.2.6 · p. 15
• AI can unexpectedly reveal personal data used for training: a chatbot exposed users’ names, nicknames, and home addresses, while preparing labeled training data can expose raw personal data to gig workers.The chatbot example involved Lee Luda; the gig-worker passage describes exposure during conversion of raw data into training-ready labeled data.Source: §4.2.6 · p. 15

4.3 Data dissemination risks

• AI creates new and amplified dissemination risks: generated or inferred information can be revealed, while disclosure can grow in scale and frequency.The risks include new exposure, distortion, and disclosure, plus greater accessibility and amplified disclosure.Source: §4.3 · p. 15

4.3.1 Exposure (17/321).

• Generative techniques can reconstruct censored content or create nonconsensual fake media, while AI can expose sensitive attributes inferred from data.The analysis attributes 10 of 17 exposure incidents to generative techniques and 7 of 17 to inferred sensitive attributes.Source: §4.3.1 · p. 16

4.3.2 Distortion (20/321).

• AI-generated images and audio can exploit identities to create realistic false depictions that people may struggle to distinguish from real ones.The authors identify these as new distortion risks.Source: §4.3.2 · p. 16

4.3.3 Disclosure (45/321).

• AI technologies can create new disclosure risks by inferring information beyond what raw data explicitly captures, and can exacerbate disclosure when personal data used to train ML models are shared.In 17/45 cases, AI did not meaningfully change disclosure risks, such as sharing personal data with law enforcement or third parties.Source: §4.3.3 · p. 16

4.3.4 Increased Accessibility (23/321).

• Publicly sharing large personal datasets makes information easier for wider audiences to access, including data that may otherwise be private or difficult to find.Open datasets can support transparency and audits, while also making personal data broadly accessible.Source: §4.3.4 · p. 17

4.4.1 Intrusion (160/321).

• In six intrusion incidents, AI did not fundamentally change the risk; store screens showing personalized ads are one example.The authors say the intrusion would remain even without AI capabilities or requirements.Source: §4.4.1 · p. 17
• AI capabilities and infrastructure requirements can enable centralized surveillance, exacerbating intrusion risks at scale in 113 of 160 incidents.The authors point to identification and behavior detection capabilities, along with requirements such as vast data stores and GPU servers.Source: §4.4.1 · p. 17
• Everyday products such as doorbells and wristbands can become nodes in ubiquitous surveillance infrastructure in 41 of 160 incidents.Examples include devices capturing nearby conversations or analyzing a user’s speech.Source: §4.4.1 · pp. 17–18

5 DISCUSSION

• Approximately 93% of incidents involved AI creating a new privacy risk or exacerbating a known one, while roughly 7% showed no AI-related change to the risk.The latter cases included incidents such as a data breach affecting a product that happened to use AI.Source: §5 · p. 18
• AI capabilities create new processing risks, phrenology/physiognomy risks from spurious classifiers, and dissemination risks through human-like media generation.Examples include identifying activity from low-quality data, forecasting outcomes, and generating realistic fake videos.Source: §5 · p. 18
• AI’s data requirements exacerbate existing risks, including pervasive surveillance, secondary use, exclusion, insecurity, disclosure, increased accessibility, and intrusion into personal space.Source: §5 · p. 18

5.1 Charting the design space for privacy-preserving AI/ML work

• Existing privacy principles cover some AI-exacerbated collection and processing risks, but omit AI-new processing and dissemination risks identified in the taxonomy.Source: §5.1 · p. 18
• Differential privacy and federated learning address only some model-output processing risks, not the full taxonomy of AI privacy risks.The approaches can help prevent re-identification from model outputs, but do not cover every risk.Source: §5.1 · p. 19
• Privacy auditing tools are limited because they are applied after training and address specific processing risks rather than broader privacy concerns.The passage gives aggregation risks arising from collective sensitive training data as one example of their limited scope.Source: §5.1 · p. 19
• Ethics checklists and toolkits may prompt consideration of broader risks, but their high-level treatment offers little guidance on AI-created or AI-exacerbated risks.Their usefulness also depends on practitioners already being aware of those risks.Source: §5.1 · p. 19

5.1.1 Future Work: Creating AI-specific privacy guidance.

• Future work can build AI-specific tools from the taxonomy to help practitioners weigh utility against intrusiveness, despite barriers including limited AI understanding and compliance-focused privacy practice.Source: §5.1.1 · pp. 19–20
• Future AI-specific privacy guidance should be developed so its artifacts can be effectively incorporated into practitioners’ workflows.Source: §5.1.1 · p. 20

5.2.1 Future Work: Creating a living taxonomy of AI privacy risks.

• Future incidents may expand the taxonomy, including risks beyond Solove’s categories; four categories absent from the analyzed incidents may also be altered by future AI advances.The authors frame the taxonomy as a structure for mapping AI capabilities and requirements to high-level privacy risks.Source: §5.2 · p. 20, §5.2.1 · p. 21

5.2 Theoretical extensions to the AI privacy risks taxonomy

• Chatbots could create interrogation risks by eliciting information, while affective computing could exacerbate them by inferring demeanor from verbal and non-verbal cues.The authors also suggest that language models could imitate people and interact in natural language to extract information.Source: §5.2 · p. 20
• Generative AI could create blackmail risks through convincing fabricated evidence, while automating the search for compromising information could exacerbate them; the incidents analyzed did not show fabricated content used for blackmail.The authors had observed fabricated incriminating content in incidents discussed under exposure and distortion risks.Source: §5.2 · p. 21
• Conversational agents that gain users’ trust in high-stakes settings such as healthcare or finance could exacerbate confidentiality breaches by sharing sensitive disclosures with third parties.The risk is framed as interpersonal disclosure of something shared in confidence.Source: §5.2 · p. 21
• Personalized propaganda, online censorship, and recommendation or persuasion systems could exacerbate decisional interference by guiding choices toward external entities’ goals over individuals’ preferences or well-being.The discussion includes advertisers and political campaigns as examples of external entities.Source: §5.2 · p. 21

5.3 Limitations

• Because the taxonomy uses documented incidents recognized as harmful, its dataset is not a representative account of every possible AI privacy risk.The authors note that additional incidents may be missing from the dataset.Source: §5.3 · p. 22
• 13 of 15 independently collected incidents appeared in AIAAIC; the same check found only five in AIID, supporting AIAAIC’s coverage for the authors’ goal.For the two incidents absent from AIAAIC, the authors found similar incidents involving the same technology but different products.Source: §5.3 · p. 22

• The taxonomy is based on a systematic analysis of documented AI privacy incidents, and the authors call for AI-specific design guidance to help practitioners negotiate utility-intrusiveness trade-offs.They also state that privacy-preserving machine-learning approaches such as federated learning address only a small subset of AI’s unique privacy risks.Source: §6 · pp. 22–23
• AI incorporation does not necessarily change a product’s privacy risks, but the taxonomy finds that it often does.Source: §6 · pp. 22–23
• AI capabilities can create new privacy risks, including exposure risks from deepfake pornography.The paper gives realistic but fake images as an example of the relevant capability.Source: §6 · p. 23
• AI data requirements can exacerbate known privacy risks, including surveillance when governments capture passenger facial images for identification.The passage links this example to AI’s ability to identify individuals from low-fidelity images.Source: §6 · p. 23

• Appendix Table 2 reports Cohen’s Kappa for each privacy risk.Source: Table 2 · p. 30