Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining

1 Introduction

• 86.8% accuracy at ε = 1 with 4 billion Web images contrasts with 39.39% for private ImageNet training without pretraining at ε = 8.The paper presents these as examples of the utility gains that have motivated Web-scale pretraining for private learning.Source: §1 · p. 2
• Web-scraped pretraining data may itself be sensitive, so a DP-finetuned model can still memorize and expose that data.The authors warn that publicizing such models as private could harm data subjects and erode trust in differential privacy.Source: §1 · p. 2, §1.1 · p. 3
• The paper is a position paper that raises concerns and offers broad suggestions, but intentionally does not technically explore solutions.It calls on the community to address challenges that it says each deserve significant attention beyond the article's scope.Source: §1 · p. 3

1.1 Paper Overview

• Benchmarks with overlapping public and private distributions can make it unclear whether private learning occurred beyond domain adaptation.For example, every CIFAR-10 class has an identical label in ImageNet; the paper argues benchmarks should better match envisioned private workloads.Source: §1.1 · pp. 3–5
• Web pretraining may generalize poorly to sensitive domains: BASIC-L achieved 59.6% top-1 accuracy on PCam, just above random guessing.The cited authors attribute this to Web training data curated without emphasis on medical images.Source: §1.1 · p. 4
• Large foundation models often cannot run on end-user devices, requiring private data to be outsourced despite DP protection of training samples.The paper calls for considering model scale and developing smaller distilled models; it notes that current cryptographic alternatives remain impractical for large models.Source: §1.1 · p. 4

2 Is publicly accessible data public?

• Differential privacy protects the finetuning dataset but affords no privacy to pretraining data, making “privacy-preserving” labels at odds with how people may interpret them.The authors argue that people may still view publicly accessible personal data as sensitive, particularly when it was not intentionally made public.Source: §2 · p. 5
• Web-scraped datasets can contain sensitive or unintentionally collected information, as shown by sensitive ImageNet content and the retraction of TinyImages.Large text datasets may also be assembled from Internet scraping with minimal filtering or curation.Source: §2 · p. 6

2.1 Two Motivating Examples

• GPT-2 memorized a real person’s phone number, and the authors report that production models such as ChatGPT know phone numbers of people who posted them online for another purpose.The paper also describes possible privacy violations involving deepfakes and easier searches that link anonymous or pseudonymous images to individuals.Source: §2.1 · p. 6
• Foundation models can amplify privacy leakage by making personal information available in broader contexts, including through new applications built on pretrained models.The authors compare this dissemination to doxing information that was already publicly accessible but difficult to discover.Source: §2.1 · p. 7
• Publicly accessible data may have been shared for a particular purpose or without the subject’s knowledge, raising privacy concerns about its use in pretraining.The examples include publicly exposed surveillance feeds and an unintentionally uploaded cryptocurrency key later memorized by Copilot.Source: §2.2 · p. 7, §2.1 · p. 7

2.2 Privacy Expectations

• Calling models privacy-preserving despite leakage from pretraining could erode trust in differential privacy and compound mismatches between DP’s guarantees and people’s privacy expectations.The authors warn that this mistaken loss of trust could carry over to settings where differential privacy is properly applied to all collected data.Source: §2.2 · p. 7

3 Are we still measuring progress on private learning?

• Benchmarks for private learning should better reflect real-world privacy-sensitive tasks, rather than treating performance on familiar tasks as a reliable proxy.The paper argues that benchmark choice should make progress on the benchmark informative about progress on the task of interest.Source: §3 · p. 8

3.1 Intra-domain versus Cross-domain Finetuning

• Many public-pretraining benchmarks use private data distributions that overlap with, or are subsets of, the public data distribution, even when the datasets themselves are disjoint.For example, ImageNet and CIFAR-10 are both drawn from Internet images, and CIFAR-10’s distribution is described as a subset of ImageNet’s.Source: §3.1 · pp. 8–9
• These benchmarks can obscure private-learning progress because they mix algorithmic improvements with representation-learning gains and use incomparable public pretraining sources.For CIFAR-10, cited public sources range from CIFAR-100 and ImageNet to a single engineered image.Source: §3.1 · p. 9
• CLIP achieves 96.2% zero-shot accuracy on CIFAR-10 without finetuning, despite never seeing its training data, hence (ε, δ) = (0, 0)-DP; report 85.7% zero-shot top-1 accuracy on ImageNet, also achieving perfect privacy.Source: §3.1 · p. 9
• CIFAR-10-with-pretraining may leave only ≈2% performance to improve, and such marginal gains may not transfer to settings with little public-private data overlap.The paper compares this setup to MNIST as a limited proxy for general computer vision progress.Source: §3.1 · p. 9

3.2 Towards Better Benchmarks

• The field cannot tell whether sensitive tasks are well represented in public data without benchmarks that resemble the privacy-sensitive tasks of interest.The paper contrasts a world where public data represents sensitive tasks with one where those tasks are mostly disjoint from public data.Source: §3.2 · pp. 9–10
• Pre-training on Internet data might offer little help in many privacy-sensitive settings, exemplified by the minor benefit of transfer learning reported by [RZKB19] on some medical tasks.Source: §3.2 · p. 10
• Sensitive datasets released for research could support benchmarks that separate non-private representation-learning advances from privacy-preserving learning advances.Examples include medical datasets, email corpora, and user reviews; even favorable transfer results would not resolve concerns about sensitive pretraining data or outsourcing large models.Source: §3.2 · p. 10

4 Large models require uploading private data

• Using enormous privately finetuned models will likely require data owners to upload sensitive training data to a remote cloud service, trading off their privacy against end users’ privacy.The paper notes state-of-the-art DP ImageNet models have over 250 million parameters and require over 100 billion FLOPs to evaluate.Source: §4 · p. 10
• For a minimum viable accuracy, the paper contrasts privately finetuning a large pretrained model with finetuning a smaller model without DP or with very low privacy guarantees.The hypothetical excludes strong-DP finetuning of the smaller model because its resulting accuracy would be too poor for use.Source: §4 · p. 10
• Owners face a choice between centrally collecting data for differentially private training and keeping data local without differential privacy guarantees.The paper says the largest models are too large for local finetuning, and local differential privacy adds too much noise for practical deployment in these settings.Source: §4 · p. 11
• Cryptographic confidentiality methods are several orders of magnitude from efficient practical use on large models, while existing trusted execution environments have security and scalability limits.The paper names fully homomorphic encryption, secure multiparty computation, and trusted execution environments as possible alternatives.Source: §4 · p. 11

Articulate granular privacy considerations for Web data

• Internet datasets need more granular privacy treatment, including consent procedures, audits for sensitive content, and disclosure of privacy concerns.The paper notes that individual privacy expectations are rarely binary.Source: §5 · p. 11

Construct privacy-friendly pretrained models

• Privacy-friendly pretrained models remain an open problem, with possible approaches including careful data curation, data-owner consent, and differentially private pretraining.The appropriate sensitivity boundary depends on context; sentence-level privacy may be insufficient when references to sensitive information remain in the data.Source: §5 · p. 11

Design better benchmarks to measure progress in private learning

• Private-learning benchmarks should better reflect sensitive-domain tasks, because standard ML datasets may lack their characteristics and correlate weakly with the target task.The authors encourage exploring alternative benchmarks more closely aligned with privacy-sensitive tasks.Source: §5 · pp. 11–12

Promote a holistic view on ML privacy

• ML privacy research should extend beyond DP training of a single model to data collection, data and model lifetimes, and related privacy considerations.The paper describes the current literature as predominantly model-centric.Source: §5 · p. 12

• Recent work using public data has shown that DP can be preserved on some complex ML problems without devastating utility losses, an important step for the field.Source: §5 · p. 12