Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers

1 Introduction

• Retractions have limited effects: DukeMTMC and MS-Celeb-1M remained available and in use, while unclear creator intentions may have left users confused.The data persisted through multiple sources, limiting access restrictions; retractions without a clear statement against further use may also have limited normative influence.Source: §1 · p. 1
• Derivatives can enable production use, add annotations, or process data, with each alteration raising distinct ethical considerations.The supplied finding identifies derivative creation and models as a source of concerns beyond dataset creation.Source: §1 · pp. 1–2
• Dataset licenses may not effectively restrict production use; only 3 of 21 repositories releasing MS-Celeb-1M-pretrained models carried its non-commercial designation.The authors also report anecdotal evidence that production use of models trained on non-commercial datasets is commonplace.Source: §1 · p. 2
• Technological and social changes can alter dataset ethics, while inaccessible documentation, ambiguous use reporting, and limited tracking hinder retrospective understanding of dataset impacts.The examples cited include LFW and ImageNet, where technological advances enabled production use and social factors changed understanding of demographic composition and annotation practices.Source: §1 · p. 2
• The paper proposes dataset stewarding: ongoing harm mitigation with responsibility distributed among creators, conference program committees, users, and the broader research community.Its recommendations adapt existing harm-mitigation guidance to address concerns across the dataset life cycle.Source: §1 · p. 2

2 Overview of datasets and analysis

• LFW, DukeMTMC, and MS-Celeb-1M were selected from a list of 54 datasets; LFW enabled longitudinal analysis, while the other two were the most cited retracted datasets in the list.LFW was introduced in 2007; DukeMTMC and MS-Celeb-1M had been retracted.Source: §2 · p. 2
• The analysis covered 946 unique papers: 275 citing DukeMTMC or derivatives, 276 citing MS-Celeb-1M or derivatives, and 400 citing LFW or derivatives.The corpus was built by identifying derivatives and associated papers, then collecting papers citing those papers through the Semantic Scholar API.Source: §2 · p. 2
• Because the corpus misses some papers using the datasets or derivatives, results are lower bounds; most analyses also exclude use outside published research.The authors describe finding derivatives absent from their initial list and say they know of no systematic way to identify all of them.Source: §2 · pp. 2–3

3 Retractions and runaway data

• Both retractions fell short of restricting access or clearly signaling that use should stop: the data remained available through multiple sources, and unclear creator intent may have confused users.The authors treat MS-Celeb-1M’s removal as a retraction in response to ethical concerns, although its creators did not officially state that ethical concerns motivated removal.Source: §3 · p. 3
• DukeMTMC and MS-Celeb-1M continued to be used in 2020, largely through derivatives.The figure passage reports continued use after the datasets were retracted in April 2019.Source: Figure 1 · p. 4
• The authors connect continued use of retracted datasets to persistent citation of retracted papers, noting proposals to clarify retraction status and check references.They say these intervention types also apply to retracted datasets.Source: §3 · p. 4
• Retraction also raises an archival tension: disappearing datasets can impede future study of systems that may shape everyday life, while access frameworks that avoid perpetuating harm remain an open question.Crawford and Paglen identify the problem of inaccessible or disappearing datasets and call for further work on such frameworks.Source: §3 · p. 4

4 Derivatives raise new ethical questions

• Derivatives reduce data-acquisition costs, but that same resource-sharing can make creating ethically dubious derivatives cheaper and faster than collecting and annotating data anew.The authors give DukeMTMC-ReID as an example of using existing annotations and bounding boxes to create a cropped person-reidentification benchmark.Source: §4 · p. 4
• Twenty-one of 41 derivatives could enable new applications with distinct ethical concerns, such as masked-face recognition that has been criticized for violating privacy.DukeMTMC-ReID is used more often than its parent for person re-identification, while SMFRD adds masks to LFW images.Source: §4 · p. 4
• Twenty-one repositories released MS-Celeb-1M-pretrained models, which can perform face recognition out of the box or support transfer learning and downstream applications.The authors note concerns about how training-data biases affect pre-trained models and their downstream applications [77].Source: §4 · pp. 4–5
• Seven of 41 derivatives added annotations, including gender, race or ethnicity, and attributes such as attractiveness, potentially enabling sensitive-attribute classification or identification.The authors identify privacy and representational harms associated with data annotation.Source: §4 · p. 5
• Other post-processing can also matter: cleaning may shift demographic composition, while alignment, cropping, or frontalization may perform differently across demographic groups.Five MS-Celeb-1M derivatives cleaned images; five LFW derivatives aligned, cropped, or frontalized them.Source: §4 · p. 5

5 Effectiveness of licenses

• The license analysis covers DukeMTMC, MS-Celeb-1M, LFW, and ImageNet, and finds shortcomings in preventing commercial use.Table 3 summarizes the intended restriction on commercial use and its practical effectiveness.Source: §5 · p. 5, Table 3 · p. 6
• Fourteen online discussion posts provided anecdotal evidence that non-commercial dataset licenses are sometimes ignored in practice.Commonly used frameworks include pretrained ImageNet models, lowering the barrier to commercial use.Source: §5 · p. 5
• Keras and PyTorch representatives suggested commercial use of pretrained weights may be allowed, but neither provided an official answer.Their comments describe uncertainty about whether model weights inherit dataset-license restrictions.Source: §5 · p. 5
• Pretrained models create a potential licensing loophole: downloading weights can have substantively the same effect as training on ImageNet, despite clearer restrictions on direct commercial training.The paper notes that creators can specify whether restrictions apply to models trained on a dataset.Source: §5 · pp. 5–6
• The evidence about practitioner behavior is explicitly preliminary, tentative, and anecdotal.The authors suggest interviews or surveys could more thoroughly examine practices and legal understanding.Source: §5 · p. 6

6 Technological and social change affects dataset ethics

• LFW and ImageNet’s ethical concerns surfaced more than a decade after release; the authors identify increasing production viability, evolving ethical standards, and changing academic incentives as possible explanations.DukeMTMC and MS-Celeb-1M were excluded from the longitudinal analysis because they were relatively recent.Source: §6 · p. 6
• LFW’s transition from a research benchmark to commercial-system evaluation followed improved technology and broader production viability.The paper notes that face-recognition use in surveillance or policing has drawn backlash, especially over disparate performance on minority groups.Source: §6 · pp. 6–7
• 41 of 49 results added after 2016 were commercial, compared with 3 of 38 results added before 2014, illustrating LFW’s rise in production use.The authors estimate when results appeared by examining archived versions of LFW’s website.Source: §6 · p. 6, Figure 2 · p. 7
• In 2019, LFW’s website warned against commercial-system evaluation because the dataset lacked diversity across demographics and image conditions.At release, the creators had emphasized LFW’s diversity relative to laboratory-image datasets; later work highlighted demographic performance disparities.Source: §6 · p. 7
• ImageNet shifted from an immature research area to a common pretraining source, while its non-commercial terms coexist with pretrained models released under commercial licenses.The paper frames this as another shift from research toward production use.Source: §6 · p. 7
• ImageNet’s people-category labels, including misogynistic and racial slurs, were removed in 2019 after researchers exposed stereotypes; the authors hypothesize that public concern, academic incentives, and norms had changed.They note that such labels would likely have been considered offensive in 2009, even though the critique appeared nearly a decade after release.Source: §6 · p. 7
• A dataset’s transition from research to production can signal success, while social factors shape whether and how ethical standards are acted upon.The paper contrasts changing views of LFW with ImageNet labels that likely would have been considered offensive when created.Source: §6 · p. 7

7 Dataset management and citation

• Management and citation can support accessible documentation, transparency and accountability, and tracking of dataset use, but current practices fall short of these aims.Citations can help reviewers check license compliance, while organized citation infrastructure can track datasets and derivatives.Source: §7 · p. 8
• None of the 38 datasets were managed in shared repositories; three could not be located, two were available only through the Wayback Machine, and retracted datasets’ licenses were no longer officially available.Persistent storage and associated metadata support access to documentation, including licensing information.Source: §7 · p. 8
• Only six of 60 sampled papers provided dataset access information, and citations could not be disambiguated in 42 of 446 cases.None of the 38 datasets had DOIs or stable identifiers; five datasets lacked names.Source: §7 · p. 8
• Missing dataset-specific identifiers make systematic tracking difficult, and papers may use datasets without citing them or cite them without using them.The authors also report that no systematic way exists to identify dataset derivatives.Source: §7 · p. 8

8 Recommendations

• Harm mitigation requires life-cycle stewardship and distributed responsibility rather than relying on dataset creation or any single stakeholder.The proposed approach assumes willingness from creators, program committees, and the research community; malicious users and contexts outside research are beyond scope.Source: §8 · p. 9

8.1 Dataset creators

• Creators should make ethically important information and restrictions clear, specific, and accessible, including whether non-commercial limits apply to trained models.Persistent documentation and requirements for users and derivative creators can make terms easier to find.Source: §8.1 · p. 9
• Creators should continuously review possible misuse and update licenses, documentation, or access controls; higher-risk cases may warrant case-by-case access decisions.Derivative rules may specify allowed or disallowed types and require permission for derivatives outside pre-approved categories.Source: §8.1 · p. 9
• Stewardship increases creator burden, a concern because dataset creation is already undervalued and added duties may further weaken incentives.The authors acknowledge a tension between ethical responsibility and minimizing creator burdens.Source: §8.1 · p. 10

8.2 Conference program committees

• Program committees can require dataset disclosure, justification, license-compliance certification, and checks for retraction during ethics review.They can also encourage public repositories and standardized management and citation practices.Source: §8.2 · p. 10
• Dataset-specific conference tracks can enable tailored ethics reviews, while advance proposal review can give researchers early feedback on value and ethics.The paper also points to registered reports as a model for pre-outcome review.Source: §8.2 · p. 10

8.3 Dataset users and other researchers

• Users should comply with terms, study documentation, assess application-specific appropriateness, identify datasets clearly, and make access information usable.The authors recommend retrospective study because dataset impacts are not fully understood at creation and traditional citations can be ambiguous.Source: §8.3 · p. 10
• The paper does not address production use of datasets or pretrained models, and notes that transparency incentives for companies may require policy intervention.This boundary concerns understanding use in deployed pipelines.Source: §8.3 · p. 10

9 Conclusion

• Ethical consequences can change after dataset creation, so harm mitigation should distribute responsibility among stakeholders throughout the dataset life cycle.The authors note that such consequences can be hard to anticipate a priori.Source: §9 · p. 10

A Broader impacts

• The authors identify raising awareness of models or datasets derived from retracted or license-nonadherent assets as a potential harm, but expect little added harm because the assets are already widely accessible.They hope the work makes ethical considerations clearer to users of these assets.Source: A Broader impacts · p. 17

B Supplemental data

• Supplemental materials include CSVs of analyzed papers and a dataset inventory; their availability is tied to Princeton CITP GitHub remaining operational.The paper also links license details through the supplemental-data site.Source: B Supplemental data · pp. 17–18

C Description of datasets analyzed

• MS-Celeb-1M included about 10 million images of 10,000 celebrities, some of whom were ordinary citizens whose images were aggregated without consent [41]; DukeMTMC used campus footage [72].DukeMTMC tracked multiple people through a camera network, with applications including security and surveillance [72].Source: C Description of datasets analyzed · p. 18
• LFW used faces scraped from Yahoo News articles, and its website later warned against using it to conclude that an algorithm was commercially suitable.Source: C Description of datasets analyzed · pp. 18–19

• The authors identified derivatives through keyword searches and manual review, retained those cited at least 100 times, and used Semantic Scholar citations to assemble papers.They parsed available PDFs with GROBID and also sampled papers to identify derivatives missed by the keywords.Source: D Methodology for overarching analysis · p. 19
• The December 2020 corpora contained 1,393 DukeMTMC, 1,404 MS-Celeb-1M, and 7,732 LFW papers, but omitted papers because of indexing gaps and an incomplete derivative list, making results underestimates.Source: D Methodology for overarching analysis · p. 19
• The analysis coded 946 unique papers: 275 on DukeMTMC, 276 on MS-Celeb-1M, and 400 on LFW or their derivatives.They sampled 20% or 400 papers, whichever was fewer, stratified by publication year.Source: D Methodology for overarching analysis · p. 19

• Five of seven DukeMTMC derivatives contained some or all original data, while six MS-Celeb-1M derivatives did; the two popular Duke derivatives remained downloadable [95].MS-Celeb-1M itself remained available through third-party sites, and 20 GitHub repositories still offered models pretrained on it.Source: E Supplement: Retractions of DukeMTMC… · pp. 19, 21
• In 2020 samples, DukeMTMC and its derivatives were used 73 times and MS-Celeb-1M and its derivatives 54 times after retraction.The authors state that these samples represent hundreds of uses in total.Source: E Supplement: Retractions of DukeMTMC… · p. 21
• Of 73 DukeMTMC uses, 72 came through derivatives, including 63 through DukeMTMC-ReID [95]; MS-Celeb-1M uses included 11 original-dataset, 17 derivative, and 26 pretrained-model uses.Source: E Supplement: Retractions of DukeMTMC… · p. 21
• The 2020 publication date does not establish when research began: it may have started before retraction or already been under review, as well as after retraction.The authors nevertheless conclude that 18 months after retractions, they had not had the hoped-for effect.Source: E Supplement: Retractions of DukeMTMC… · p. 21
• The DukeMTMC explanation was not posted on its dataset website, MS-Celeb-1M’s removal cited the ended research challenge rather than ethics, and neither retraction asked users to stop using the data.The authors note that not all users would encounter DukeMTMC’s explanation.Source: E Supplement: Retractions of DukeMTMC… · p. 21
• The dataset websites’ disappearance also removed access to license information; the MS-Celeb-1M license was located only in an archived version.That license prohibited redistribution of the dataset or derivatives.Source: E Supplement: Retractions of DukeMTMC… · p. 21

F Supplement: Analysis of license restrictions

• DukeMTMC's license allows non-commercial production use, while commercial use of models trained on DukeMTMC or MS-Celeb-1M remains legally ambiguous.DukeMTMC's license requires derivatives to include the original license, but terms are effective only when actively followed and inherited.Source: F Supplement: Analysis of license… · pp. 22–23
• LFW's lack of an original license left its use unrestricted until 2019, and its later disclaimer has no legal weight.The authors identified four commercial systems advertising performance on LFW; one quoted system claims 99.88% performance sourced to LFW.Source: F Supplement: Analysis of license… · p. 23
• LFW's relatively small size makes production training unlikely, but its use as a commercial benchmark can encourage overconfidence among system creators and clients.Source: F Supplement: Analysis of license… · p. 23
• Only 3 of 21 repositories containing MS-Celeb-1M-pretrained models included the original non-commercial designation, and seven used commercial-permitting MIT licenses.One repository used Apache 2.0, one used BSD-2-Clause, and nine had no license; only four of seven MS-Celeb-1M derivative datasets carried the designation.Source: F Supplement: Analysis of license… · p. 23

G Supplement: Posts discussing legality of pre-trained models

• The authors searched Google and four online platforms for discussions of commercial use of models pretrained on non-commercial data, examining the top 10 Google results per search.They also followed relevant links found in those discussions.Source: G Supplement: Posts discussing… · p. 24

H Supplement: Dataset management and citation

• None of the 38 datasets analyzed had a standardized permanent identifier, and none was managed through a shared repository.After DukeMTMC and MS-Celeb-1M were taken down, their terms-of-use sites were also unavailable.Source: H Supplement: Dataset management and… · pp. 24–25
• 42 papers included dataset references the authors could not fully disambiguate, often because papers conflated a dataset with derivatives or failed to identify a specific derivative.Examples include nine papers attributing annotations to LFW without specifying their source and seven papers referring to a cleaned MS-Celeb-1M without naming the derivative.Source: H Supplement: Dataset management and… · p. 25
• Only six of 60 sampled papers using the datasets or derivatives provided access information, and citations of associated papers do not directly convey dataset access.The authors were unable to locate three datasets even after searching archived sites.Source: H Supplement: Dataset management and… · pp. 25–26
• Associated-paper citations are poor proxies for dataset use, and current practices make tracking dataset use and derivative creation difficult.For DukeMTMC-ReID, 29 papers cited the original dataset, 63 cited the derivative, and 50 cited both; existing derivative lists were incomplete.Source: H Supplement: Dataset management and… · p. 26

I Supplement: Identifying models pre-trained on MS-Celeb-1M and ImageNet

• The authors searched linked repositories, third-party implementations, and GitHub results for models pretrained on six MS-Celeb-1M model classes, identifying 21 repositories.A parallel search of the first 20 GitHub results for “ImageNet pretrained” identified 12 repositories.Source: I Supplement: Identifying models… · p. 27

J Recommendations: The role of the IRB

• IRBs can review datasets before creation and may bring ethics expertise, but their narrow human-subjects remit excludes many machine-learning datasets and downstream harms.The authors advise researchers and IRBs to clarify that IRB approval does not mean research is ethical.Source: J Recommendations: The role of the IRB · p. 28