Source-linked AI summary
The Reported Engagement with AI Level (REAL) Rating: A Framework for Disclosing Human-AI Collaboration
Imène Goumiri, Mayleen Cortez-Rodriguez, Eric Bell, Amanda Muyskens
TL;DR
Existing provenance and disclosure approaches often rely on binary labels or technical watermarking that lacks user-facing clarity, despite differing uses of AI in production. The paper proposes the REAL Rating, a six-level disclosure framework for AI involvement in creation and user experience. It presents the framework as a way to provide audiences and institutions context about content origins and production methods.
Problem
Current disclosure methods often use binary categories or watermarking without user-facing clarity, despite differences in AI use across production workflows.
Source: §1 · pp. 1–2Method
Results
The paper presents REAL as a criteria-based method for disclosing human–AI collaboration and providing audiences and institutions context about content origins and production methods.
Source: §5 · p. 9Takeaways & Limitations
Standardized disclosure gives journals a transparency framework and can let reviewers focus on the research rather than reverse-engineering its production process.
Source: §4 · p. 9REAL is self-reported and relies on creators’ honest and accurate self-assessment.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Generative artificial intelligence has become increasingly incorporated into digital media and more generally into production workflows with which the public frequently interacts. Current provenance standards and disclosure methods frequently rely on binary categorizations, differentiating only between entirely human-authored and AI-generated content, or depend on technical watermarking that lacks user-facing clarity. However, there are very different risks and outcomes depending on the different uses of AI in the creation and consumption of end products. Therefore, we propose the Reported Engagement with AI Level (REAL) Rating framework which addresses these concerns by introducing a six-tier scale (Levels 0-5) that discloses the extent of AI involvement in both product creation and user experience. This paper details the structural methodology of the REAL Rating system and the specifics of its application across text, image, audio, video, and software products.
1 Introduction
Existing disclosure approaches do not adequately communicate the spectrum of human–AI collaboration, while AI-generated content raises questions about authenticity and reliability. The paper proposes a voluntary, graduated vocabulary for disclosing AI involvement.
1 Introduction
- High-confidence AI hallucinations and the reliability risks of probabilistic systems make disclosure relevant to the scrutiny users apply to content.The passages describe hallucinations as factually incorrect or nonsensical information presented with high confidence.Source: §1 · p. 1
- AI tools range from basic spelling correction and noise reduction to generative systems, yet are often conflated, potentially mischaracterizing human authorship.The authors argue that a granular scale can acknowledge useful tools without implying that creators abdicated the creative process.Source: §1 · p. 2
- Detection tools can produce false positives and miss sophisticated AI content, while metadata may be stripped and binary labels cannot capture nuanced workflows.The cited limitations include disproportionate false positives for non-native English speakers [7] and fragile metadata that may not provide immediate user-facing disclosure.Source: §1 · p. 2
- The REAL Rating replaces binary assessments with a standardized, voluntary scale describing the degree of human–AI interaction.The authors establish Levels 0–5, rating and marking methodology, and implementation examples.Source: §1 · p. 2
- The framework draws a parallel to SAE J3016’s Levels 0–5, using a spectrum to express human intervention and clarify responsibility and agency.The automotive analogy contrasts graduated levels with the binary notion of “self-driving”.Source: §1 · p. 2
2 The REAL Rating Framework
The REAL Rating defines a general six-level scale for disclosing AI involvement in content creation or in the user experience. It pairs the levels with standardized visual markings and implementation guidance.
2 The REAL Rating Framework
- The framework rates AI involvement from Level 0 to Level 5, as disclosure rather than judgment, across media and products.The definitions are intentionally general; the rating concerns AI in creation or AI consumers may interact with while using a product.Source: §2 · p. 3
- Level 0 means no AI; Level 1 covers automation when ideas, structure, and presentation remain human-generated.Examples include spell-check, grammar correction, audio cleanup, and green-screen or cutout features.Source: §2 · p. 3
- Level 2 covers AI ideation or organization with a human-constructed final output; Level 3 covers generative components that do not alter the work’s core meaning.Examples include LLM-generated outlines or summaries for Level 2 and peripheral generative fill for Level 3.Source: §2 · p. 3
- Level 4 denotes substantial AI contribution, while Level 5 covers fully AI-generated media or documents and direct user interaction with AI models.Level 4 includes work equivalent to a text co-author or AI that substantially changes visual media’s meaning, context, or primary object.Source: §2 · p. 3
- The framework states that opportunities for bias, hallucination, and loss of human agency increase from Level 1 through Level 5.The authors say higher levels require greater consumer awareness.Source: §2 · p. 3
2.1 Markings and implementation
- Standardized visual identifiers are provided for each level and released into the public domain to encourage unrestricted adoption.The markings are intended for prominent placement in the user’s primary field of view; documents, visual media, and communications have suggested locations.Source: §2.1 · pp. 3–4, Figure 1 · p. 4
- For posted video, the marking need appear for only three seconds at the beginning, but should stay visible throughout live video.The paper also says marking aspect ratios, colors, and other identifying features should remain unchanged.Source: §2.1 · p. 4
3 System Application and Methodology
The REAL Rating uses modality-specific decision trees to rate mixed-media products, with the overall rating set by the highest-rated modality. Its visual, audio, and text evaluations distinguish different AI workflows.
3 System Application and Methodology
- Creators progress through binary decision-tree questions that route each modality to a low-involvement or high-involvement track before assigning its rating.Initial questions assess AI’s foundational role, including ideation and synthetic elements in source materials; subsequent criteria distinguish workflows.Source: §3 · p. 4
- A multimodal product receives the highest REAL level achieved by any constituent modality evaluated independently.The paper illustrates this with a game rated Level 4 overall because its AI-generated background music is Level 4 despite human-created code and models at Level 0.Source: §3 · p. 4
3.1 Visual Evaluation
- Visual evaluation distinguishes fully generated imagery, central modifications, peripheral generative fill, AI-assisted ideation, and basic automated adjustments.It covers static images, physical product designs, user interfaces, and video’s visual components.Source: §3.1 · p. 4, Figure 3 · p. 6
3.2 Audio Evaluation
- Audio evaluation covers AI voice generation, synthetic music, generated sound effects, and automated cleaning, separating audio generation from script or lyric drafting.The assessment applies to standalone tracks and audio within broader multimedia projects.Source: §3.2 · p. 7, Figure 4 · p. 7
3.3 Text Evaluation
- Text evaluation distinguishes direct LLM copying, substantial co-authorship, structural organization, and automated proofreading across prose, software, and production scripts.Its scope includes research and lyrics used in audio and video production.Source: §3.3 · pp. 7–8, Figure 5 · p. 8
4 Case Studies in Academic Research
The paper applies REAL ratings to hypothetical academic workflows, from fully manual research to AI-generated review writing. It argues that standardized disclosure can help reviewers assess research without reverse-engineering its production process.
4 Case Studies in Academic Research
- The academic examples are hypothetical scenarios intended as a baseline for consistent self-assessment.They illustrate levels of AI involvement across research workflows.Source: §4 · p. 9
- At Level 0, investigators handle data collection, analysis, literature review, and manuscript preparation manually.The example specifies manual searching and reading and typesetting without automated assistants or generative tools.Source: §4 · p. 9
- At Level 1, basic spelling and grammar checks and citation formatting coexist with a human-driven intellectual narrative and core content.At Level 2, an LLM may summarize gaps in selected literature, while humans perform subsequent experiments, interpretation, and writing.Source: §4 · p. 9
- The examples include AI-generated utility code alongside human-written algorithms and AI-edited prose whose meaning and content must not significantly change.The paper also describes Level 4 AI contributions to data-processing scripts and initial results, requiring intensive human validation and editing.Source: §4 · p. 9
- A Level 5 hypothetical has an automated agent conduct literature search, data extraction, and meta-analysis before drafting a review paper.Human involvement is limited to setting parameters and checking submission formatting.Source: §4 · p. 9
- Standardized disclosure gives journals a transparency framework and lets reviewers focus on the research rather than reconstructing its production process.The framework is presented as helping distinguish human-derived conclusions from those influenced by algorithmic processing.Source: §4 · p. 9
5 Conclusion
The REAL Rating offers a six-tier way to report AI involvement, giving audiences and institutions context to evaluate content origins and production methods. It is self-reported, complements rather than replaces technical provenance tools, and prioritizes transparent public communication.
5 Conclusion
- The REAL Rating replaces binary labels with a six-tier criteria-based system for reporting AI involvement and evaluating content origins and production methods.It gives audiences and institutions specific context for that evaluation.Source: §5 · p. 9
- The framework is self-reported and depends on creators' honest, accurate self-assessment.Source: §5 · p. 9
- REAL complements rather than replaces cryptographic provenance standards such as C2PA or automated detection, serving as a user-facing semantic layer.Source: §5 · p. 10
- Developed by independent nonprofit Real Good AI, the framework prioritizes transparent audience communication and public interest over proprietary or platform-specific disclosure tools.Source: §5 · p. 10
Generative AI Disclosure
The paper reports its own AI collaboration as REAL Level 2: Brainstorming, based on using AI to brainstorm and summarize research.
Generative AI Disclosure
- The paper classifies its own AI collaboration as REAL Level 2: Brainstorming.Source: Generative AI Disclosure · p. 10
- AI was used as a collaborative tool for brainstorming.Source: Generative AI Disclosure · p. 10
- AI was also used to summarize research.Source: Generative AI Disclosure · p. 10