Source-linked AI summary
Do Not Copy/Paste: Soft Barriers for Copying in AI-Assisted Programming
Iyiola E. Olatunji, Alberick Euraste Djire, Jacques Klein, Tegawendé F. Bissyandé
TL;DR
AI coding assistants leave the handoff from generated text to executable software largely unmanaged, creating a need for mechanisms that discourage unexamined transfer. This paper probes output-level soft barriers using Unicode perturbations and finds high but model- and task-dependent copy-paste resistance, with pilot evidence of more editing and reconstruction.
Problem
AI coding assistants leave the transition from generated code to executable or committed software largely unmanaged, despite risks in education, review, and security-sensitive development.
Method
The paper proposes soft barriers and probes them with Unicode output perturbations, evaluating copy-paste resistance across models and tasks alongside an exploratory human pilot.
Results
Unicode barriers achieve high copy-paste resistance, but effectiveness varies sharply by model, task, and perturbation family; the pilot also indicates more editing and reconstruction.
Takeaways & Limitations
The findings motivate handoff-aware programming assistants that treat the boundary between generated text and executable software as a design surface.
Takeaways & Limitations
Unicode barriers are not secure because users can request clean output, normalize text, or retype the code.
Abstract
from arXiv · showhide
Copying a function from a chat window into an editor takes less than a second. For many uses of AI coding tools, that speed is the point; in settings such as programming education, code review, and security-sensitive development, it can also be the problem. This paper frames copy-paste as an \emph{AI code handoff problem}: the moment model-generated text crosses from a conversational context into executable or committed software is a design boundary that current tools leave largely unmanaged. We argue that AI coding assistants should not only be evaluated by the code they generate, but also by how they mediate the transfer of that code into software artifacts. We propose \emph{soft barriers} as one class of handoff-aware mechanisms. Soft barriers preserve access to AI assistance while making unexamined transfer less frictionless. As an initial technical probe, we instantiate this idea using Unicode output perturbations that preserve visual readability but disrupt naive copy-paste execution. We introduce Copy-Paste Resistance (CPR), the fraction of functionally correct clean solutions that become syntactically invalid after perturbation. Across HumanEval and MBPP with four LLMs and four perturbation families, we find that output-level barriers can achieve high copy-paste resistance, but their effectiveness is highly model- and task-dependent. An exploratory pilot with 18 participants provides early evidence that soft barriers can shift users from direct transfer toward editing and reconstruction. We do not present Unicode perturbations as a deployment-ready solution; rather, we use them as a minimal probe for a broader research agenda on practical, transparent, and policy-aware AI code handoff.
1 Introduction
AI coding assistants create an unmanaged handoff boundary when generated code moves from conversation into software artifacts, potentially preceding understanding, testing, review, or attribution. The paper treats this handoff as a design surface and proposes soft barriers that preserve AI access while making unexamined transfer less frictionless.
- AI code handoff problem: The AI code handoff problem is the movement of generated code from a chat window into an editor, notebook, repository, or pull request.This boundary is unmanaged when code crosses into execution before it has been understood, tested, reviewed, or attributed.
- Motivation: AI assistants make natural-language requests into working code with unprecedented immediacy and little adaptation.The paper distinguishes this from earlier copying because generated artifacts are often complete enough to run directly.
- Motivation: Frictionless transfer can bypass code review, security analysis, provenance tracking, and license checks beyond educational settings.Surveys also report reduced critical engagement when people use AI-generated content.
- Contribution: Soft barriers preserve access to AI-generated code while nudging users away from unexamined transfer through added friction.They are proposed as handoff-aware mechanisms rather than bans on AI use or blocks on generated-code access.
2 Unicode Soft Barriers as a Probe
This section treats Unicode perturbations as an output-level probe of soft-barrier handoff, preserving visual readability while discouraging unexamined copy-paste transfer. The mechanism tests whether the handoff boundary can be shaped without retraining models or modifying execution environments.
- Soft-barrier mechanism: Unicode perturbations preserve visual readability while inserting characters that can disrupt parsing after copy-paste.This exposes a gap between code’s displayed appearance and its internal representation.
- Observed effect: A homoglyph intervention can appear visually valid yet trigger an execution error when pasted into an editor.This illustrates how visual readability and executable validity can diverge at the copy-paste boundary.
- Soft-barrier mechanism: The intervention operates at the output layer, testing handoff shaping without retraining the model, changing the decoder, or modifying the execution environment.The mechanism is implemented through a system prompt that instructs Unicode insertion during generation.
- Implementation: Specific Unicode characters are injected into user-defined identifiers while keywords, operators, literals, and built-in names remain unchanged.The intervention targets generated identifiers rather than the surrounding code elements.
3 Evaluation
The evaluation tests whether output-level mechanisms disrupt naive transfer of correct generated code, whether effects vary across models and tasks, and whether friction changes user behavior. It uses HumanEval and MBPP while separating correctness degradation from handoff-barrier effects through Pass@1 and CPR.
- Evaluation goals: The evaluation examines whether output-level barriers disrupt naive transfer, remain stable across models and tasks, and influence user behavior.
- Evaluation setup: The study evaluates 164 hand-written Python problems from HumanEval and 974 entry-level tasks from MBPP.
- Metrics: Pass@1 measures whether the first generated solution passes all tests and is reported for clean and perturbed solutions.This characterizes correctness degradation under intervention.
- Metrics: CPR isolates handoff-barrier effects from generation failures by measuring executability changes among functionally correct clean solutions.
- Metrics: A CPR of 1.0 means every correct solution becomes syntactically unexecutable after intervention, whereas 0.0 means the perturbation has no effect on executability.
4 Results
The results argue that Pass@1 can misrepresent soft-barrier performance, so Copy-Paste Resistance (CPR) is used to measure disruption of naive transfer more directly. CPR varies substantially by model and perturbation family, with Claude highly affected on MBPP but showing one ineffective HumanEval family.
- Copy-Paste Resistance: CPR more directly measures whether a soft barrier disrupts naive copy-paste execution than Pass@1.The results analyze the same outputs with CPR after first examining Pass@1’s misleading narrative.
- False Narrative of Pass@1 under Soft Barriers: Pass@1 is insufficient for evaluating soft barriers because it conflates immediate executability with the intervention’s goal of preserving readability while disrupting naive transfer.A lower Pass@1 under intervention does not necessarily indicate barrier failure.
- Model- and Task-Dependent Effects: 0.997, 0.986, 0.982, and 0.975 CPR: Claude reaches these values on MBPP for invisible characters, homoglyphs, deletions, and reorderings, respectively.Claude is the most affected by the tested soft barriers on MBPP.
- Model- and Task-Dependent Effects: 0.943, 0.846, and 0.766 CPR: Claude shows strong HumanEval resistance for reorderings, deletions, and invisible characters, respectively.Homoglyphs are ineffective for Claude on HumanEval, reaching 0.008 CPR.
5 Exploratory Behavioral Pilot
An exploratory pilot with 18 participants tested whether Unicode-based soft barriers change how users transfer AI-generated code. The results provide preliminary evidence of less direct copying, more editing and reconstruction, and higher reported understanding and learning benefit without preventing task completion.
- Study design: The pilot involved 18 participants using GPT-5.2 in a browser-based programming tool with an editor and embedded AI assistant.Participants completed a warm-up task followed by two Python tasks: range formatting and meeting-conflict detection.
- Behavioral results: Group A reported less direct copying and more code modification than Group B across both tasks.Reported direct-copying/code-modification values were 4.00 vs. 1.00 for Task 1 and 3.67 vs. 1.00 for Task 2.
- Behavioral results: Group A reported higher code understanding and greater learning benefit than Group B for both tasks.Code-understanding/learning-benefit values were 4.00 vs. 2.83 for Task 1 and 5.00 vs. 3.50 for Task 2.
- Interpretation: Overall, the pilot provides preliminary evidence that Unicode-based soft barriers shift AI-code handoff from direct transfer toward active editing and productive friction.The reported shift occurred without preventing task completion.
- Participant interpretation: Participant responses described the barrier as requiring fuller understanding of assistant-generated code before modification or execution.Responses characterized copy-paste problems as forcing participants to read and fully understand the code before executing it.
6 Discussion
The AI code handoff problem extends beyond programming education to professional software engineering and campus-managed assistants. Soft-barrier designs could serve as policy layers between LLMs and development or learning systems, with context-specific defaults.
- Broader Implications Beyond Education: The AI code handoff problem generalizes beyond programming education to professional software engineering.In professional settings, AI-generated code may enter codebases without review, tests, or provenance tracking.
- Broader Implications Beyond Education: Campus-managed assistants could support explanation-first, test-gated, or soft-barrier modes.The passage gives UniGPT-style services as an example of campus-managed assistants that could adopt these modes.
- Broader Implications Beyond Education: The proposed designs could form a policy layer between an LLM and an IDE, LMS, or repository.Different defaults could be configured for different contexts, including beginner users.
7 Conclusion
The paper frames the transfer of AI-generated code into running programs as an AI code handoff problem that current evaluation and tooling leave unmanaged. It proposes soft barriers and Unicode perturbations as an initial probe, with Copy-Paste Resistance isolating handoff friction from generation quality.
- Conclusion: The AI code handoff problem concerns the consequential, currently unmanaged moment when generated code enters a running program.The paper argues that evaluating code generation alone misses this transfer boundary.
- Conclusion: Soft barriers are proposed as output-level mechanisms for shaping the handoff boundary.They are intended to address transfer into software artifacts rather than code generation alone.
- Conclusion: Unicode perturbations serve as the paper’s first technical probe of output-level handoff mechanisms.The probe operationalizes the soft-barrier concept at the generated-output layer.
- Conclusion: Copy-Paste Resistance isolates handoff friction from generation quality.The metric separates whether solutions are functionally correct before perturbation from the friction introduced during handoff.