Source-linked AI summary
Position: AI Lock-In Is in Progress, and We Must Be Prepared
Jaeho Kim, Seokhyun Lee, Jieun Lee, Changhee Lee
TL;DR
AI safety research has not adequately addressed the risks of dependence on AI systems themselves. This position paper examines how AI Lock-In can emerge across individual, organizational, and national levels and argues that proactive mitigation is needed to preserve human autonomy and resilience.
Problem
AI safety research has largely addressed model alignment and societal impacts, leaving the risks of growing dependence on AI systems underexplored.
Method
The paper uses scenarios across individual, organizational, and national levels to analyze AI Lock-In’s escalation and outline mitigation and preparedness guidance.
Results
The paper concludes that AI adoption is pushing society toward deskilling and increasingly embedded reliance that erodes autonomous thought and skilled action.
Takeaways & Limitations
AI Lock-In should be addressed proactively, while sovereign AI can mitigate geopolitical vulnerabilities from foreign AI and cloud-platform dependence without eliminating the broader risk.
Takeaways & Limitations
The position’s scope is limited by uneven global AI development, making its concerns less immediately relevant to countries with limited infrastructure and access.
Abstract
from arXiv · showhide
AI safety research has mainly focused on two areas: technical alignment (ensuring AI systems produce human-aligned outputs) and the regulation of generative AI's societal impacts (including unemployment risk and labor market disruption). However, an equally important dimension remains underexplored: the risk inherent in dependence on AI systems themselves. In this position paper, we argue that AI safety research should address AI Lock-In, the phenomenon whereby excessive reliance on AI systems leads to human deskilling, diminishes human capacity for independent functioning, and creates systemic vulnerabilities when AI systems become unavailable or compromised. We highlight that AI Lock-In is a systemic threat that is already emerging at individual, societal, and national levels, one that could be dramatically amplified by AI service disruptions or geopolitical conflicts. Drawing on detailed scenarios, we investigate how AI Lock-In emerges and escalates across multiple levels, ranging from individual skill atrophy to national-scale infrastructure failures. To address this, we provide guidance on how such risks can be mitigated and prepared for at each level. We contend that proactively addressing AI Lock-In before such dependencies become entrenched, or even irreversible, is essential for preserving individual autonomy and national security.
1. Introduction
AI Lock-In is an increasingly entrenched dependence on AI that raises vulnerability to failures, attacks, and geopolitical coercion while making reversion to human-only functioning increasingly costly. The paper proposes preserving resilience through AI-independent literacy, organizational bet-hedging, and national AI-resilience measures.
- AI Lock-In: AI Lock-In occurs when cognitive, cultural, economic, and infrastructural reliance on AI makes switching back to human skills increasingly prohibitive.Reversion to a pre-AI state may eventually become nearly impossible, leaving society to accept associated risks.
- Problem: AI Lock-In is already emerging through skill atrophy and strategic dependencies, increasing exposure to accidental failures, cyber attacks, and geopolitical coercion.The paper characterizes the risk as ongoing rather than speculative and links it to a new geopolitical AI arms race.
- Individuals: Individuals should develop both AI-engaged literacy and AI-independent literacy to use AI critically while preserving the capacity to function without it.The paper defines these as complementary capacities for living with and without AI.
- Organizations: Organizations should retain some human workers in automatable roles and continue training junior employees, sacrificing short-term efficiency to preserve tacit knowledge and resilience.This bet-hedging strategy avoids maximum immediate productivity gains while maintaining human capability.
- Nations: Nations should require AI-independent operation of critical infrastructure, incentivize organizational bet-hedging, and conduct periodic AI-free drills.These measures prepare individuals and organizations for sudden disconnection from AI services.
2. The Lock-In Theory
AI Lock-In describes a shift from an initially advantageous option among many to dependence on a technology as alternatives disappear. Society is approaching this juncture as AI adoption increasingly replaces human-performed tasks.
- Lock-In Theory: Lock-In begins when a technology’s immediate advantages make switching alternatives seem irrational, but over time those alternatives may disappear until it becomes the only choice.This dependence can arise by design or through the gradual loss of competing options.
- Biological Specialization: Koalas illustrate how efficiency-driven specialization can become vulnerability when environmental change removes access to the resource on which survival depends.Their specialization in eucalyptus leaves left them unable to revert to alternatives as eucalyptus forests decreased.
- AI Adoption Trajectory: AI adoption follows a similar trajectory: initially a productivity-enhancing tool for writing, drawing, coding, and other tasks, it can progressively displace alternatives.The paper explicitly compares AI adoption with koala specialization in Table 1.
- Critical Juncture: Society remains skilled, but individuals, organizations, and nations are adopting or institutionalizing AI to replace previously human-performed tasks and improve productivity.The paper identifies this period as a critical juncture at which AI use is becoming the default.
3. The AI Lock-In is in Progress
AI Lock-In is already emerging as productivity gains encourage cognitive offloading, deskilling, and dependence across individuals, organizations, and nations. As adoption becomes embedded in workflows and public institutions, these dependencies can create systemic vulnerabilities if AI systems fail or are disrupted.
- Individual Level: AI use can speed individual task completion by approximately 80%, creating strong incentives to rely on AI for tasks previously performed through human cognition.The productivity gains vary across task types, but quantitative results indicate substantial individual-level improvements with AI use.
- Individual Level: Cognitive offloading to AI now spans memory, attention, reasoning, problem-solving, and basic writing, reducing the cognitive effort people invest in these functions.The section links this broad offloading to increasing reliance on AI for task completion.
- Individual Level: Continuous cognitive offloading causes deskilling, while indiscriminate AI use can atrophy human capabilities and strengthen dependency, threatening autonomous thought and skilled action.Evidence of deskilling is already emerging, including declines in doctors’ adenoma detection rates during non-AI-assisted procedures after exposure to AI-assisted colonoscopy.
- Organizational Level: Organizational AI Lock-In develops through reliance on AI-embedded workflows and erosion of human capital as AI progressively substitutes for entry-level positions.These pathways reinforce one another, fostering long-term reliance on AI and diminishing organizations’ capacity to operate without it.
- National Level: AI adoption at the national level elevates cognitive offloading across workforces and public institutions, accelerating deskilling through policies that encourage deployment throughout society.The section describes national initiatives such as the UK’s AI Opportunities Action Plan and U.S. provision of ChatGPT Enterprise to federal agencies and national laboratories.
- Systemic Vulnerability: If deeply rooted AI dependencies are disrupted by accidents, cyberattacks, or geopolitical coercion, productivity, coding, traffic systems, and autonomous vehicles could abruptly fail.The scenario depicts widespread operational stoppage when AI systems suddenly stop functioning.
4. Threat Scenario
AI Lock-In makes accidental failures, cyber attacks, and geopolitical coercion more likely while reducing humans’ capacity to respond. Interdependent AI systems, vulnerable supply chains, concentrated providers, and deepening adoption can turn disruptions into widespread systemic threats.
- Three Threat Scenarios: AI disruption can arise through accidental system failures, cyber attacks, or geopolitical coercion, with the main danger being diminished human capacity to respond.The paper argues that disruption becomes more frequent and likely as AI Lock-In deepens.
- Accidental Failures: As AI systems and software services become interdependent, minor errors can propagate across complex infrastructures, making Normal Accidents increasingly routine.Multi-agent systems, inference pipelines, and software updates may depend on services from multiple providers.
- Accidental Failures: The 2016 Left-Pad removal disrupted thousands of projects and services, including Facebook, Netflix, and PayPal, affecting millions of users for several hours.The package contained only 11 lines of code, illustrating how a small dependency can produce large-scale disruption.
- Cyber Attacks: Cyber attacks can exploit AI capabilities and software supply chains, while coordinated attacks can compromise multiple AI systems across sectors and amplify societal damage.The LiteLLM attack involved malicious packages that leaked users’ credentials, API keys, and SSH keys.
- Geopolitical Coercion: Dependence on a small number of cloud, GPU, and API providers creates vendor Lock-In that could be exploited through geopolitical conflicts or service compromises.The paper distinguishes AI dependency itself from dependence on foreign providers: domestic infrastructure and Sovereign AI initiatives do not eliminate AI Lock-In risk.
- Need for Intervention: The intervention window is narrowing as AI adoption accelerates, even though society remains capable of choosing a less AI-Locked-In trajectory.The paper presents accidental failures, coordinated cyber attacks, and geopolitical coercion as highly probable if AI Lock-In deepens.
5. Call to Action: Building Resilience
Building resilience against AI Lock-In requires coordinated action across individuals, organizations, and nations. The paper advocates dual AI literacy, organizational bet-hedging, and national resilience measures to preserve human and societal functioning during AI disruption.
- Individual resilience: AI literacy should include both critical AI use and the ability to function independently when AI is unavailable.The paper calls these dimensions AI-engaged literacy and AI-independent literacy, and considers both essential and complementary.
- Individual resilience: Practicing manual writing, reading, and other tasks helps preserve cognitive skills that diminish when consistently offloaded to AI.AI-independent literacy accepts modest efficiency trade-offs to maintain resilience during sudden AI unavailability.
- Institutional support: Organizations and nations must fund AI literacy and public-awareness efforts so people can use AI knowledgeably while retaining cognitive autonomy without it.Individual measures are insufficient without institutional support because deskilling can produce systematic dependency.
- Organizational resilience: Organizations should bet-hedge AI workflows by retaining junior employees to review AI outputs while senior professionals provide final approval and developmental feedback.This approach sacrifices short-term efficiency but helps detect errors, sustain tacit-knowledge transfer, and preserve the human-capital pipeline.
- National resilience: Nations should mandate resilience audits of critical infrastructure, incentivize junior hiring and training, and conduct AI-free drills to preserve essential functions during outages.The paper argues that these costly preemptive investments will be smaller than the societal costs after AI Lock-In becomes entrenched.
6. Alternative Views
The paper considers whether AI Lock-In repeats historical technological anxieties and whether ubiquitous, reliable AI access could eliminate disconnection risks. It argues that AI Lock-In remains qualitatively distinct because AI-driven deskilling can erode higher-order judgment and autonomy even when access is uninterrupted.
- A1. Recurring Pattern of Technological Anxiety: Historical warnings that technology weakens human intellect or autonomy have accompanied innovations from the Spinning Jenny to smartphones, yet humanity has adapted.The paper presents this recurring pattern as an alternative perspective challenging its central argument.
- A1. Recurring Pattern of Technological Anxiety: AI Lock-In is qualitatively distinct from earlier technologies because AI can disrupt human autonomy in cognitive judgment, not merely replace isolated skills.Calculators and GPS replaced specific functions while leaving higher-order judgment—such as recognizing when and how to use them—intact.
- A1. Recurring Pattern of Technological Anxiety: AI is penetrating and disrupting occupations at unprecedented pace and breadth, giving society drastically less time to adapt than earlier automation allowed.Earlier automation unfolded sequentially, with new roles emerging around mechanized systems.
- A2. Development of Technology: On-device AI and related advances could provide reliable access anywhere and anytime, potentially resolving the sudden-disconnection scenarios described in the paper.The paper acknowledges these technical directions as valuable and notes that current technological progress may solve such access problems in the near future.
- A2. Development of Technology: Constant AI access would not prevent cognitive offloading from causing deskilling, as doctors may lose independent diagnostic ability and students may struggle to solve problems without assistance.This degradation could occur because AI continually performs tasks for users, rather than because the system becomes unavailable.
7. Discussion and Limitation
The discussion distinguishes AI Lock-In from vendor Lock-In, argues for adult AI re-education and Sovereign AI as partial mitigations, and emphasizes user agency. It also acknowledges limitations from uneven AI access and the correlational evidence available for an emerging phenomenon.
- User Agency Matters: Technology dependence varies by individual use: critical users may realize more benefits, while passive users face greater negative effects.The authors reject a linear or deterministic account of technology’s impact.
- Need for Adult Re-education: AI literacy should extend beyond K-12 curricula to adult re-education because many working adults use AI without safeguards.The proposal targets adults who lacked AI literacy in formal schooling but already frequently use AI tools.
- Investment on Sovereign AI: Sovereign AI cannot eliminate AI Lock-In but may hedge against vendor Lock-In by diversifying AI infrastructure and service sources during geopolitical coercion.A nation with Sovereign AI remains vulnerable to AI Lock-In.
- Distinguishing AI Lock-In from Vendor Lock-In: AI Lock-In concerns dependence on AI itself, whereas vendor Lock-In concerns dependence on a specific provider; the two risks can compound.Addressing vendor Lock-In does not resolve AI Lock-In, which the authors identify as the more fundamental problem.
- Limitation: The position is limited by uneven global AI development, while its evidence remains correlational because AI Lock-In is still an emerging and ongoing process.AI use is concentrated in nations with strong digital infrastructure and capital, potentially making these concerns less immediately relevant to developing countries.
8. Conclusion
The paper argues that rapid, indiscriminate AI adoption is moving society from skilled to deskilled, threatening individual and societal autonomy. Beyond a critical threshold, individuals, organizations, and nations may become AI Locked-In.
- Conclusion: Rapid, somewhat indiscriminate AI adoption is pushing society from a skilled state toward a deskilled one.The paper identifies this transition as a critical juncture for society.
- Conclusion: AI-driven convenience may come at the cost of individuals’ and societies’ autonomy.
- Conclusion: After the deskilling threshold is crossed, individuals, organizations, and nations may become AI Locked-In.The paper frames AI Lock-In as a state of reliance on AI systems.
A. Lock-In Case Studies
The case studies show that AI Lock-In has documented historical precedents: automation can erode critical skills, while dominant platforms can create population-scale dependency. AI systems may combine both dynamics across professional and personal domains.
- Historical precedents: Historical technology-induced lock-in has produced measurable harm, showing that AI Lock-In mechanisms are neither speculative nor novel.The case studies are presented as precedents to motivate and inform responses to AI Lock-In.
- Case 1: Cockpit Automation and Pilot Deskilling: Cockpit automation improved aviation safety and efficiency but gradually eroded pilots’ manual flying skills, contributing to catastrophic consequences.The paper identifies Air France Flight 447 as a case in point.
- Case 1: Cockpit Automation and Pilot Deskilling: 228 people died on Air France Flight 447 after unreliable speed indications caused by blocked pitot tubes disconnected the autopilot, and none of the three crew members recovered the aircraft.The BEA attributed the failure to an inability to diagnose the stall and take effective recovery actions.
- Institutional mitigation: Aviation institutions countered automation-induced deskilling through increased manual flying, basic airmanship instruction, high-stress simulator scenarios, and manual proficiency exercises.Recommendations covered initial and recurrent training, feasible line operations, upset recovery, stall prevention, go-arounds, and visual approaches without full automation.
- Case 2: Platform Monopoly Lock-In: E-commerce platform monopoly illustrates population-scale dependency created by the absence of viable alternatives, while Amazon’s ecosystem spans logistics, streaming, smart-home devices, and cloud computing.The paper links this dependency to repeated large-scale privacy violations, including a C746 million CNPD fine in 2021 for targeted advertising without valid consent.
- AI Lock-In convergence: AI systems converge both dynamics by inducing cognitive deskilling and creating ecosystem-level structural dependency across virtually all professional and personal domains.The paper contrasts this convergence with the separate dynamics illustrated by aviation and e-commerce.
B. Operationalizing AI Literacy in Scale based on Media Literacy · B.1. Structured AI Literacy Workshop
The paper operationalizes AI literacy through a structured workshop based on five core concepts and proposes a multi-level AI Lock-In Index to monitor dependency. The workshop progresses from verifying outputs to examining user variation, cultural gaps, and engagement-oriented system design.
- B. Operationalizing AI Literacy in Scale based on Media Literacy: The proposal combines a structured AI literacy workshop based on five core concepts with a multi-level AI Lock-In Index for monitoring dependency at micro and macro levels.The workshop and index are presented as complementary operationalizations of AI literacy and AI Lock-In monitoring.
- B.1. Structured AI Literacy Workshop: The workshop adapts the Center for Media Literacy’s Five Core Concepts into concrete exercises deployable across schools, organizations, and national education initiatives.This creates a unified AI literacy program rather than a set of unrelated activities.
- B.1. Structured AI Literacy Workshop: Exercise 1 trains participants to request original sources for AI-generated factual outputs and cross-check them against primary materials.The exercise addresses risks of unverified information by institutionalizing source verification habits.
- B.1. Structured AI Literacy Workshop: Exercise 2 uses confidently worded but incorrect AI responses to train participants to separate linguistic fluency from factual correctness.It targets the illusion of competence and unwarranted trust in authoritative-sounding outputs.
- B.1. Structured AI Literacy Workshop: Exercise 3 has participants compare responses to the same question, revealing how prompting habits, phrasing, and conversational history can produce user-specific filter bubbles.The comparison makes divergent outputs visible and supports discussion of how input framing shapes responses.
- B.1. Structured AI Literacy Workshop: Exercise 4 asks participants to examine AI responses on culturally sensitive or globally relevant topics for included and omitted perspectives.The breakfast example illustrates how Western descriptions may appear while other cultural traditions are omitted.
- B.1. Structured AI Literacy Workshop: Exercise 5 examines engagement-sustaining design patterns, such as follow-up questions, to show how commercial choices can shape usage behavior.Participants assess whether continued interaction reflects their needs or the service’s engagement objectives.
- B.1. Structured AI Literacy Workshop: The five exercises form one structured program progressing from output verification to user and cultural variation, then to systemic AI service design.This progression develops AI-engaged literacy through increasingly broad levels of critical interaction with AI.
C. Role of AI Researchers … C.1.2. MACRO-LEVEL: STRUCTURAL DEPENDENCY MEASUREMENT
The paper calls on AI researchers to measure, design for, and test against AI Lock-In, proposing an index that tracks dependency at individual, organizational, and national scales. Its macro-level measures assess institutional embedding, workforce effects, outage impact, infrastructure reliance, provider concentration, and AI-free fallback capacity.
- C. Role of AI Researchers: AI researchers should measure current AI Lock-In, design models to preserve human agency, and develop dependency-testing frameworks analogous to red teaming.The paper also calls for organizational and national support for these technical research directions.
- C.1. Development of Multi-Level AI Lock-In Index as a Measurement Tool: The proposed AI Lock-In Index is a standardized framework for periodically monitoring dependency progression across micro-level individual and macro-level structural dimensions.It is intended for international and national organizations and recognizes that AI Lock-In operates simultaneously across both scales.
- C.1.1. MICRO-LEVEL: INDIVIDUAL DEPENDENCY MEASUREMENT: At the individual level, the index measures cognitive dependency through self-reported reliance frequency, AI-independent confidence, and preemptive AI consultation.These dimensions are measured on 7-point Likert scales for longitudinal tracking.
- C.1.1. MICRO-LEVEL: INDIVIDUAL DEPENDENCY MEASUREMENT: A metacognitive calibration test compares participants’ trust in AI accuracy with the actual accuracy of AI responses to identify overreliance that self-reports may miss.The behavioral assessment uses domain-relevant questions and complements self-report data.
- C.1.2. MACRO-LEVEL: STRUCTURAL DEPENDENCY MEASUREMENT: At organizational and national levels, the index assesses how deeply AI is embedded in institutional workflows and critical infrastructure, capturing systemic dependency that compounds individual reliance.The macro-level layer covers both organizations and nations.
- C.1.2. MACRO-LEVEL: STRUCTURAL DEPENDENCY MEASUREMENT: Organizational indicators include AI-embedded workflow proportion, workforce substitution rate, AI outage impact, and human capital pipeline health.The final indicator uses entry-level hiring and junior-to-senior progression ratios to assess whether tacit knowledge transfer is maintained.
- C.1.2. MACRO-LEVEL: STRUCTURAL DEPENDENCY MEASUREMENT: National indicators include critical infrastructure AI reliance, AI service provider concentration, and AI-free fallback availability for critical systems.These measures cover essential-service dependence, concentration among a small number of providers, and continued operation without AI.
C.1.3. FROM INDEX TO ACTION
The AI Lock-In Index is intended to guide policy intervention, not merely diagnose AI dependency. By tracking micro- and macro-level indicators over time, it can help identify critical thresholds and prompt resilience measures.
- From Index to Action: The AI Lock-In Index provides a basis for policy intervention beyond serving as a diagnostic tool.Its purpose is to support action against emerging AI dependency.
- From Index to Action: Tracking micro-level and macro-level indicators over time enables governments and organizations to monitor AI dependency.The index is described as analogous to GDP and the Human Development Index, which track economic output and societal well-being.
- From Index to Action: The index can identify when AI dependency approaches critical thresholds and support implementation of proposed resilience measures.The passage specifically names AI literacy programs among the measures proposed in Section 5.