Source-linked AI summary
From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI
Margaret-Anne Storey
TL;DR
Generative AI accelerates software development while potentially shifting risk from technical debt toward cognitive and intent debt. The paper proposes a Triple Debt Model spanning code, people, and externalized knowledge, and discusses practices for preserving understanding and intent. It concludes that healthy AI-assisted software requires deliberate investment in both, while debates remain about documentation and AI’s role in creating it.
Problem
AI can generate code faster than developers understand it, creating a need to account for erosion of shared understanding and missing externalized rationale alongside technical debt.
Source: Cognitive Debt: The Invisible Layer · pp. 2–3, Intent Debt: The Forgotten Layer · p. 4Method
The paper proposes a Triple Debt Model covering technical debt in code, cognitive debt in shared understanding, and intent debt in externalized goals, constraints, and rationale.
Source: Three Layers of Software System Health · p. 6Results
The paper identifies cognitive and intent debt as increasingly important risks in AI-assisted development and discusses practices for diagnosing and mitigating them.
Source: Cognitive Debt: The Invisible Layer · p. 2, Intent Debt: The Forgotten Layer · p. 4, Implications for Practice · p. 7Takeaways & Limitations
Teams should invest deliberately in shared understanding and intent artifacts alongside code quality when developing software with AI.
Source: Conclusion · p. 9Whether intent should be documented and whether AI should autogenerate such documentation remain unresolved practitioner debates.
Source: Should we document “intent”? · p. 8, Can AI help make implicit knowledge… · p. 8
Abstract
from arXiv · showhide
Generative AI is accelerating software development, but may quietly shift where the most significant risks lie. As AI generates code faster than teams can understand it, two under appreciated forms of debt accumulate: cognitive debt, the erosion of shared understanding across a team, and intent debt, the absence of externalized rationale that developers and AI agents need to work safely with code. This article proposes a Triple Debt Model for reasoning about software health, built around three interacting debt types: technical debt in code, cognitive debt in people, and intent debt in externalized knowledge. Cognitive debt is a team-level, project-level property reflecting the erosion of shared understanding across a software system over time, leading to increasingly inadequate shared mental models for reasoning about and safely changing the system. Intent debt refers to the absence or erosion of explicit rationale, goals, and constraints that guide how humans and agents evolve the system. We discuss how generative AI changes the relative importance of these debt types, how each can be diagnosed and mitigated, and surface points of debate for practitioners.
What is a Software System, Really?
A software system spans goals and intent, executable code and structure, and shared understanding. System health depends on alignment across all three layers, not code quality alone.
- Goals and intent include the requirements, constraints, and objectives captured in specifications, tests, and documentation.Source: What is a Software System, Really? · p. 2
- Code and structure implement system intent through source code, architecture, dependencies, and deployment infrastructure.Source: What is a Software System, Really? · p. 2
- Shared understanding consists of team members’ mental models of how the system works and can be reasoned about.Source: What is a Software System, Really? · p. 2
- Safe change and coordination require sufficient shared understanding across the team, rather than complete knowledge by one individual.Source: What is a Software System, Really? · p. 2
- System health depends on alignment among all three layers; unclear intent can cause purpose drift, while inadequate understanding makes safe change difficult.Source: What is a Software System, Really? · p. 2
Technical Debt: The Well-Known Layer
Technical debt is the long-term cost of code-quality shortcuts and constrained architectural decisions. Its visibility and established practices make it comparatively manageable, while generative AI increasingly automates remediation.
- Technical debt arises when short-term delivery takes priority over long-term code quality, producing messy code and constrained future evolution.Source: Technical Debt: The Well-Known Layer · p. 2
- Technical debt accrues interest because postponing remediation makes future system changes more costly.Source: Technical Debt: The Well-Known Layer · p. 2
- Technical debt is comparatively visible and manageable through practices such as test-driven development, refactoring, and code review.Source: Technical Debt: The Well-Known Layer · p. 2
- Generative AI increasingly supports technical-debt reduction through automated refactoring, code-smell detection, and test generation.Source: Technical Debt: The Well-Known Layer · p. 2
Cognitive Debt: The Invisible Layer
Cognitive debt is the team-level erosion of shared understanding that makes systems harder to reason about and change safely. AI-assisted development can accelerate this accumulation when developers accept generated code without building comparable mental models.
Cognitive Debt: The Invisible Layer
- Cognitive debt is a team- and project-level accumulation of gaps in shared understanding over time.Source: Cognitive Debt: The Invisible Layer · p. 2
- The paper distinguishes its usage from individual neural-engagement and comprehension-debt accounts by emphasizing team-level, longitudinal erosion.Source: Cognitive Debt: The Invisible Layer · pp. 2–3
- Cognitive debt is not new, but AI-assisted development may increase the rate of accumulation and make the gap harder to detect.Source: Cognitive Debt: The Invisible Layer · p. 3
- AI-generated code can be accepted without developers building the mental models needed to understand system behavior and reason about change.Source: Cognitive Debt: The Invisible Layer · p. 3
- Cognitive surrender adopts AI outputs with minimal scrutiny, potentially inflating confidence even when the AI is wrong.Source: Cognitive Surrender Leads to Cognitive… · p. 3
- Teams may deliberately trade understanding for delivery speed, accumulating cognitive debt while pursuing rapid feedback.Source: Cognitive Surrender Leads to Cognitive… · p. 3
- Signals include resistance to change, unexpected results, slow onboarding, weakened transactive memory, and a low bus factor.Source: Diagnosing Cognitive Debt · pp. 3–4
- Mitigation combines human review and pair programming with walkthroughs, retrospectives, deliberate communication, and reimplementation.Source: Practices for Mitigating Cognitive Debt · p. 4
Intent Debt: The Forgotten Layer
Intent debt is the absence or erosion of explicit goals, constraints, rationale, and specifications needed to guide system evolution. The paper proposes living intent artifacts while noting that AI cannot replace human judgment in deciding what best serves users.
Intent Debt: The Forgotten Layer
- Intent debt accumulates when goals, constraints, specifications, and rationale are unclear, poorly articulated, or absent from consultable artifacts.Source: Intent Debt: The Forgotten Layer · p. 4
- Practices such as specifications, tests, domain knowledge, and design rationale may become critical again as AI generates more code.Source: Intent Debt: The Forgotten Layer · p. 5
- Requirements documents, decision records, plans, tests, and specifications externalize system intent for both teams and AI agents.Source: Intent Debt: The Forgotten Layer · p. 5
- Intent is best captured when key decisions are made because recovering it later can be difficult or impossible.Source: Intent Debt: The Forgotten Layer · p. 5
- Recognizable symptoms include behavior drift, AI agents needing extensive clarification, and loss of articulated non-functional constraints.Source: Diagnosing Intent Debt · p. 5
- Intent-first workflows use executable specifications and tests, decision and rationale records, and context artifacts for AI-assisted development.Source: Practices for Creating and Maintaining… · pp. 5–6
- These approaches cannot replace human decision-making and creative judgment about which software solution best serves users.Source: Practices for Creating and Maintaining… · p. 6
Three Layers of Software System Health
Software system health depends on three interacting layers: code, shared understanding, and externalized intent. Their erosion limits changeability, safe reasoning, and continued alignment with what the team meant to build.
Three Debt Types
- Technical debt lives in code and accumulates when implementation decisions compromise future changeability.Source: Three Layers of Software System Health · p. 6
- Cognitive debt lives in people and grows when shared understanding erodes faster than it is replenished.Source: Three Layers of Software System Health · p. 6
- Intent debt lives in artifacts and arises when goals and constraints are poorly captured or maintained.Source: Three Layers of Software System Health · p. 6
Interactions
- The three debts reinforce one another: unclear intent impairs understanding, weak understanding encourages poor implementation decisions, and messy code further reduces understanding.Source: Three Layers of Software System Health · p. 6
- Managing software health requires proactive attention to all three layers rather than only the easiest one to measure.Source: Three Layers of Software System Health · p. 6
How AI Is Shifting the Balance
Generative AI may reduce technical debt through automation while increasing the relative importance of cognitive and intent debt. It produces code faster than teams can understand it and can weaken the feedback that previously supported understanding and intent capture.
Changing Debt Dynamics
- AI-assisted refactoring, code review, and test generation may reduce technical debt accumulation and ease legacy-system improvement.Source: How AI Is Shifting the Balance · p. 6
- AI can partially mitigate cognitive and intent debt, but becomes a risk multiplier when humans surrender cognition and fail to capture intent.Source: How AI Is Shifting the Balance · p. 6
- AI generates code faster than teams can build the understanding required to change it safely.Source: How AI Is Shifting the Balance · p. 7
- Underspecified prompts, plausible-looking outputs, and reduced implementation friction can produce results that miss system needs or intent.Source: How AI Is Shifting the Balance · p. 7
Implications for Practice
The paper recommends treating understanding and intent as explicit parts of AI-assisted development. Teams should build genuine understanding, capture intent early, and monitor all three debt dimensions together.
Treat Understanding as a Deliverable
- Teams should treat shared understanding as a first-class deliverable supported by walkthroughs, retrospectives, knowledge transfer, and suitable tools.Source: Implications for Practice · p. 7
Intent-First Workflows
- Intent-first workflows capture requirements, constraints, decisions, plans, and acceptance criteria before AI-assisted development proceeds.Source: Implications for Practice · p. 7
Resist Automation of Understanding
- Teams should resist automating understanding because generated explanations can create its appearance without building accurate mental models.Source: Implications for Practice · p. 7
Monitor the Three Layers in Tandem
- Teams should monitor technical, cognitive, and intent debt together using measures such as onboarding time, knowledge concentration, requirements coverage, and intent-behavior audits.Source: Implications for Practice · p. 7
- Software systems depend on intent in artifacts, behavior in code, and understanding distributed across teams and artifacts.Source: Figure 1 · p. 8
Points for Debate
The paper identifies unresolved debate over whether intent should be documented and how AI should support that work. It also leaves uncertain how much cognitive debt is acceptable as development becomes more automated.
Should We Document Intent?
- Practitioners disagree about whether capturing intent is worth the effort, and the debate over documenting intent remains unsettled.Source: Should we document “intent”? · p. 8
Can AI Help Make Implicit Knowledge Explicit?
- Some propose that AI can surface implicit knowledge, while others value documentation primarily for the human understanding created during its production.Source: Can AI help make implicit knowledge… · p. 8
- Whether AI should autogenerate documentation remains an open question.Source: Can AI help make implicit knowledge… · p. 8
Debt as a Risk or Strategy
- The acceptable level of cognitive debt remains uncertain because understanding may be distributed across people and artifacts and may not require implementation-level detail.Source: Debt as a risk, or debt as a strategy? · p. 8
Conclusion
Healthy software in the AI-assisted era depends on preserving alignment among intent, code, and shared understanding. Teams should manage understanding with the same care and urgency as code quality.
Conclusion
- AI-assisted development may shift risk from technical debt toward cognitive and intent debt.The paper frames this as a hidden reversal requiring tools and processes to support paying down both forms.Source: Conclusion · p. 9
- Software teams that navigate AI-assisted development successfully will invest deliberately in understanding and intent, not only code quality.Source: Conclusion · p. 9
- Maintaining software health requires alignment among intent, code, and shared understanding.Source: Conclusion · p. 9