PaperlayerView exact revision on arXiv

Source-linked AI summary

Factors that Affect Software Systems Development Project Outcomes: A Survey of Research

Laurie McLeod, Stephen G. MacDonell

arXiv:2101.08442v1cs.SE

TL;DR

Software systems project outcomes have long been studied, but the relevant influences remain difficult to organize and success remains challenging. This paper surveys empirical literature from 1996–2006 and synthesizes it into a four-dimensional classification framework, concluding that project factors interact dynamically and require more than generic prescriptions.

  • Problem

    Research has produced extensive but difficult-to-organize evidence about factors influencing software systems development and deployment project outcomes.

  • Method

    The paper surveys empirical studies published from 1996 to 2006 and synthesizes their reported factors into a classificatory framework.

  • Results

    The framework groups influences into people and action, project content, development processes, and institutional context, while empirical studies indicate that outcomes involve complex, dynamic interactions among factors.

  • Takeaways & Limitations

    Simple prescriptive lists of generic success factors are inadequate; future research should examine changing development practices and complex interrelationships among project factors.

Abstract

from arXiv · show

Determining the factors that have an influence on software systems development and deployment project outcomes has been the focus of extensive and ongoing research for more than 30 years. We provide here a survey of the research literature that has addressed this topic in the period 1996-2006, with a particular focus on empirical analyses. On the basis of this survey we present a new classification framework that represents an abstracted and synthesized view of the types of factors that have been asserted as influencing project outcomes.

1. INTRODUCTION

Research on software systems project success and failure has persisted for decades, yet achieving successful development and deployment remains difficult. The paper situates its survey within this longstanding effort and outlines a literature-based classification of influencing factors.

  • Researchers have investigated determinants of software systems project success and failure since at least 1975, with mixed results.
  • Despite greater understanding of how systems are developed, ensuring successful development and deployment remains a significant challenge.
  • The paper reviews prior classifications, presents a synthesized classificatory framework, and describes the empirical-review approach used to develop it.
  • The review examines how researchers define project outcomes and surveys potentially influential factors reported in empirical studies published from 1996 to 2006.

2. CLASSIFYING THE EMPIRICAL RESEARCH LITERATURE ON FACTORS INFLUENCING PROJECT OUTCOMES

The paper develops a contemporary framework for organizing empirical research on software systems project outcomes. It extends earlier classifications by integrating project content, development processes, people and action, and institutional context.

  • Prior classifications: Earlier classification schemes were limited by outdated evidence, narrow detail or scope, or attention to specific project types and factor subsets.
  • Survey contribution: The paper synthesizes empirical studies covering success, failure, performance, and abandonment across a wide range of project outcomes.
  • Conceptual basis: Content, process, and context provide a generic, parsimonious basis for classifying diverse influences on systems project outcomes.
  • Framework dimensions: People and action are added explicitly because software systems development is recognized as a social process shaped by human actors.
  • Framework dimensions: The framework organizes influences into project content, development processes, people and action, and institutional context.
  • Framework dimensions: Interactions among the four dimensions influence project trajectories and outcomes, whose evaluation is complicated by multidimensional, stakeholder-dependent, negotiated criteria.

1 While the use of overlapping circles to represent various dimensions

The review uses explicit inclusion criteria and database searching to synthesize empirical research into a comprehensive framework. Its analysis emphasizes broad coverage while recognizing scope boundaries and complex, interrelated influences.

  • Framework interpretation: The framework is intended as an analytical device for ordering extensive evidence without privileging any factor, because outcomes are highly situational.
  • Framework interpretation: Factors can interact in complex, dynamic ways, represented by overlapping dimensions encompassing project content, processes, people’s actions, and context.
  • Framework comparison: Compared with prior schemes, the new framework offers more comprehensive and systematic coverage across four dimensions and multiple analytical levels.
  • Review scope: The review primarily addresses software embedded in organizational or individual contexts and generally excludes operating, real-time embedded, and infrastructural systems.
  • Review scope: The review targets empirical studies of software systems project outcomes published from 1996 to 2006, primarily in archival journals.
  • Review method: Searches of two EBSCO databases used project success or failure with software or information-technology terms, initially identifying 289 publications.
  • Review method: Reviewed studies used surveys, interviews, Delphi studies, and case studies; reported factors were assigned to framework dimensions and refined into common categories.

3. PROJECT OUTCOMES

Project outcomes are difficult to define consistently because success and failure are multidimensional, contested, and assessed differently across stakeholders. The paper therefore treats outcomes broadly while preserving the empirical studies’ own terminology.

  • Success and failure lack consensus because they are vague, difficult to measure, and interpreted differently by different stakeholders.
  • Project outcomes encompass technical, economic, behavioral, psychological, and political dimensions.
  • Studies assess outcomes through process and product quality, including completion, budget, system quality, user satisfaction, and net benefits.
  • Development success, implementation success, and solution success may represent distinct outcomes rather than a single project result.
  • Outcome judgments can change over time and differ among sponsors, users, and other stakeholders.

4. INFLUENTIAL FACTORS – PEOPLE AND ACTION

People-and-action factors concern the individuals and groups involved in systems development, their roles, interactions, and competing commitments. Because roles depend on context and may change over time, rigid stakeholder categories can misrepresent participants’ perspectives.

  • Relevant roles include users, developers, managers, project leaders, team members, sponsors, customers, vendors, and top management.
  • Individuals may hold multiple roles or change roles and perspectives over time.
  • Actions are shaped by competing social, professional, organizational, societal, cultural, and sectional commitments.
  • Role definitions and group assignments depend on context and timeframe, and unreflective categorization can introduce researcher bias.

4.1 Developers

Developer characteristics and practices influence project outcomes through technical capability, communication, domain understanding, motivation, professional values, and assumptions about users. Effective development therefore involves both technical and social judgment.

  • Technical expertise, experience, training, and competent staffing are associated with positive project outcomes, while their absence may contribute to abandonment.
  • Problem-solving competency is perceived as critical because developers identify problems, generate solutions, review alternatives, and evaluate options.
  • Poor developer communication can reduce user satisfaction regardless of technical expertise.
  • Understanding organizational operations, culture, politics, user departments, and application domains can affect project outcomes.
  • Motivation and commitment support successful completion and are influenced by leadership, work environment, involvement, feedback, autonomy, and responsibility.
  • Professional norms may prioritize technical issues over human or organizational concerns, adversely affecting project outcomes.
  • Developers’ values and assumptions about users can embed inappropriate roles or usage patterns, producing dissatisfaction, resistance, or poor system-user fit.

4.2 Users

Users influence project outcomes through expectations, attitudes and involvement, and characteristics affecting system use. Research emphasizes managing expectations and recognizing that users are heterogeneous rather than a single uniform group.

  • Users comprise groups with different organizational locations, characteristics, interests, and perspectives rather than a homogeneous population.
  • Users affect outcomes through system expectations, attitudes and involvement, and characteristics influencing their ability to use the system.
  • User expectations may be explicit requirements or remain vague and unarticulated, shaping the project even when they are not formalized.
  • Higher user expectations increase the need to manage mismatches between expected and delivered systems.
  • Unmanaged or unrealistic expectations are identified as project risks and may inhibit successful development.
  • Negative attitudes can lead to resistance when systems seem irrelevant, alter work practices, or threaten professional identity and autonomy.
  • User involvement is defined as perceived importance and personal relevance, while insufficient support or commitment is associated with poorer project success.

4.3 Top Management

Top management is widely reported as influential to software systems project outcomes through sponsorship, resourcing, oversight, strategic alignment, and support for users and organizational change.

  • Top management support, commitment, or understanding is consistently reported as important for project outcomes across national and institutional contexts.
  • Senior management support is associated with projects being completed rather than redefined or abandoned, while its absence is implicated in challenged, abandoned, or failed projects.
  • Top management may be especially important for executive information systems, strategic or business-critical projects, high-interdependence projects, and projects challenging users’ professional autonomy.
  • Top management support helps secure project budgets and human resources, oversee development, and align projects with organizational strategies and goals.
  • Visible sponsorship can signal organizational commitment, encourage user participation, counter negative attitudes, and create a positive context for organizational change.
  • Boards of directors and other influential decision-makers may also affect projects, particularly through project sponsorship.

4.4 External Agents

External consultants and contractors increasingly connect system consumers with software vendors, but their involvement introduces contractual, communication, cost, support, participation, and organizational-change challenges.

  • External consultants have become increasingly important as organizations source made-to-order systems and ready-to-install software packages externally.They can bridge the gap between system consumers and software vendors.
  • External engagements may create contractual ambiguity over errors, enhancements, and unforeseen costs, alongside communication problems between consultants and organizational participants.
  • Additional risks include high expense, reduced user participation, and inadequate internal systems support after external consultants depart.
  • External developers may struggle when systems change work practices, task divisions, status, or authority because outsiders may miss organizational consequences and political undercurrents.They may focus more on technical aspects than on organizational implications.

4.5 Project Team

Project team composition, expertise, roles, relationships, and management practices shape outcomes, with evidence favoring effective, skilled, and appropriately sized teams over reliance on tools alone.

  • Strong project team effectiveness and cohesiveness are positively associated with project outcomes and performance.
  • Larger teams and teams without prior collaboration are identified as risks, while larger team size is associated with lower performance, dissatisfaction, reduced productivity, and weaker problem solving.
  • Teams with varied experience and skills are more likely to perform well across allocated tasks, undefined elements, uncertain objectives, and emerging issues.
  • Strong technical and interpersonal skills support task performance, user interaction, complex-requirements identification, and implementation success.
  • Team skill, project-manager involvement in day-to-day work, and similar experience levels influenced performance more than development methods and tools in requirements-analysis teams.Automated development tools had no explanatory effect on software product quality or project team performance in another study.
  • Unclear roles and responsibilities are perceived as completion risks and are significantly negatively related to system success.

4.6 Social Interaction

Software development is a social process in which interactions among diverse participants, their changing interests, and their communication practices influence project outcomes, collaboration, conflict, and shared understanding.

  • Social interaction: Software development involves interactions among participants occupying roles such as users, developers, managers, sponsors, customers, vendors, and consultants.
  • Experts and meaning: Experts can shape how others understand the development process, products, roles, and activities by managing meanings within the project.
  • Interaction quality: The quality of user–developer interactions can influence outcomes, while changes in group influence and critical encounters can alter a project’s course.
  • Communication: Workshops, intermediaries, prototyping, interviews, testing, surveys, email, bulletin boards, and observation provide channels for participant interaction and user–developer relationships.
  • Interaction quality: Emerging system artifacts mediate user–developer interaction by incorporating user reactions and exposing both groups to technological limitations.
  • Communication: Interaction can align goals, expectations, and understanding, but differences, misunderstandings, and communication breakdowns can produce contradictory outcomes.
  • Goals and expectations: Participants may differ in goals, expectations, cultures, professional interests, interpretive frames, and organizational interests, with these interests also changing during development.
  • Goals and expectations: Pre-project partnering was positively associated with project performance and was noted to reduce poor user support while improving project-team characteristics and project-manager performance.

5. INFLUENTIAL FACTORS – PROJECT CONTENT

Project content influences outcomes through project characteristics, goals, resources, and technology. Empirical studies associate size, complexity, organizational newness, resource provision, and technology choices with varied project results.

  • Project characteristics: Project size, technical complexity, and organizational newness are associated with software project outcomes.Large projects can involve greater complexity, interdependence, duration, resource needs, communication, and conflict; organizationally new projects may lack relevant knowledge and skills.
  • Project characteristics: Smaller projects may have better-defined domains, while large projects can be decomposed into smaller sub-projects or developed incrementally.These approaches are described as efforts to increase the likelihood of successful outcomes.
  • Project scope, goals and objectives: Clear, achievable goals and appropriate scope are repeatedly associated with more successful project outcomes.Ambiguous goals can produce expectation failure when technical, organizational, or user perspectives dominate narrowly.
  • Resources: Adequate money, people, and development time are central resources, whereas insufficient allocation, cost overruns, and schedule changes can damage outcomes.Resources are particularly critical for expensive, time-consuming, organization-wide systems.
  • Technology: Technology has inconsistent empirical support as a direct success factor, but appropriate infrastructure, expertise, architecture, and tools can support performance and complexity management.Standard software infrastructure is suggested as beneficial where infrastructure constitutes a high proportion of application code.

6. INFLUENTIAL FACTORS – SYSTEMS DEVELOPMENT PROCESSES

Development-process influences include requirements determination, planning, project management, methods, user participation, training, and change management. Empirical studies generally support clear requirements, planning, competent coordination, and participation, while emphasizing contextual limits and process risks.

  • Requirements Determination: Well-defined requirements support project success, while misunderstood, unclear, or changing requirements are associated with risk, challenged projects, and abandonment.Iterative design, prototyping, and agile methods are suggested responses to changing requirements.
  • Requirements Determination: Requirements determination depends on allocating sufficient time and involving suitable users, while conventional approaches may overlook emergent, social, and political dimensions.The latter limitation concerns approaches that treat requirements as fixed objective artifacts specified at the outset.
  • Project Management: Planning defines projects, estimates size, cost, and schedules, assesses risks, and develops plans; it reduced budget variances and team instability but not completion rates in one survey.Inadequate planning, poor estimates, and weak risk management are associated with detrimental outcomes.
  • Project Management: Formal project-management methods can facilitate management and target-date completion, especially when projects are larger or more technically complex.Project managers also mediate between groups by translating requirements, building stakeholder consensus, and buffering project teams from external influences.
  • Use of a Standard Method: Standard methods may help when properly used by experienced developers, but their association with success is inconsistent and their value depends on context.Methods can hinder development in some small organizations or small projects with small teams.
  • User Participation: User participation has been positively associated with success, satisfaction, acceptance, completion, performance, and system impact, with earlier participation often having greater acceptance effects.High participation does not guarantee success: one long-running case retained low usefulness and usage perceptions and was eventually terminated.
  • User Training: User training is identified as a factor that can influence project outcomes and has been found important for system success, although it may be time-consuming in large projects.

7. INFLUENTIAL FACTORS – INSTITUTIONAL CONTEXT

Institutional context encompasses organizational properties and wider environmental conditions that can enable or constrain software systems development. The reviewed studies show that these influences operate through culture, established practices, organizational history, and changing external conditions.

  • Institutional context: Institutional context includes organizational structures, practices, relations, and wider socio-economic conditions surrounding software systems development.Organizational properties include norms, resources, procedures, structures, coordination mechanisms, and reward structures.
  • Organizational properties: Organizational structures and context-specific features can enable or constrain available resources, communication, cooperation, methods, and tools.Authority structures can restrict system-related expenditure, while culture may affect communication between functional units.
  • Organizational culture: Organizational culture shapes interactions between users and developers, inter-departmental cooperation, and intended system use.A consensus-oriented culture may encourage communication and conflict resolution, whereas divergent organizational interpretations can undermine system acceptance.
  • Organizational culture: A university enterprise-system project became an average system that satisfied nobody, prompting workarounds, continued use of existing systems, and alternative development.The project shifted from centralized standardization toward a user-oriented approach to accommodate different organizational units.
  • Organizational policy and practice: Established policies and practices can become institutionalized, constrain innovations, and also change over time through organizational approaches to participation and change management.Routine practices can become part of organizational culture, while participative policies influenced how user participation and development-related change were enacted.
  • Environmental conditions: External conditions such as ownership changes, overseas acquisition, and rapid organizational or economic growth may exert more influence on outcomes than project-level factors.Changes in the external environment can also make a proposed system lose its former relevance.

8. DISCUSSION

The discussion identifies persistent foundational factors, changing development conditions, the importance of people and process, institutional context, and interactions among influences. It also argues that generic prescriptions are insufficient because factors vary dynamically and are difficult to define consistently.

  • Discussion: Five themes emerge: persistent traditional factors, changing development, people and process, institutional context, and interrelationships among influences.The themes synthesize the reviewed empirical literature on software systems project outcomes.
  • Persistent factors: Traditional positive influences remain important but are necessary rather than sufficient, including skilled developers, committed users, supportive management, effective teams, clear goals, resources, technology, requirements, methods, participation, and training.These factors continue to be perceived and empirically demonstrated as important across contemporary systems development.
  • Persistent factors: Project failures persist despite widespread knowledge of commonly cited success factors, challenging explanations based solely on failure to follow best practices.The discussion frames this tension through Cobb’s Paradox and Sauer’s analysis of recurring failure-associated practices.
  • Limits of prescriptions: Prescriptive factor lists often lack specificity, are difficult to act upon, may be inhibited by organizational conditions, and can exacerbate other project problems.Examples include undefined adequacy of resources, unclear meanings of active user participation, and the practical difficulty of obtaining management support.
  • Interrelationships and interaction: Factors are not independent or equally important: their relative influence changes over time, requiring attention to dynamic interactions during projects.The timing, direction, and strength of influence may vary across the project trajectory.

2 We are grateful to an anonymous reviewer for bringing Cobb’s

The review describes a changing development landscape marked by new technologies, organizational arrangements, and project forms, alongside growing recognition of social and organizational influences. It concludes that project outcomes emerge from interacting factors within complex change processes.

  • Changing development: Recent software development changes reflect rapid technological advances, complex global business demands, and evolving development practices.These changes are often inter-related rather than isolated.
  • Changing development: Modern projects increasingly involve business-unit funding, packaged software customization, outsourcing, reduced in-house development, and shorter or incremental delivery.Smaller projects or staged delivery may increase the chances of successful outcomes, while enterprise-wide and distributed systems can increase complexity.
  • Stakeholders and complexity: Outsourcing and enterprise-wide systems expand the stakeholder group to include vendors, contractors, and external consultants.These participants can increase the need for coordination and mediation between project parties.
  • People and process: Organizational, political, and human-related issues are increasingly recognized as more important than technical issues in determining many system outcomes.The literature reports that failures are rarely caused by technical problems alone.
  • People and process: Underperformance can persist after technical problems are addressed when task-technology fit remains incongruent with users’ work.This illustrates why redesign alone may not resolve organizational or human-related causes of non-use.
  • Interrelationships and interaction: Empirical studies report complex, dynamic interactions among factors, but inconsistent terminology and outcome measures make findings difficult to synthesize.The review calls for common definitions, measurement instruments, and scales, while identifying deeper interaction analysis as future work.
  • Interrelationships and interaction: The paper characterizes systems development as a multi-dimensional change process embedded in a complex web of social conditions and factors.This framing supports studying project outcomes through relationships among technical, organizational, human, and contextual influences.

9. CONCLUSION

The survey synthesizes empirical research into a four-dimensional framework for understanding software systems project outcomes. It supports holistic analysis and potential risk-management applications by connecting people, project content, processes, and institutional context.

  • The empirically grounded framework groups influential factors into people and action, project content, development processes, and institutional context.These dimensions represent content, process, action, and context in software systems development and deployment.
  • The framework makes the literature more manageable by abstracting eighteen individual factors into four dimensions while retaining finer-grained analysis when needed.Researchers can use the dimensions for analytical abstraction and return to individual factors when greater detail is required.
  • Considering interactions among dimensions avoids the myopic analysis produced by focusing exclusively on a single factor or dimension.Project content is connected with development context, processes, and the people who perform them.
  • The framework can support holistic, multi-level analysis by organizing factors across multiple dimensions rather than treating them in isolation.The authors characterize narrow focus on individual dimensions and factors as inadequate in prior work.
  • The framework could inform inclusive project risk management by identifying risk areas, grouping them thematically, and incorporating organizational history for learning from past projects.The four dimensions can structure initial risk identification and ongoing risk management as projects proceed.
  • Future empirical research should address continuing changes in software development and acquisition while attending to persistent aspects and recurring perceived project failures.The authors expect further changes and argue that research must account for both change and continuity.
Loading 2101.08442v1…