Source-linked AI summary
The fallacy of evidence based policy
Andrea Saltelli, Mario Giampietro
TL;DR
The paper examines how science-for-policy arrangements struggle to separate evidence gathering from political imperatives when facts and values are intertwined. Drawing on science and technology studies and bioeconomics, it argues for robust policy assessed through feasibility, viability, and desirability rather than relying on simplified predictive narratives.
Problem
Science-for-policy cannot be cleanly separated from political processes because facts and values are intertwined in hybrid arrangements.
Method
The paper draws primarily on science and technology studies and bioeconomics, with NUSAP for managing and communicating uncertainty in quantitative policy information.
Results
The paper proposes evaluating robust policy through feasibility, viability, and desirability rather than relying on isolated quantitative representations and simplified narratives.
Takeaways & Limitations
Science should contribute to reassessing feasibility, viability, and desirability across normative considerations relevant to different actors.
Takeaways & Limitations
Quantitative information requires qualitative coherence checks because combining non-equivalent models and arbitrarily constrained explanations can increase indeterminacy and uncertainty.
Abstract
from arXiv · showhide
The use of science for policy is at the core of a perfect storm generated by the insurgence of several concurrent crises: of science, of trust, of sustainability. The modern positivistic model of science for policy, known as evidence based policy, is based on dramatic simplifications and compressions of available perceptions of the state of affairs and possible explanations (hypocognition). This model can result in flawed prescriptions. The flaws become more evident when dealing with complex issues characterized by concomitant uncertainties in the normative, descriptive and ethical domains. In this situation evidence-based policy may concur to the fragility of the social system. Science plays an important role in reducing the feeling of vulnerability of humans by projecting a promise of protection against uncertainties. In many applications quantitative science is used to remove uncertainty by transforming it into probability, so that mathematical modelling can play the ritual role of haruspices. This epistemic governance arrangement is today in crisis. The primacy of science to adjudicate political issues must pass through an assessment of the level of maturity and effectiveness of the various disciplines deployed. The solution implies abandoning dreams of prediction, control and optimization obtained by relying on a limited set of simplified narratives to define the problem and moving instead to an open exploration of a broader set of plausible and relevant stories. Evidence based policy has to be replaced by robust policy, where robustness is tested with respect to feasibility (compatibility with processes outside human control); viability (compatibility with processes under human control, in relation to both the economic and technical dimensions), and desirability domain (compatibility with a plurality of normative considerations relevant to a plurality of actors).
Section 1 – Science for policy. Predicaments and doubts
The paper argues that science-for-policy faces intertwined crises of science, trust, and sustainability that cannot be addressed merely by separating evidence from politics. It draws on STS, bioeconomics, and Post Normal Science to reassess how knowledge should inform governance amid contested values and uncertainties.
- Evidence-based policy can become policy-based evidence, but the paper argues that the underlying problem is deeper than inadequate separation between science and policy.The deeper problem involves concurrent crises of science, trust, and sustainability.
- The analysis uses science and technology studies and bioeconomics to examine science-for-policy and quantitative analysis.Post Normal Science is used for situations where facts are uncertain, values are disputed, stakes are high, and decisions are urgent.
- Post Normal Science treats policy-relevant inquiry as epistemic governance in which knowledge for policymaking must be understood and governed.Its context is characterized by uncertain facts, disputed values, high stakes, and urgent decisions.
- Policy controversies such as GMO food and climate change involve cultural and normative commitments that scientific proficiency cannot silence.The paper argues that different parties may disagree about the nature of the problem itself.
1.2 Science’s own crisis and its roots
The paper presents science as facing a crisis of reliability, quality control, legitimacy, and trust. It contrasts the traditional separation of science from politics with approaches that acknowledge entanglement among facts, values, experts, and stakeholders.
- Scientific credibility is weakened by declining reproducibility, retractions, suspected unreliable findings, systematic bias, and concerns about research incentives.The paper presents these concerns across natural, medical, behavioral, and economic sciences.
- The crisis of science extends beyond statistical training because peer review and the broader quality-control system are also described as compromised.The paper links industrialized science and weakened quality control to serious consequences for science’s morale, recruitment, and survival.
- Knowledge practices are presented as direct concerns of the polity because solutions to knowledge problems are also connected to solutions to problems of social order.The paper connects the legitimacy of science with the legitimacy of governance.
- The demarcation model protects science from political interference by separating institutions that provide science from those that use it.The paper presents this as the epistemology underlying calls for a stricter science-policy separation.
- More recent epistemologies challenge clean separation between facts and values, emphasizing hybrid arrangements and the insistence on integrity within them.The paper identifies Post Normal Science and co-production of knowledge as alternatives.
- Extended participation broadens deliberation across disciplines and across communities of experts and stakeholders, moving toward working deliberatively within imperfections.This model foregrounds participation, legitimacy, transparency, and accountability.
1.3 Trust, modelling, uncertainties
The paper argues that evidence and mathematical models can support policies whose consequences persist after the underlying findings are rejected. It identifies modelling practices that compress uncertainty and complexity while science is also used selectively in political disputes.
- Growing mistrust toward institutions is connected to science for policy because rules result from policies that are defended on the basis of evidence.The paper uses public-budget austerity as a relevant example of this connection.
- Policies defended by evidence can remain in place after the evidence is repudiated, as illustrated by the retracted 90% public-debt-to-GDP threshold.A later reanalysis traced the finding to a coding error, while the paper reports substantial damage in Britain and Europe.
- Mathematical modelling can support flawed policies through hubris and through the rhetorical or ritual use of disproportionate models.The paper also identifies tacit assumptions, expedient treatment of uncertainty, linearization, and weak sensitivity analysis.
- Science can become ammunition in partisan disputes when opposing sides selectively mobilize scientific claims to legitimate their interests.In such controversies, experts on different sides may cancel one another out while stronger political or economic interests prevail.
2.1 Science to tame human vulnerability: the Cartesian Dream
The paper traces science’s policy role to a modern promise that knowledge can reduce vulnerability through prediction, control, and rational ordering. It argues that this Cartesian Dream remains influential despite its collision with complex contemporary crises.
- Humans use anticipation and organized communities to reduce vulnerability while remaining dependent on processes outside their control.Social contracts exchange part of individual autonomy for protection against needs including security, health, and environmental threats.
- Modernity replaced religious authority with science as a source of social problem-solving, drawing on Bacon, Descartes, Condorcet, and Bush.These visions connected scientific knowledge with power, welfare, justice, happiness, employment, and economic growth.
- Historical scientific utopias extended from Baconian hopes for transforming human conditions to Condorcet’s belief that physical errors underlie political and moral errors.Bush later framed basic scientific research as scientific capital supporting new products and employment.
- The Cartesian dream remains a prevailing narrative despite extensive criticism from social science and science-and-technology-studies scholarship.The paper presents this persistence as a reason to revisit science’s role in policy.
- The Cartesian Dream sought ecosystems and social systems that could be fitted into precise, manageable categories through prediction and control.The paper states that this agenda now confronts the complexities of the present crisis.
2.2 Reductionism, hypocognition and socially constructed ignorance
Evidence-based policy compresses complex problems into simplified frames, producing hypocognition and socially constructed ignorance. This can drive optimization within the wrong problem space, reduce adaptability, and sustain irrelevant models despite known limitations.
- The evidence-based policy model relies on dramatic simplification and linearization of the problem space.
- Compression excludes known gaps, uncomfortable knowledge, and established scientific knowledge that does not fit the chosen problem structuring.These exclusions are described as socially constructed ignorance, including “known unknowns” and “unknown knowns.”
- Focusing analysts on finite attributes and goals can produce optimization in the wrong problem space.
- Optimization reduces behavioral diversity and adaptability by eliminating alternatives and neglecting attributes and goals outside the chosen frame.
- Using reductionist methods for complex issues can make rationality dysfunctional when relevant processes require multiple narratives and scales.Bioeconomics calls for integrating non-equivalent narratives and different descriptive domains; the corn-bioethanol example illustrates the consequences of narrow framing.
- Irrelevant models cannot necessarily be corrected through learning by doing and may therefore cause damage for longer periods.
- Scientific predictions are conditional and cannot establish that the conditions required for their conclusions will actually be fulfilled.
- Evidence-based policy can persist despite recognition that displacement is incorrect, associating wellbeing with stabilization of existing institutional arrangements.
2.3 Legitimacy versus simplification
Deterministic, reductionist science is poorly suited to complex socio-ecological systems whose behavior involves accumulated uncertainties, agency, and multiple interacting scales. Quantification can nevertheless lend policy decisions an appearance of impartial legitimacy, motivating a shift toward deliberate doubt and scrutiny of framing.
- Deterministic models assume complex self-organizing systems can be predicted and that methodological rigor ensures policy-input quality.This assumption overlooks the accumulation of uncertainties.
- Financial models that failed to predict economic crises are nonetheless expected to inform systems involving institutions, societies, economies, and ecologies.
- Climate issues connect energy, water, food, and human institutions across scales, making complexity an inherent property rather than a problem to solve.
- Quantification gives bureaucratic decisions an appearance of fairness and impartiality when officials lack electoral or divine authority.
- Evidence-based policy assumes prediction and control can eliminate moral hesitation, whereas the proposed alternative deliberately reintroduces doubt and scruples.
- Improving science for governance requires examining how issue frames generate predefined data, indicators, and mathematical models.
3.1 Responsible use of quantitative information
Responsible quantitative information requires controlling the effects of hypocognition rather than relying on spurious precision. The paper highlights NUSAP and sensitivity auditing as practical tools for scrutinizing uncertainty, assumptions, framing, and policy inference.
- Responsible policy analysis should avoid indicators with spurious accuracy and fantastic model-generated numbers.
- NUSAP manages and communicates uncertainty by characterizing quantitative statements through Numeral, Unit, Spread, Assessment, and Pedigree.
- Sensitivity auditing extends model sensitivity analysis to policy inference by questioning framing, assumptions, uncertainty assessment, transparency, and legitimacy.
3.2 Taming scientific hubris
The paper argues that effective science for governance requires taming scientific hubris and distinguishing calculable risks from uncertainties, ignorance, and indeterminacy. Scientific knowledge remains conditional because analyses freeze contexts that may not persist in real-world systems.
- The paper distinguishes risks, whose odds can be computed, from uncertainties, whose odds are not known.
- The proposed taxonomy also includes ignorance, where unknowns remain unknown, and indeterminacy, where causal chains or networks remain open.
- Scientific risk analysis freezes surrounding conditions, making its resulting knowledge conditional on whether those assumptions remain valid.
- Taming scientific hubris is presented as fundamental to using science more effectively for governance.
3.3 Evidence based policy versus robust policy
The paper proposes replacing evidence-based policy’s simplified, prediction-oriented approach with robust policy that filters options through falsification across multiple domains and stakeholder perspectives.
- Robust policy: Robust policy filters potential policies through falsification rather than relying on evidence-based policy’s hyper-quantified simplified narratives.The approach draws on socially robust knowledge filtered through different stakeholders and normative stances.
- Quality checks: Policies are quality-checked for feasibility, viability, and desirability before informed deliberation.These domains concern external constraints, internal constraints, and normative values, respectively.
- Quality checks: An infeasible, unviable, or undesirable policy indicates a bottleneck, political issue, or true impossibility requiring attention.The checks replace planning based on prediction and control with strategic learning through falsification and flexible management.
- Pluralism: Robust policy shares with clumsy solutions a pluralistic, satisficing orientation emerging from negotiation among actors with differing reasons or principles.The paper connects this orientation to Post Normal Science’s extended participation model.
- Analytical perspective: Identifying the three domains requires examining states of affairs through different analytical dimensions and scales.The paper highlights this difficulty in quantitative sustainability analysis, including food-security requirements and supply.
- Robust policy: The approach explores a multidimensional policy space with parsimonious experimental design instead of concentrating detail around one point.It aims to balance efficiency with adaptability in view of sustainability and support multi-criteria evaluation.
3.4 Quantitative story telling for governance
Quantitative storytelling widens the frames used to structure policy problems, generating plausible and relevant narratives while requiring coherence checks across models, scales, and dimensions.
- Purpose: Quantitative storytelling expands available frames to reduce hypocognition in problem definition and structuring.Its goal is to generate plausible and relevant stories rather than retain one simplified narrative.
- Purpose: Story quality depends on reducing neglected relevant narratives and accounting for unavoidable unknown unknowns.The paper describes these omissions as consequences of hypocognition associated with chosen problem structuring.
- Assessment: Policy fitness is gauged by integrating relevant narratives, plausible explanations, and pertinent perceptions.The approach treats policy assessment as a synthesis across multiple forms of representation.
- Coherence checks: More data and larger models can increase indeterminacy when explanations and perceptions are arbitrarily constrained.A qualitative coherence check is therefore essential when quantitative information comes from non-equivalent models.
- Coherence checks: Chosen stories must be validated with quantitative analysis coherent across scales and dimensions, including socio-ecological openness to trade.Otherwise relevant aspects of the problem may be externalized.
3.5 Getting the right narratives before crunching numbers
Before indicators, data, and models are built, the paper argues that analysts should examine multiple narratives, illustrated by how GMO debates are framed around concerns beyond food safety.
- Narrative selection: The paper recommends selecting and exploring multiple stories before building indicators, collecting data, and running models.This means doing the right sums rather than merely doing sums correctly within one frame.
- GMO example: GMO controversy illustrates a wicked issue requiring more than a nutritional risk-to-health frame.Opposition is commonly portrayed as anti-science, while citizens’ concerns involve broader institutional and political questions.
- GMO example: Participatory evidence shows citizens asking why GMOs are needed, who benefits, who decided, and whether consumers receive meaningful choice.These concerns differ from the prevailing safe-food-versus-recalcitrant-citizens framing.
- GMO example: Citizens also question whether regulators can counterbalance large companies developing genetically modified products.This extends the frame to institutional power and regulatory capacity.
- GMO example: The variety of citizen frames makes the safe-GMO-food-versus-recalcitrant-citizens frame irrelevant to the decision.The example supports opening the problem frame before quantitative analysis.
Conclusions
The conclusion challenges evidence-based policy’s use of simplified quantitative frames and prediction-oriented science for complex, contested systems. It supports robust policy based on plural narratives, continuously revisable feasibility, viability, and desirability conditions.
- Conclusions: Evidence-based policy should be revised because accumulating data, indicators, and models around a frozen frame can reinforce hypocognition.The paper argues that semantic opening of the frame is more important than disproportionate mathematical precision.
- Conclusions: Complex adaptive systems cannot be deterministically predicted because they continuously change while reproducing themselves.Their trajectories must remain feasible, viable, and desirable simultaneously.
- Conclusions: Feasibility, viability, and desirability must be continuously updated as external conditions, internal processes, and normative values change.These changes impair long-term deterministic predictions.
- Conclusions: Normative frames emerge from negotiation and shifting power relations, so previously useful narratives can become misleading or dangerous.The paper uses the changing meaning of the ‘Endless Frontier’ metaphor as an example.
- Conclusions: Science’s established role as a driver of techno-scientific progress remains tied to institutional strategies of prediction and control.The paper criticizes plans informed by models and cost-benefit analyses when applied to complex sustainability problems.
- Conclusions: Mathematical modelling can become ritualistic when forecasts are retained for planning despite verification showing they perform no better than chance.The example is used to illustrate science employed for its formal role rather than reliable guidance.
- Conclusions: Western progress often externalized negative consequences to the environment, future generations, or other countries.This complicates attempts to transfer the same development achievements elsewhere.