PaperlayerView exact revision on arXiv

Source-linked AI summary

Global Index on Responsible AI 2026 : Conceptual Framework and Methodology

Fola Adeleke, Rachel Adams, Ayantola Alayande, Daniela Benavente, Ana Florido, Nicolás Grossman, Leah Junck

arXiv:2608.18122v1cs.CYcs.AI

TL;DR

Responsible AI governance lacked globally comparable evidence on how countries establish and implement governance measures. GIRAI’s second edition refines its framework and measurement approach, producing a statistically coherent and robust assessment while noting that alternative assumptions may vary scores and rankings.

  • Problem

    GIRAI addresses limited systematic, globally comparable evidence on how countries respond to AI opportunities and risks through responsible AI governance.

  • Method

    GIRAI combines primary survey data on AI policy, civil society engagement, and unacceptable-risk AI deployment with secondary data on enabling conditions.

  • Results

    The second edition strengthens conceptual clarity, implementation assessment, and measurement of differences in governance responses, with an overall framework found statistically coherent and robust.

  • Takeaways & Limitations

    GIRAI provides a structured, globally comparable assessment of governance frameworks, implementation measures, civil society engagement, and enabling conditions.

  • Takeaways & Limitations

    GIRAI does not directly assess governance-measure effectiveness or the real-world impacts of AI systems on human rights, democratic governance, or social outcomes.

Abstract

from arXiv · show

This report presents the methodology of the Global Index on Responsible AI (GIRAI), 2nd Edition. This edition refines the 1st Edition by strengthening the distinction between framework existence and implementation, restructuring dimensions from three to five thematic areas, introducing more granular variables for framework quality, and applying a multi-stage review and validation process. An independent statistical pre-audit was conducted to assess the coherence and robustness of the framework. GIRAI assesses responsible AI governance across five dimensions: Inclusion and Diversity, Ethics and Sustainability, Labour and Skills, Trust and Safety, and Use of AI in Public Service. Each dimension has a number of indicators (38 in total), organised into three pillars, namely AI Policy (17 indicators on government frameworks and implementation, assessed through primary data), CSO Engagement (5 indicators, primary data), and Enabling Conditions (15 indicators on the structural factors shaping responsible AI governance, assessed through secondary data), and a government Use of Unacceptable Risk AI (URAI) indicator (primary data), applied separately as an accountability penalty to the final score. Data was collected by 135 country-level researchers through a structured global survey, complemented by secondary datasets. The count, scope, enforceability, thematic coverage, and implementation levels of the data points are coded into numerical variables, normalised to a scale of 100, aggregated through pillar weights of 60% (AI policy), 10% (CSO Engagement), and 30% (Enabling conditions). A deduction penalty is applied for countries with evidence of URAI. This documentation enables systematic cross-national comparison, supporting policymakers, civil society, and AI developers to identify where commitments are translating into enforceable protections and where critical gaps remain.

1. INTRODUCTION

The Global Index on Responsible AI (GIRAI) was established to address the lack of globally comparable evidence on responsible AI governance from a human rights-based perspective. Its 2026 second edition refines measurement of frameworks, implementation, risks, and comparability while documenting policy commitments and implementation evidence.

  • Rationale: GIRAI addresses limited systematic, globally comparable evidence on countries’ responses to AI opportunities and risks, particularly from a human rights perspective.Existing initiatives assessed national AI readiness and institutional capacity, but did not fully address this measurement gap.
  • Index contribution: GIRAI focuses on responsible AI governance, assessing governance frameworks, implementation measures, civil society engagement mechanisms, and enabling conditions through primary and secondary data.This complements initiatives focused on AI capacity, innovation performance, readiness, or ecosystem development.
  • 2026 edition: The 2026 edition is GIRAI’s second edition and refines the framework to improve conceptual clarity, strengthen implementation assessment, consider emergent risks, and enhance data consistency and comparability.These refinements build on lessons learned from the first edition.
  • Measurement scope: GIRAI combines evidence on AI policy frameworks and initiatives, civil society engagement, and broader enabling conditions to assess responsible AI governance across countries and regions.The resulting comparable evidence supports policymakers, researchers, civil society organisations, international institutions, and other stakeholders in identifying opportunities for improvement.
  • Purpose: GIRAI documents both policy commitments and implementation evidence to support informed decision-making, encourage accountability, and understand governance strengths, weaknesses, and development trajectories.Its purpose extends beyond ranking countries to providing a structured evidence base for responsible AI governance.

2. Conceptual Framework

GIRAI is a human-rights-based framework for assessing responsible AI governance through five dimensions, three pillars, and 38 indicators. It combines primary and secondary data to examine countries’ commitments, practices, and enabling conditions.

  • Normative alignment: GIRAI operationalises values, principles, and policy areas from the UNESCO Recommendation and related international frameworks to assess country-level implementation.The initiative seeks to measure whether countries are establishing governance conditions necessary for responsible AI.
  • Conceptual basis: Responsible AI treats AI as socio-technical systems that should respect human rights, protect people from harm, promote inclusion, and serve the public good.The framework recognises that AI is shaped by social, economic, political, and cultural contexts.
  • Dimensions: GIRAI organises assessment across five dimensions: Inclusion and Diversity, Ethics and Sustainability, Labour and Skills, Trust and Safety, and AI Use in Public Service.These dimensions address broad areas in which countries build responsible AI frameworks, institutions, practices, and enabling conditions.
  • Assessment structure: The 38 indicators and three pillars provide a multidimensional assessment of countries’ commitments and practices related to Responsible AI.The first two pillars rely entirely on primary data, while Enabling Conditions uses secondary data.
  • Pillars: The framework uses three complementary pillars: AI Policy, CSO Engagement, and Enabling Conditions.AI Policy examines government frameworks, actions, and implementation; CSO Engagement covers participation, accountability, advocacy, and affected-community representation; Enabling Conditions captures broader country-level factors.

3. Changes in the Second Edition

The second edition refines GIRAI’s conceptual structure and methodology by reorganising dimensions, pillars, and indicators while strengthening assessment of governance-framework quality and implementation. Because these changes limit direct score comparability, cross-edition analysis focuses on equivalent indicators and evidence types.

  • Revised framework structure: The second edition replaces three dimensions and nineteen thematic areas with five dimensions, three pillars, and 38 indicators.Its revised matrix assigns each indicator to a specific pillar, with aggregation beginning at the pillar/dimension level.
  • Revised framework structure: AI Policy merges Government Frameworks and Government Actions, while Non-State Actors is narrowed to CSO Engagement.Enabling Conditions is embedded directly as the third pillar rather than used as coefficients adjusting pillar scores.
  • Framework quality and implementation: The methodology moves beyond framework existence by assessing reach, sectoral scope, exemptions, thematic coverage, consultation, and implementation provisions.Operationalisation includes implementation bodies or mechanisms, plans, budget estimates, and monitoring and evaluation arrangements.
  • Framework quality and implementation: Implementation assessment is integrated more closely into AI Policy, with up to three government-led initiatives assessed per indicator instead of one.This expands the range of implementation evidence captured by the index.
  • Cross-edition comparability: Direct comparison of overall GIRAI scores across editions is limited because differences may reflect both country performance and measurement-framework changes.Comparisons instead use equivalent evidence, including framework existence, type, enforceability, and initiative existence.

4. Data Collection and Sources

GIRAI combines primary Global Survey data with secondary indicators to assess responsible AI governance across 135 countries and jurisdictions. In-country research, standardised procedures, and layered review mechanisms support locally grounded, cross-country comparable assessments.

  • Data sources: Primary data support the AI Policy and CSO Engagement pillars and assess government deployment of Unacceptable Risk AI Systems, while secondary data construct Enabling Conditions.The methodology therefore combines survey-based evidence with secondary indicators for the enabling environment.
  • Coverage: 135 countries and jurisdictions were covered, with primary data collected from 1 November 2023 to 30 September 2025.Country inclusion depended on suitable country researchers and institutional-partner networks, not government involvement.
  • Research network: More than 160 in-country researchers, reviewers, coordinators, supervisors, and institutional partners contributed to the 2026 edition.Researchers used original-language sources and local expertise, while standardised coding guidance and validation procedures supported consistency.
  • Quality assurance: A multi-layer quality assurance process archived evidence, reviewed submissions at country or regional and global levels, and recorded revisions and approvals through an audit trail.The Knowledge Forum provided centralised guidance on methodological questions, while questionnaires could be returned for revision multiple times before approval.
  • Secondary indicators: Secondary indicators were selected for relevance, distinctiveness, global coverage, comparability, source credibility, and data availability, with preference for transparent international datasets.They achieved an average country coverage of over 96 per cent; predefined imputation rules addressed unavailable country data while maintaining comparability.

5. Global Survey Methodology

The Global Survey is GIRAI’s primary, evidence-based instrument for collecting AI Policy and CSO Engagement data and assessing government deployment of Unacceptable Risk AI Systems. It uses questionnaires covering policy frameworks, government initiatives, civil society engagement, government mechanisms for CSO inclusion, and URAI deployment.

  • Survey scope: The survey constructs twenty-three indicators across the AI Policy and CSO Engagement pillars and assesses government deployment of Unacceptable Risk AI Systems.Researchers must submit publicly available, verifiable sources supporting all coding decisions.
  • Survey instruments: Questionnaires cover AI policy frameworks, government-led initiatives, Civil Society Engagement initiatives, Government Mechanisms for CSO Inclusion, and government deployment of URAI systems.The survey is organised around separate questionnaires for these areas.
  • AI policy frameworks: Up to two frameworks may be assessed per indicator, covering national or federal laws, regulations, policies, strategies, and guidelines, with certain internationally enforceable frameworks accepted as equivalents.The European Union AI Act is given as an example of an internationally enforceable equivalent.
  • Government-led initiatives: Up to three government-led initiatives may be assessed per framework, evidencing implementation of existing frameworks or direct government action where no framework exists.The maximum is three initiatives per framework, with up to two frameworks per indicator.
  • Civil society engagement: CSO Engagement is assessed through one questionnaire per dimension, capturing up to one initiative per initiative type across six types, while GMC focuses on government-facilitated participation.For AI Use in Public Service, GMC assesses mechanisms enabling CSO consultation, participation, and oversight.
  • Unacceptable Risk AI: URAI assessment examines publicly documented or credibly reported government uses posing unacceptable risks to human rights, democratic governance, or the rule of law.The categorisation draws on the EU AI Act and evidence availability may be constrained where civil society capacity is limited.

6. Computing the GIRAI Score

GIRAI computes scores from 38 indicators across five equally weighted dimensions, combining normalised pillar scores and applying a separate penalty for documented unacceptable-risk AI deployments. The method codes qualitative evidence, standardises indicators to 0–100, weights pillars, averages dimensions, and reduces scores for government systems posing unacceptable risks.

  • Indicator structure: 38 indicators underpin GIRAI: 37 span five equally weighted dimensions and three pillars, while the URAI indicator is applied separately as a final-score penalty.The pillars are AI Policy, CSO Engagement, and Enabling Conditions.
  • Scoring and normalisation: Qualitative and secondary-source evidence is coded so higher values indicate stronger governance, with binding and broader-coverage frameworks receiving higher scores.Enabling Conditions data are checked for missing values, outliers, and directionality.
  • Scoring and normalisation: All indicators are normalised to a common 0–100 scale before aggregation.
  • Aggregation: Pillar scores are combined within each dimension using weights of 60% for AI Policy, 10% for CSO Engagement, and 30% for Enabling Conditions.Normalised indicator scores are first averaged into pillar scores.
  • Aggregation: The five dimension scores are averaged equally to produce the raw GIRAI score.
  • Accountability penalty: 4% for one documented URAI case and up to 10% for four or more cases are deducted from countries with government deployments posing unacceptable risks.The reduction is proportional and prevents stronger performance elsewhere from offsetting governance failures.

7. Scope and Limitations

GIRAI provides a structured, globally comparable assessment of responsible AI governance, but its scores measure governance arrangements rather than their effectiveness or real-world impacts. Results depend on available evidence and methodological choices, despite independent robustness assessment.

  • GIRAI provides a structured, globally comparable assessment of governance frameworks, implementation measures, civil society engagement, and enabling conditions for responsible AI.
  • Composite scores and rankings simplify complex institutional, legal, and social realities and should be interpreted alongside underlying evidence and qualitative findings.
  • GIRAI measures governance measures and their characteristics, but not their effectiveness or AI’s real-world impacts on human rights, democratic governance, or social outcomes.
  • Differences in transparency, information access, publication requirements, and documentation availability may affect the evidence identified and validated across countries.GIRAI relies on publicly available, verifiable evidence collected through structured research and quality assurance procedures.
  • Independent statistical pre-audit and robustness assessment found the framework statistically coherent and robust, although alternative methodological assumptions may vary country scores and rankings.Methodological choices include indicator selection, scoring, weighting, normalisation, and aggregation.

Technical Annexures … A.3.1 Composite indicators included in the Enabling Conditions pillar

The GIRAI converts primary survey and secondary evidence into normalised indicator, pillar, dimension, and final scores, with URAI assessed separately as a final penalty. Its Enabling Conditions pillar includes 15 secondary-source indicators, including two composite measures built from related variables.

  • A.1 Overview: The scoring pipeline computes indicators, normalises them to 0–100, aggregates pillar and dimension scores, and applies the URAI penalty to obtain the final GIRAI score.Primary data supports AI Policy, CSO Engagement, and URAI assessment, while secondary data constructs Enabling Conditions.
  • A.2 Computation of Indicators from Primary Data: 23 of 38 indicators use Global Survey data: 17 AI Policy indicators, 5 CSO Engagement indicators, and 1 URAI indicator.The remaining 15 indicators are derived from secondary sources for the Enabling Conditions pillar.
  • A.2.1 AI Policy Pillar: Each AI Policy indicator equals Frameworks plus Initiatives, with maximum component scores of 16 and 6, respectively.Framework scores reflect document type, scope, consultation, operationalisation, and thematic coverage, while initiatives use twice the maximum framework-specific initiative count.
  • A.2.1.1 Frameworks score: Framework scores average two frameworks, halve a single partially reaching framework, assign zero when none exists, and apply a 0.9 multiplier for defence and security exemptions.The framework score is built from five variables, with operationalisation varying by document type.
  • A.2.1.2 Initiatives score: The Initiatives score is twice the maximum count across two framework-associated initiative sets, each containing up to three initiatives.Initiatives may also be recorded without a framework as government actions under init1.
  • A.2.3 Unacceptable Risk AI score (URAI penalty): The URAI indicator is computed separately as a threshold-based proportional penalty for documented government deployments of AI systems posing unacceptable risks.It concerns 35 of 135 countries, and urai_counts can reach seven, although no current country has more than four recorded deployments.
  • A.3.1 Composite indicators included in the Enabling Conditions pillar: 15 internationally comparable secondary-source indicators form the Enabling Conditions pillar, including composites for Data Sharing and Access and Access to Public Information.The Data Sharing and Access composite combines three DPI Map variables, while Access to Public Information averages two World Justice Project variables correlated at r = +0.813.

A.4 Imputation of missing data … B.4.1 Modelling choices

The methodology addresses missing data, normalization, aggregation, and robustness by combining structured imputation and scoring rules with an independent pre-audit of alternative modelling choices. GIRAI scores use weighted pillar aggregation and a separate URAI penalty, while robustness checks examine missingness, coherence, and ranking sensitivity.

  • A.4 Imputation of missing data: 96.4% average coverage across 15 indicators limited missingness to relatively few country and territory cases, concentrated especially in Palestine and Kosovo.Median coverage was 97.8%; Palestine had eight missing indicators and Kosovo seven.
  • A.4 Imputation of missing data: Missing values were preferentially imputed from earlier same-source reference years or peer-group means, falling back across UNSD geographic levels combined with World Bank income groups.Historical data were preferred when available; comparator groups were sense-checked when multiple plausible groups existed.
  • A.5 Treatment of outliers: None of the 15 indicators exceeded the outlier thresholds of absolute skewness greater than 2 and Pearson kurtosis greater than 3.5, so no outlier treatment was applied.GIRAI indicators combine primary Global Survey data with secondary internationally comparable datasets.
  • A.6.2 Aggregation of indicators at the country level: 60% AI Policy, 10% CSO Engagement, and 30% Enabling Conditions determine each dimension score through weighted pillar means, with equal indicator weights within each pillar and dimension.Country-level pillar scores may also be calculated as simple means across the five dimensions.
  • A.6.2 Aggregation of indicators at the country level: The final GIRAI score applies the URAI penalty after raw aggregation; one, two, three, and four-or-more systems yield 0.96, 0.93, 0.91, and 0.9 of the raw score, respectively.The separate adjustment preserves the interpretation of indicator, pillar, and dimension scores and prevents documented misuse from being fully offset elsewhere.
  • B. Statistical Pre-Audit and Robustness Assessment: The independent COIN-based pre-audit tested score and ranking stability under alternative assumptions, including imputation, normalization, weights, aggregation, and indicator influence.With zero-imputation, most countries had limited ranking changes, but Antigua and Barbuda and Palestine shifted by at least 10 positions, while Hong Kong SAR, Oman, and Ukraine shifted by at least four.

B.4.2 Uncertainty analysis results

The uncertainty analysis summarizes 47 ranking sets using median ranks and country-specific rank intervals. GIRAI ranks are especially robust at the extremes, although some countries have wider uncertainty intervals.

  • Ranking uncertainty: 47 ranking sets are summarized through a median rank and a rank interval for each country.The analysis uses imputed data and final rankings after applying the URAI penalty.
  • Ranking uncertainty: 32,6% of countries shift 5 positions or less, while 65,2% shift 10 positions or less across linear models.GIRAI ranks are particularly robust at the top and bottom, but a few countries show relatively large rank intervals.
  • Reporting uncertainty: Reporting uses 13 linear arithmetic models, excluding each country’s lowest and highest ranks to present a more interpretable uncertainty interval.Figure 2 displays the GIRAI ranking as a red line and country-level ranking intervals as error bars.

B.4.3 Sensitivity analysis results · B.4.4 Sensitivity to indicators · B.5 Concluding remarks

Sensitivity analysis finds that GIRAI rankings are least affected by arithmetic and linear approaches but become increasingly sensitive under alternative weighting, ordinal, and nonlinear methods, with geometric averages producing the largest departures from baseline ranks. Indicator-level results identify the strongest rank drivers, while the pre-audit confirms statistical coherence and highlights data-quality priorities for future editions.

  • B.4.3 Sensitivity analysis results: Ranking sensitivity rises from linear normalization and arithmetic scores through weighting schemes and ordinal techniques to nonlinear methods, with Copeland and geometric averages furthest from GIRAI ranks.The sensitivity sequence runs from standard linear and arithmetic approaches to GIRAI versus random or equal weights, then dataranks and Borda, and finally Copeland and geometric averages.
  • B.4.3 Sensitivity analysis results: Arithmetic averages produce low sensitivity because GIRAI uses modified min-max normalization and arithmetic aggregation, while median minmax and z-scores also center indicators at 50.Median minmax centers indicators at a median of 50 and z-scores at a mean of 50, though their distributional sensitivity differs.
  • B.4.3 Sensitivity analysis results: Weighting schemes show high sensitivity, with GIRAI, random, and equal weights ordered by impact because equal weights replace GIRAI’s 0.6 AI Policy and 0.1 CSO Engagement weights.Equal weights act as a stress test and substantially affect a handful of countries, particularly when combined with geometric averages.
  • B.4.3 Sensitivity analysis results: Geometric averages have maximum sensitivity because their multiplicative structure and treatment of zeros reward balanced profiles, unlike fully compensatory arithmetic averages.Low scores cannot be offset as readily under geometric aggregation, making this choice especially consequential for a multidimensional and recently developing policy field.
  • B.4.3 Sensitivity analysis results: Confidence intervals use 13 out of 15 linear models after trimming extreme results, providing transparent ranking uncertainty consistent with GIRAI’s methodological assumptions.The robustness table is titled “Robustness of GIRAI ranks, imputed data (13/15 linear models).”
  • B.4.4 Sensitivity to indicators: The largest leave-one-indicator rank effects are Low-carbon energy share at 34 positions, Cultural and Linguistic Diversity at 30, and Access to Redress and Remedy at 24.Because CSO Engagement has lower weight, omitting any of its five indicators has less impact, with GMC the main exception.
  • B.5 Concluding remarks: The COIN pre-audit confirms that GIRAI is reliable and statistically coherent, while recognizing the developers’ efforts to produce replicable and transparent results.The audit also finds GIRAI statistically balanced in its pillars with few exceptions, and most indicators meaningfully inform score variation.
  • B.5 Concluding remarks: Sensitivity to equal versus GIRAI pillar weights, index structure, and aggregation method reflects methodological choices rather than an index weakness and underscores the need for stronger CSO Engagement data.Improved data collection would support confidently increasing the CSO Engagement weight in future editions, while balanced profiles remain challenging in a nascent AI field.

C. Supporting Materials and Data Availability

The GIRAI Public Repository makes the research and scoring materials publicly available to support transparency, reproducibility, and future research. It includes documentation covering survey design, researcher guidance, data collection, indicator interpretation, EU framework application, and secondary data sources.

  • Repository purpose: The GIRAI Public Repository publicly provides the materials used in the index’s research and scoring.Its stated purposes are transparency, reproducibility, and future research.
  • Survey and researcher guidance: The Global Survey Methodology connects questionnaire design with the resulting dataset, while the Researchers Handbook explains the framework, index structure, and indicator-level evidence guidance.The handbook supports researchers in identifying, assessing, and documenting evidence for each indicator.
  • Data and indicator documentation: The repository also documents data-collection procedures, indicator interpretation, EU framework application, and metadata for secondary data sources.These materials cover collection tools, responsibilities, communication and quality review, indicator scope, and the application of EU frameworks within GIRAI.
Loading 2608.18122v1…