PaperlayerView exact revision on arXiv

Source-linked AI summary

Transparent Information Flow Modeling for Role-Aware AI Act Assessment

Mattias Brännström, Themis Dimitra Xanthopoulou, Lili Jiang

arXiv:2610.16916v1cs.AI

TL;DR

Existing assessments can treat AI systems as monolithic artifacts, without structuring how decisions relate to broader socio-technical processes. The paper applies IFM in a public-sector recruitment assessment, where it helped identify potential risks and biases and evaluate system integration against AI Act requirements.

  • Problem

    Technical assessments often evaluate model-level accuracy, bias, or fairness without structuring how decisions relate to broader socio-technical decision-making.

    Source: §1 · p. 1

  • Method

    The paper applies IFM to model decision processes and examine system integration, risks, and responsibilities in a public-sector recruitment assessment.

    Source: §1 · p. 2, §6 · p. 16

  • Results

    In the recruitment assessment, IFM helped identify potential risks and biases and enabled a fine-grained evaluation of whether system integration aligned with AI Act requirements.

    Source: §1 · p. 2

  • Takeaways & Limitations

    IFM provides a structural scaffold for analyzing information, decisions, and responsibilities across human and machine roles without losing the broader context of impacts.

    Source: §6 · p. 16

    IFM’s technical modeling layer may hinder adoption, and its use requires interpretation and familiarity with the social and technical domain being modeled.

    Source: §6.2 · p. 17

Abstract

from arXiv · show

The European Union's Artificial Intelligence Act introduces a structured framework for governing high-risk AI systems, emphasizing transparency, risk management, and role-specific accountability. However, current assessment practices often rely on monolithic system views or disconnected checklists, limiting their ability to capture the socio-technical complexity of real-world deployments. We explore the use of the Information Flow Methodology (IFM), a modeling-based approach to AI Act compliance assessment. IFM represents decision systems -- whether algorithmic, human, or hybrid -- as networks of information sites and transformation channels, enabling structured analysis of how information, decisions, and responsibilities flow across organizational and technical boundaries. We apply IFM in a real-world case involving a recruitment support tool used by a public-sector organization, evaluating both the Provider's system and the Deployer's recruitment process. Our results demonstrate that IFM can serve as a scaffold to bridge both component-based and holistic perspectives, and how technical systems are situated within socio-technical contexts. In doing so, IFM supports transparency, identifies risk pathways, and enables alignment analysis between intended and actual use. We argue that such modeling provides a foundational tool for effective, context-sensitive AI governance and accountability -- offering capabilities beyond conventional checklist-based assessments.

1 Introduction

The paper addresses the limits of monolithic AI assessments by applying IFM to examine information flows and responsibilities across technical and organizational decision-making. In a public-sector recruitment case, it reflects on IFM’s performance as an assessment tool.

1 Introduction

  • Existing technical assessments often evaluate model-level accuracy, bias, or fairness without structuring how decisions relate to broader socio-technical processes.This can lead to the framing trap: inferring system-level fairness or compliance from component-level properties.Source: §1 · p. 1
  • IFM models AI-supported decision systems as interconnected information sites and channels, tracing information from inputs to final decisions across technical and human processes.Embedding components in the overall decision process preserves a holistic view while tracing responsibility and downstream outcomes.Source: §1 · pp. 1–2
  • In a public-sector recruitment assessment, IFM helped identify potential risks and biases and evaluate whether system integration aligned with AI Act requirements.The assessment examined decision-making processes and responsibilities shared between system developers and the procuring organization.Source: §1 · p. 2
  • The paper critically reflects on how IFM performed in a real-world AI Act assessment, focusing on its advantages and limitations rather than the case results themselves.Source: §1 · p. 2
  • The paper argues that modeling socio-technical information flows can reveal how models and users interact with formal Provider and Deployer roles, unlike monolithic assessments.Source: §1 · p. 2

2 Background

The background describes how role-based regulation and common assessment approaches can miss how responsibility and impacts emerge in real deployments. It motivates methods that represent technical components and socio-technical decision structures together.

2 Background

  • The AI Act distributes obligations such as transparency, human oversight, quality management, and risk mitigation between Providers and Deployers.The distinction reflects real-world differences in access, expertise, control, and responsibilities.Source: §2 · p. 2
  • In practice, role boundaries can blur as systems are adapted, documentation may not reflect actual use, and oversight may misalign with decision-making.These ambiguities complicate determining whether obligations are met or assigned to the correct party.Source: §2 · pp. 2–3

2.1 Related Work

  • Monolithic technical and governance frameworks can obscure socio-technical context, distributed responsibility, and how impacts materialize in deployment.Prior critiques also warn that model-level fairness tests do not establish fairness across a socio-technical system [26].Source: §2 · p. 3
  • Prior work motivates assessment that links decisions, information flows, actors, obligations, and real-world impacts rather than relying on documentation or fairness metrics alone.Related approaches address auditing, accountability, risk assessment, or agent roles, but do not model the full socio-technical decision structure and automation’s effects across system layers.Source: §2 · p. 3, §2.1 · p. 4, §2.2 · p. 4

2.2 The Need for Structural and Context-Aware Methodologies

  • The proposed methodological need is to assess decision-making structurally and contextually, connecting system elements with roles, responsibilities, and impacts.Source: §2.2 · p. 4
  • The paper examines IFM as a scaffold for structured AI Act assessment of Provider and Deployer obligations across algorithms, decision-makers, and organizational contexts.It aims to reveal mismatches between intended and actual use and make structures that support or undermine accountability visible.Source: §2.2 · pp. 4–5

3 The IFM Framework

The IFM framework represents decision processes as information flows and supports analysis of bias, impact, and responsibility within a unified structure. Its modeling concepts range from sites and channels to networks that can be nested and analyzed compositionally.

3 The IFM Framework

  • IFM provides a structured representation of decision processes that can help identify potential biases and evaluate how outcomes affect stakeholders.The framework treats bias and impact as distinct but connected structural properties of the modeled process.Source: §3 · p. 5, §3.2 · p. 6

3.1 An Overview of the IFM Model

  • An IFM model represents information states as sites, transformations or decisions as channels, and connected sites and channels as a directed network from inputs to outcomes.A process can be represented as a network, with sub-decisions as channels and inputs, outputs, and intermediate results as sites.Source: §3.1 · p. 5

3.1.1 Formal Definition of the IFM Model.

  • IFM networks can be nested, allowing a complex subnetwork to be abstracted as a channel within a larger network.Valid networks are directed acyclic graphs; nesting supports abstraction and causal reasoning about network properties.Source: §3.1.1 · p. 6

3.2 Bias and Impact

  • IFM defines bias as deviation from a decision process’s intended function and impact as potential real-world harm, tracing pathways between decisions and stakeholder outcomes.The model can compare declared responsibilities with operational influence and identify control points for obligations such as oversight and transparency.Source: §3.2 · p. 7

4 The Use Case

The use case is a pre-deployment assessment of a recruitment support tool in a public-sector municipality, conducted with an independent auditor. The assessment considered the Provider’s system, the Deployer’s workflow, and possible integration scenarios.

4 The Use Case

  • An independent auditor collaborated on a pre-deployment assessment of an AI-supported recruitment tool for a municipality preparing for the EU AI Act.The municipality sought to understand how recruitment tools, classified as high-risk, could be assessed under the regulation.Source: §4 · p. 7
  • A private vendor’s service provided interview-based screening, candidate scoring, and shortlisting, and the proposed deployment integrated it into the municipality’s recruitment process.The system had been adopted in several organizational settings but had not undergone external review against AI Act requirements.Source: §4 · p. 7
  • The assessment examined how integration could reshape responsibilities, risks, and decision-making in the local institutional context.The Provider and Deployer shared documentation and participated in stakeholder interviews; the assessment modeled the Provider’s system, the Deployer’s workflow, and integration scenarios.Source: §4 · pp. 7–8
  • The use case involved candidates, recruiters, and recruiting managers, with the manager holding formal responsibility for assigning a position to a candidate.Source: §4 · p. 8

5 A Structured, Role-Sensitive Assessment Methodology

The assessment combines analyses of the Provider’s AI tool, the Deployer’s pre-integration recruitment process, and potential integration scenarios. Each part uses participatory IFM modeling to describe decisions, identify risks, and support AI Act compliance evaluation.

5 A Structured, Role-Sensitive Assessment Methodology

  • The assessment links Provider, Deployer, and integration-scenario analyses to evaluate recruitment decisions and support AI Act compliance.Each stage used participatory IFM modeling to describe decision processes and identify risks.Source: §5 · p. 8
  • Confidentiality limits direct citation of stakeholder documents and interviews, and some details are omitted for anonymization.The authors present insights through the modeling process, focusing on method over case outcomes.Source: §5 · p. 8

5.1 IFM Step-Wise Methodology

IFM constructs and analyzes socio-technical decision structures through a stepwise process. The resulting models support analysis of bias, stakeholders, risk, transparency, oversight, traceability, and alignment with responsibilities or regulatory obligations.

5.1 IFM Step-Wise Methodology

  • Sketches capture process stages, inputs, and outputs with stakeholders, interviews, or documentation as sources.The six-step process begins with informal sketches.Source: §5.1 · p. 8
  • Refinement turns sketches into an IFM network of decision channels, information sites, and their interdependencies.Channels represent decision points, while sites represent information states.Source: §5.1 · p. 8
  • Types and Roles assigns semantic descriptions to sites and channels and associates them with responsible actors or roles.Descriptions clarify the transformations occurring at each step.Source: §5.1 · p. 8
  • Bias and Decisions examines transformations to detect potential local biases and problems.Decision properties include transformation types and potential bias propagation.Source: §5.1 · p. 9
  • Stakeholders and Risk connects decision points and outcomes to affected groups and possible downstream impacts.This supports analysis of interactions between decisions, bias, and impact.Source: §5.1 · p. 9
  • Analysis uses the model to evaluate transparency, oversight, traceability, and alignment with declared responsibilities or regulatory obligations.In the use case, the procedure was applied separately to the Provider’s tool and Deployer’s process, then combined in a final integration phase.Source: §5.1 · p. 9

5.2 Analysis of the Provider

The Provider’s assessment maps how its recruitment tool produces candidate rankings and examines risks across technical and human components. It identifies gaps between declared use, system information, risk controls, and the Deployer’s eventual recruitment context.

5.2 Analysis of the Provider

  • The Provider’s high-risk classification and stated support-only intended use frame the assessment of its tool.The tool is intended to support recruiters rather than make recruitment decisions itself.Source: §5.2 · p. 9

5.2.1 Information Flow Structure.

  • The tool separates candidate interviewing from scoring: a chat module records structured answers, while scoring combines question-level scores into a ranked list.Scoring may use rules, semantic models, or outsourced human assessors; Figure 1 summarizes the model’s inputs and ranked, scored outputs.Source: §5.2.1 · p. 9, Figure 1 · p. 10

5.2.2 Bias and Risk.

  • Question-level scoring appeared to limit potential group-specific bias, but human fallback scoring of whole interviews retains more contextual cues and potential subjectivity.The low potential identified from the information-flow model applies at the scoring-component level and does not rule out system-level discriminatory effects.Source: §5.2.2 · p. 10
  • Recruiters can inspect question-level information, but the scoring source is not disclosed, creating opacity relevant to oversight and accountability.The information is present in the system but not disclosed to the user.Source: §5.2.2 · p. 11
  • Candidates were not told that the conversational interview and scoring use separate models, and some scoring may instead be performed by humans.The natural conversational design can lead candidates to attribute evaluation to the conversational interface.Source: §5.2.2 · p. 11
  • Although the Provider describes the tool as scoring candidates rather than making hiring decisions, recruiters cannot edit scores, potentially making responsible use more work-intensive.In high-volume recruitment, limited time and lack of score-editing support may encourage reliance on the ranking.Source: §5.2.2 · p. 12

5.2.3 Grounding Risk and Compliance Analysis.

  • The Provider’s risk assessment emphasized model-level accuracy and fairness, while leaving hybrid human scoring and automation-bias risks incompletely addressed.IFM exposed coverage gaps, including unclear responsibility assignments and unaccounted residual risks.Source: §5.2.3 · p. 12
  • Residual risks, particularly automation bias and candidate misunderstanding, may shift toward the Deployer in practice.The Provider analysis therefore informs modeling of the Deployer’s process and assessment of how integration choices might amplify or mitigate those risks.Source: §5.2.3 · p. 12

5.3 Analysis of the Deployer pre-integration

The Deployer’s recruitment process was modeled from interviews and organizational documents, revealing both a strong commitment to fairness and important variation across recruitment contexts. In the high-volume track, information order and formal scoring shape decision-making and accountability.

5.3 Analysis of the Deployer pre-integration

  • Ten stakeholder interviews and internal documents revealed extensive fairness guidance, varied operational practices, and a consistent commitment to human control and fair recruitment.Interviewees included recruiters, HR personnel, technical staff, and recruiting managers.Source: §5.3 · p. 13
  • The Deployer models represent recruiters and the Applicant Tracking System in one socio-technical model, illustrating IFM’s unified treatment of human and automated decisions.Interviews were the primary source for modeling the Deployer environment, with assessors refining sketches compiled during note-taking.Source: §5.3 · p. 13
  • Changing decision order changes which evidence is available and can alter the grounds for final decisions.This matters when considering AI integration or replacement of process components.Source: §5.3 · p. 13
  • Recruitment varies by context: executive hiring is collaborative and multi-step, whereas high-volume hiring prioritizes throughput and role-matching.Both tracks were modeled, but this paper focuses on the high-volume track.Source: §5.3 · p. 13
  • The high-volume process relies on video interviews, formal records of candidate attributes, and recruiter sorting, while the recruiting manager participates only at the start and end.The video assessment shapes initial impressions and gives visual and linguistic presentation significant evaluative weight, potentially exposing the process to recruiter bias.Source: §5.3 · pp. 13–14
  • The public-sector process is already high-risk from a Fundamental Rights perspective, given its diverse candidates and effects on access to employment and public services.The organization has internal policies and legal frameworks to support accountability, but the analysis also raises ethics questions about human decision procedures.Source: §5.3 · p. 14

5.4 Model Composition and Integration

The section models alternative ways to integrate the AI tool into the high-volume recruitment process. Comparing these configurations makes visible how automation changes information flows, decision authority, oversight, and potential impacts.

5.4 Model Composition and Integration

  • The integration model compares three configurations: pre-screening only, tool scoring alongside interviews, and tool-based shortlisting that replaces interviews.The composite models show how integration choices shift decision points, responsibilities, and information availability.Source: Figure 3 · p. 14, §5.4 · p. 14
  • In minimal integration, the tool filters candidates before recruitment, but its score is discarded and later interviews, evaluation, and sorting remain unchanged.Automation bias is structurally limited because outputs do not affect downstream decisions, though early filtering may still be over-relied on.Source: §5.4 · p. 15
  • In double evaluation, recruiters see tool and interview scores without a rule for reconciling them, creating ambiguous decision logic and potentially harder-to-detect impacts.Recruiter agency may be reduced if human evaluations become secondary.Source: §5.4 · p. 15
  • In maximal integration, the tool replaces interviews and generates shortlists, centralizing evaluation and limiting recruiter agency.Technical traceability improves, but semantic richness is lost; responsibility may become ambiguous when recruiters lack tools or incentives to challenge outputs.Source: §5.4 · p. 15
  • The three scenarios support structured comparison of how automation configurations reshape responsibility, oversight, and potential bias or harm.They model the same socio-technical context while varying where and how the tool is integrated.Source: §5.4 · pp. 14–15

5.5 Project Outcomes

Follow-up interviews found that the IFM assessment influenced organizational understanding and practice. Its report informed process changes, education, continued review, and further activities related to automation bias and transparency.

5.5 Project Outcomes

  • The municipality implemented formal checkpoints and documentation routines to clarify responsibility and information flow in hiring.The assessment report supported management discussions and planning by identifying information gaps and decision dependencies.Source: §5.5 · p. 16
  • The municipality used modeling insights in internal education and process development, and stakeholders valued participatory sessions as opportunities for collective reflection.The process was described as demanding, and its technical language sometimes challenged participants’ familiarity.Source: §5.5 · p. 16
  • Recruitment stakeholders continued using the report for review and practice alignment, while technical stakeholders valued it as an educational instrument about automation and human decision-making.The work also inspired municipal initiatives on automation bias and transparency, and interest in further audit and governance collaborations.Source: §5.5 · p. 16

6 Discussion

The discussion presents IFM as a way to analyze AI integration within a unified socio-technical structure rather than treating systems or deployment processes in isolation. It also considers how the approach relates to assessment practice and the barriers and future directions that qualify its use.

6 Discussion

  • IFM decomposes systems and deployment processes into interrelated elements while preserving context, making shifts in system behavior and responsibility visible across integration choices.The approach supports both granular structural analysis and holistic reasoning about transparency, user responsibility, stakeholder visibility, and distribution of impact.Source: §6 · p. 16

6.1 Reflections on the assessment

  • The assessment surfaced tension between automation bias, intended use, and users’ informational needs, alongside questions about Providers’ partial responsibility for outcomes.Recruiters’ categorization work requires fine-grained access to candidate responses and contextual cues; the discussion proposes active semantic interpretation as an alternative to passive oversight.Source: §6.1 · p. 17

6.2 Limitations and Further Work

  • IFM’s technical modeling layer and the interpretation needed to connect informal accounts with technical and socio-organizational context may present adoption barriers.The method requires familiarity with the domain being modeled, including its social and technical components.Source: §6.2 · p. 17
  • Applications beyond recruitment, including role-dependent transparency analysis and private-sector processes, are presented as promising directions for further work.The discussion notes that applying IFM to transparency or explainability claims can involve tracking information access across roles and components.Source: §6.2 · p. 17

7 Conclusion

The paper applies IFM in a real-world external assessment of a high-risk recruitment system, using it as a scaffold for situated reasoning about integration, responsibility, and stakeholder impact. It argues that modeling decision processes can connect technical and sociotechnical analysis and make assessment results more actionable.

7 Conclusion

  • IFM served as a structural scaffold for reasoning about integration, responsibility, and stakeholder impact in a real-world recruitment assessment; the study did not evaluate the system itself.The authors describe IFM as supporting both granular and holistic understanding of automation’s situated effects.Source: §7 · p. 17
  • Representing human and automated actors within a shared formalism bridges procedural understanding and system-level analysis, supporting transparency, user responsibility and agency, and stakeholder-impact tracing.This counters reducing complex sociotechnical processes to technical objects or treating fairness and compliance as internal system properties.Source: §7 · pp. 17–18
  • Decision modeling reframes analysis from isolated technical properties to decision processes, revealing real-world structures, risks, and effective remedies in AI integration.The authors argue that linking compliance to visible, transparent positive effects can make assessments more actionable and help avoid tokenistic compliance.Source: §7 · p. 18

Related papers

Loading 2610.16916v1…