PaperlayerView exact revision on arXiv

Source-linked AI summary

Connectedness, Cognitive Load, and Human-AI Oversight in Cyber Operations

Nathan Conklin, Peng Gao, Chris North

arXiv:2610.02384v1cs.HC

TL;DR

Human review of AI-generated cyber reasoning traces is constrained by operator attention, yet risk-based routing assumes capacity is constant. The paper proposes using platform telemetry to estimate operator load and connectedness, then allocate review actions accordingly. Its central conclusion is that connectedness serves as both a load signal and a boundary for collective support, while the empirical prediction remains a hypothesis.

  • Problem

    Risk-based routing uses model confidence or uncertainty but assumes constant reviewer capacity, although environmental load can sharply change the operator’s ability to oversee traces.

    Source: §1 · pp. 1–2

  • Method

    The proposed connectedness-aware engine uses existing platform telemetry to estimate operator capacity and allocates trace review depth and escalation based on capacity and risk.

    Source: §1 · p. 2, §6 · p. 5

  • Results

    The paper presents connectedness as both an environmental load signal and a boundary determining whether the human-AI dyad can reach a broader collective for support.

    Source: §6 · p. 5, §8 · p. 6

  • Takeaways & Limitations

    The framework aims to route scarce human attention where it protects the mission while preserving the watchstander’s ability to reason, believe, remember, judge, and act.

    Source: §8 · p. 6

    Whether environmental telemetry predicts measurable degradation in human review performance is a falsifiable hypothesis, not a reported result.

    Source: §7 · p. 6

Abstract

from arXiv · show

AI-assisted cyber situational awareness triggers machine-generated reasoning traces (step-by-step justifications for anomaly classifications) that a human operator is expected to review. Because cyber signals and their traces arrive faster than any operator can process, human review is the limiting constraint on oversight. The standard approach is to identify the riskiest cyber events for review using model-side signals such as confidence or uncertainty. That framing ignores the operator's cognitive capacity which varies sharply with the operational environment. We propose an alternative where the system's environmental and connectivity telemetry serves as an available, non-invasive proxy for operator load. That same telemetry determines whether the human-AI partnership can reach the broader collective for support. In a maritime platform, environmental and connectivity attributes including depth, number of active communications paths, density of the tracked contact picture, and operational tempo all carry this signal. Need for operator oversight becomes a decision that materializes as a combination of both risk and environment-derived operator capacity. We present a reference architecture for a connectedness-aware oversight engine, demonstrating everyday use cases alongside its intended incorporation into the submarine cyber-defense toolkit. Two themes emerge: 1) the operator's environmental state is itself a connectedness measurement, and 2) connectedness drives the cognitive load and defines a collective boundary in human-AI cyber operations.

1 Introduction

AI reasoning traces arrive faster than people can review them, making operator attention the constraint on oversight. The paper argues that oversight must account for environmentally driven changes in operator capacity, not just model-side risk.

1 Introduction

  • Model-side risk routing assumes reviewers have constant capacity, but environmental load can leave operators unable to evaluate data, accuracy, or usefulness.The paper frames this as a cognitive-security problem affecting attention, belief formation, memory, judgment, and agency.Source: §1 · p. 1
  • On submarines, Fire Control Technicians oversee cybersecurity alerts across a platform of nearly a thousand nodes.Many alerts are misconfigurations or conditions that cannot be cleared at sea; alert fatigue can lead operators to stop trusting and attending to the display.Source: §1 · pp. 1–2
  • The cybersecurity display presents network and system status, threat posture, and active alerts for operator review and action.Source: Figure 1 · p. 2
  • In maritime operations, the conditions that raise operator load are also those in which a real intrusion can do the most damage.Source: §1 · p. 2
  • The proposed approach makes oversight a joint function of model-side risk and an estimate of operator capacity derived from existing platform telemetry.The introduction also frames cognitive load as a vulnerability, the platform as a non-invasive sensor, and connectedness and load as signals for exposure and operations.Source: §1 · p. 2

2 Background & Related Work

Prior work describes how cognitive limits, stress, and alert overload can impair monitoring, and how automation can produce both disuse and overtrust. The paper situates its approach among adaptive automation and non-invasive environmental load estimation, while identifying intermittent connectivity as an additional operational constraint.

2 Background & Related Work

  • Working memory is narrowly bounded, and exceeding its capacity can collapse processing rather than merely slow it.Stress can narrow attention and degrade working memory, judgment, and flexibility (Girotti et al. 2018; Klein & Boals 2001; Montan ez et al. 2020).Source: §2 · p. 2
  • Alert overload is associated with learned disregard: security teams average 4,484 alerts daily, cannot action 67%, and report 83% false positives (Vectra AI 2023).Clinical alarm studies also report high invalid-alarm rates and slower responses to true events (Burdick et al. 2023; Bonafide et al. 2015; Drew et al. 2014).Source: §2 · p. 2
  • Automation can produce both distrust and disuse after false alarms, and complacency or overtrust when reliable systems lull multitasking operators.Adaptive automation reallocates work to keep demand in a workable band.Source: §2 · pp. 2–3
  • Physiological load estimation can exceed 90% laboratory accuracy but depends on body-worn or eye-tracking sensors, while environmental-context methods avoid instrumenting the operator (Ma et al. 2024; Meiser et al. 2022; Piechulla et al. 2003).Source: §2 · p. 3
  • Reasoning traces may be unfaithful to the computation behind them, and a live security-operations study found analysts used models for sensemaking rather than final determinations (Korbak et al. 2025; Lanham et al. 2023; Turpin et al. 2023; Singh et al. 2025).The paper adds that mobile operations face intermittent connectivity, which affects both cyber exposure and access to help.Source: §2 · p. 3

3 Cognitive Load as a Cognitive Vulnerability

The paper treats environment-driven cognitive load as a cognitive vulnerability: high-volume AI alerts meet bounded working memory and can produce alert fatigue and learned disregard. It links that vulnerability to degradation across all five cognitive-security dimensions.

3 Cognitive Load as a Cognitive Vulnerability

  • High-volume machine-generated alerts and reasoning traces meet bounded working memory under environmental stress, producing alert fatigue and learned disregard of the display.The paper characterizes this combination as a cognitive vulnerability that corrodes oversight across all five cognitive-security dimensions.Source: §3 · p. 3
  • High load can narrow attention, harden beliefs that the display is noise, consume working memory, and degrade judgment about true positives versus benign misconfigurations.Source: §3 · p. 3
  • Chronic overload can push operators toward disuse or automation deference, leaving them no longer reasoning for themselves.Source: §3 · p. 3
  • The paper depicts Cognitive Load as an indicator estimated non-invasively from platform environmental and connectivity telemetry.Source: Figure 2 · p. 3

4 Fire Control Technician Use Case

The Fire Control Technician monitors a cybersecurity display alongside demanding submarine duties and a network spanning nearly a thousand nodes. Heterogeneous alerts and high-load operational states can undermine review precisely when intrusion consequences and recovery difficulty are greatest.

4 Fire Control Technician Use Case

  • A submarine Fire Control Technician manages combat-control duties and monitors cybersecurity findings across a network of nearly a thousand nodes.The display includes asset health, subnet hierarchy, topology, an alert queue, and per-node findings such as rootkit indicators.Source: §4 · p. 3
  • The alert list mixes malicious command-and-control connections and rootkits with firmware, antivirus, and disk-space notices, some of which crews cannot clear at sea.The resulting benign-alert learning can lead the operator to close or ignore alerts, rendering the display background noise.Source: §4 · p. 3
  • Load rises during self-protect events, shallow or surfaced transit, hull-opening evolutions, dense contact pictures, and high maneuver tempo.These are conditions in which additional reasoning traces compete with mission-critical work.Source: §4 · pp. 3–4
  • The conditions that increase load also make intrusion more damaging and harder to recover from because the boat may be unable to reach shore support.A model-side-only router would send the same notification volume during quiet and demanding transits.Source: §4 · p. 4
  • The depicted operator is overloaded with alert fatigue and begins ignoring notifications.Source: Figure 3 · p. 4

5 Telemetry-Derived Cognitive Load and Cyber Exposure

Platform telemetry estimates room-level cognitive load and cyber exposure separately, then uses them to adapt alert escalation to operator capacity and external risk.

5 Telemetry-Derived Cognitive Load and Cyber Exposure

  • The estimator separates room-level cognitive load L(t) from cyber exposure E(t), mapping L(t) inversely to operator-capacity budget Ĉ(t).The estimates use combat-system telemetry and require no wearables, biometric sensors, new emissions, or crew interaction; the proposed heuristics still require calibration against recorded telemetry and human-review performance.Source: §5 · p. 4
  • High load supports deferring low-value notifications, while high exposure lowers the escalation threshold, even when the two quantities move in opposite directions.Routine communications can mean low load with high exposure; torpedo evasion without external connectivity can mean high load with low exposure.Source: §5 · p. 4
  • When both load and exposure are high, alert importance and connective boundary B(t) determine whether the system suppresses, interrupts, or escalates.Source: §5 · p. 4
  • Routine displays, navigation, mission products, manual overrides, display-change rate, and non-cyber alert volume are excluded signals; most are non-discriminating, and some may be adversary-influenced or attack-correlated.Source: §5 · p. 4

6 Connectedness-Aware Oversight

The connectedness-aware oversight engine combines per-trace risk with environment-derived capacity to allocate review, while platform connectivity also determines whether escalation to a wider collective is available.

6 Connectedness-Aware Oversight

  • The engine joins per-trace risk R(τ), environment-derived capacity C(t), and an allocator that selects a review action for each trace.Risk draws on confidence and uncertainty, faithfulness proxies, and trace structure; trace length also estimates review cost for the capacity budget.Source: §6 · p. 5
  • Figure 4 frames alert management as using operator cognitive load to control notifications.Source: Figure 4 · p. 5
  • Platform telemetry—including depth, active communications paths, contact density, display state, operational tempo, and alert volume—feeds C(t), shown to the watch as the Operator Cognitive Load indicator.Source: §6 · p. 5
  • Connectivity defines boundary B(t): with communications available, ambiguous high-risk traces can reach a shore cyber cell; when deep and emissions-constrained, the dyad must self-resolve.The same physical conditions that raise operator load can also close access to the broader collective.Source: §6 · p. 5
  • The allocator spends capacity on higher-value reviews, from auto-handle and log through spot-check, full audit, and collective escalation when B(t) permits.As capacity falls, the system raises the bar for human review and presents the highest-risk traces as highlighted steps; elevated exposure lowers the escalation threshold independently of load.Source: §6 · p. 5

7 Discussion

The discussion treats calibrated reliance as a design concern, specifies audit and sampling safeguards, and identifies both an untested empirical hypothesis and broader scope conditions.

7 Discussion

  • Load-driven automation can reduce disuse but also risks complacent rubber-stamping, so the design retains auto-handled traces and periodically forces review.The stated goal is appropriate reliance, which the paper connects to team performance, especially in complex, uncertain tasks where human and machine error patterns diverge (Schemmer et al. 2023).Source: §7 · pp. 5–6
  • The CVMC framing lists catch-rate-per-unit-capacity, miss rate, time-to-disposition, and calibrated reliance as evaluation metrics, alongside auditability and burden-and-equity checks.Source: §7 · p. 6
  • Environmental telemetry predicting measurable degradation in human review performance remains a falsifiable hypothesis, not a result.If environmental signals add nothing over model confidence, the contribution narrows to the routing baseline; the paper identifies instrumented, environment-driven settings with intermittent connectivity as a broader target scope.Source: §7 · p. 6

8 Conclusion

The conclusion frames AI oversight as dependent on human capacity, which operational environments shape and platforms already measure.

8 Conclusion

  • The conclusion links environment-derived capacity and connectedness to routing scarce human attention while preserving watchstanders’ conditions for reasoning, belief, memory, judgment, and agency.Source: §8 · p. 6

APPENDIX A

Appendix A records the prompt for Figure 1’s submarine cybersecurity dashboard, including its scale, interface requirements, and visual style; actual system figures are identified as unpublishable in an open forum.

APPENDIX A

  • The text says example figures are DIST A releasable, while actual system figures are CUI or SECRET and not publishable in an open research forum.Source: Appendix A · p. 8
  • The prompt describes a submarine system with nearly 500 Linux compute machines, nearly 200 switches, and roughly 100 IoT systems.The dashboard is intended to represent cybersecurity activity across roughly 800 nodes.Source: Appendix A · p. 8
  • The requested overview shows node health, prioritized alerts, a SECRET banner, an alert panel, and tabs for audit, cross-domain, and data-flow details.Alert colors distinguish red as most important, yellow as intermediate, green as least critical, and gray as notification or configuration items.Source: Appendix A · p. 8
  • The visual specification calls for a black 16:9 display, dark-gray healthy nodes, and red, yellow, or green indicators for alerted nodes.Source: Appendix A · p. 8
  • The prompt names Linux cyber clients including LUKS/cryptsetup, SELinux, AppArmor, Wazuh, firewalld/nftables, and auditd.Source: Appendix A · p. 8

Related papers

Loading 2610.02384v1…