PaperlayerView exact revision on arXiv

Source-linked AI summary

Spike-Killer: Evidence-Gated LLM Assistance for Safe Performance Diagnosis on a Real Windows Workstation

Baocheng Zeng, Jinhao Yang

arXiv:2608.21069v1cs.SE

TL;DR

Performance-diagnosis agents can be useful but can also mutate the wrong target or perturb measurement on a real workstation. Spike-Killer uses human-approved, evidence-gated transactions to audit diagnosis on one Windows laptop, producing integrity-validated CS2 traces while deliberately making no frame-time improvement claim.

  • Problem

    LLM-assisted performance diagnosis lacks a safe, auditable way to handle heterogeneous evidence and risky actions or collectors on a real workstation.

  • Method

    Spike-Killer uses human-approved transactions that classify risk, snapshot exact state, apply narrow actions, verify postconditions, and retain failed measurements.

  • Results

    90.69 and 85.85 seconds: two CS2 local-Bot WPR traces passed integrity checks with cs2.exe attribution, DxgKrnl Present metadata, and zero lost ETW buffers or events.

  • Takeaways & Limitations

    The study demonstrates an auditable, human-in-the-loop pattern for trustworthy assistance with explicit stopping when evidence is insufficient.

  • Takeaways & Limitations

    The single-machine, same-day case produced no valid controlled frame-time evidence, P99 estimate, causal intervention effect, or autonomous-agent evaluation.

Abstract

from arXiv · show

LLM-assisted agents can synthesize system evidence, propose configuration changes, and automate diagnostic tasks, but their flexibility makes an imprecise action or an intrusive collector an operational risk. We present Spike-Killer, a human-approved workflow for diagnosing frame-time complaints on one real Windows workstation. The workflow treats each action as an evidence-gated transaction: it records the exact target state, classifies risk, preserves a snapshot, verifies a postcondition, and retains failed measurements as first-class evidence. This experience paper reports a completed same-day study with Counter-Strike 2 as a demanding target application. The evidence bundle contains preserved state snapshots, exploratory microbenchmarks, a ten-run same-state repeatability probe, live telemetry, a repaired over-broad registry action, incompatible presentation-capture attempts, an invalid local replay, and a system-level tracing replacement. Windows Performance Recorder produced two CS2 local-Bot GPU traces of 90.69 and 85.85 seconds; both were attributed to cs2.exe, exposed DxgKrnl Present metadata, and had zero lost ETW buffers or events. These results qualify trace integrity, not performance: the study reports no frame intervals, P99 estimate, or intervention effect. The contribution is an auditable, human-in-the-loop pattern for trustworthy agent assistance on a real workstation, including explicit stop conditions when evidence is insufficient.

1 Introduction

Spike-Killer frames intermittent-stutter diagnosis as a frame-time and safety problem rather than an average-FPS optimization exercise. Its narrow same-day case asks whether human-approved LLM assistance can make troubleshooting on one laptop more auditable and experimentally testable.

  • Motivation: Frame time, rather than average FPS alone, is the relevant outcome for intermittent-stutter complaints.Average FPS can conceal a small number of long frames, while frame-timing variation changes perceived smoothness.
  • Motivation: Potential causes cross application, user, driver, and privileged operating-system boundaries, making evidence and rollback essential.Capture hooks, login bursts, rendering resolution, and OEM power settings can all affect the system.
  • Motivation: An LLM can connect heterogeneous evidence and generate inspection code, but a syntactically valid action or collector can target the wrong object or perturb the machine.The paper treats flexibility as both an assistance capability and an operational risk.
  • Study question: Spike-Killer asks whether safety-constrained LLM assistance can make frame-time troubleshooting on one fixed laptop more auditable and experimentally testable within one day.The study does not compare models, claim autonomy, or generalize to other computers.
  • Evidence boundaries: The pilot supports cross-layer evidence, telemetry, symptom observations, and preserved failures, but not P99 reduction, causal attribution, collector non-interference, or unsupervised agent safety.A multi-day randomized crossover estimating CS2 P99 intervention effects is explicitly out of scope.
  • Contributions: The paper contributes a transaction-like workflow, a traceable single-machine case, first-class failure analysis, and executable checks for loss-free CS2-attributed WPR traces.The checks verify integrity without exporting private game contents.

2 Background and Related Work

The background motivates frame-time endpoints and treats measurement itself as an intervention. Spike-Killer therefore narrows its claim to auditable operation on one machine rather than persistent performance improvement or population-level safety.

  • Frame-time motivation: Long or irregular inter-frame intervals can remain perceptible when average throughput is high, motivating P99 and long-frame-rate endpoints.The paper cautions against selecting a result after inspecting many FPS summaries.
  • Measurement effects: Presentation tracing, overlays, graphics capture, hardware capture, and application-controlled timing observe different pipeline points and impose different costs.A familiar interface around the same underlying collector is not an independent fallback.
  • Measurement effects: Collectors are treated as interventions that must pass responsiveness, focus, completeness, and data-quality gates before producing publication data.This extends safety controls from configuration changes to measurement tooling.
  • Related work: Performance engineering differs from ordinary desktop completion because the desired state is statistical, the machine has thermal history, and privileged actions may have delayed side effects.A successful command does not establish a correct state transition or performance benefit.
  • Related work: The experience complements harm benchmarks with an in-situ example in which ordinary maintenance intent produced an unsafe target scope.The paper uses the incident to derive enforceable guards without inferring population-level safety rates.
  • Scope: The study uses one same-day machine experience to test whether its workflow, guards, and measurement path operate on the target computer, not whether an intervention persists.Later N-of-1 or randomized crossover work is not reported.

3 Spike-Killer Design

Spike-Killer decomposes performance maintenance into evidence-gated stages and reversible intervention layers. Human approval, deterministic target control, exact postconditions, snapshots, and retained failures constrain the LLM’s role.

  • Eight-stage workflow: The workflow observes system evidence, hypothesizes a tail-latency mechanism, classifies risk, and snapshots the exact original state before mutation.These stages establish the evidence and authority boundary before action.
  • Eight-stage workflow: It then applies the narrowest low-privilege change, validates the exact postcondition, measures a frozen workload, and decides whether to retain, restore, or mark the action inconclusive.Command exit status is not treated as sufficient validation.
  • Intervention layers: The artifact organizes reversible layers from a restored baseline through CS2/display configuration, capture-path removal, background cleanup, and system tuning.The layer records include intended mechanisms and individually checked state fields.
  • Intervention layers: Capture-path removal targets Windows Game DVR and NVIDIA recording or overlay hooks.This is represented as a distinct layer rather than bundled with game configuration.
  • Intervention limits: The layers estimate conditional increments rather than independent factorial main effects, and remote-driver removal was not randomized because restoration was untrustworthy.Defender exclusions are excluded because security policy would add risk and a mechanism confound.
  • Implementation boundary: The formal controller remains incomplete because action separation, Defender handling, and per-field original-value restoration still require changes before confirmatory collection.The paper distinguishes a workflow diagram from guards already enforced in scripts.
  • Audit and authority: Each action record retains identifiers, hypotheses, exact targets, privileges, reversibility, risks, postconditions, side effects, and final dispositions, including failures and repairs.Publication artifacts must redact machine-specific sensitive data.
  • Audit and authority: A deterministic controller, not the LLM, should enforce allowlisted targets, snapshots, approval for consequential actions, and requested-state comparison.The human operator retains mutation approval, workload observation, and stopping authority.

4 Study Method

The completed study combines pilot evidence streams on one Windows gaming laptop, including state transitions, repeatability, telemetry, and collector outcomes. It establishes feasibility and trace integrity, but leaves intervention effects and autonomous-agent performance unanswered.

  • Study questions: The study reports a completed motivating pilot and a prospective protocol addressing feasibility, current system evidence, safety and measurement, and prospective P99 performance.Pilot evidence covers RQ1–RQ3, while RQ4 remains unanswered.
  • System and case: The test system was one Windows 11 gaming laptop with an Intel Core i9-14900HX, RTX 4080 Laptop GPU, approximately 32 GB RAM, and a 1920x1080 144 Hz display path.The July 25 manifest recorded exact Windows and NVIDIA driver versions, but those versions would be frozen anew for a future effect study.
  • System and case: The motivating CS2 case combined 3840x2160 rendering with 4x MSAA on a 1920x1080 display, enabled capture paths, OEM PCIe power saving, login applications, and virtual devices.The workflow bundled interventions, rebooted, repeated a local microbenchmark, collected later live-match telemetry, and obtained an operator report.
  • Measurements: Three-run CPU, memory-copy, and sequential-disk measurements were recorded before intervention, after user-level actions, and after reboot.Reported medians characterize script invocations rather than independent experimental sessions.
  • Measurements: A 10-run same-state probe after warm-up estimated short-session repeatability for fixed SHA-256 and memory-copy workloads without comparing treatment conditions.Process, memory, and GPU state were recorded at five-second intervals.
  • Measurements: The live-match stream contained 864 samples over 29.93 minutes with process-memory and GPU telemetry, but no valid per-frame series, baseline match, or controlled workload.It provides ecological context rather than a frame-time improvement test.
  • Measurements: The unblinded operator report recorded whether the original symptom remained but used no rating scale and cannot establish magnitude or mechanism.This is a subjective observation rather than a controlled outcome.
  • Evidence streams: The protocol retained state snapshots, microbenchmarks, repeatability data, telemetry, failed collectors, and WPR validation artifacts.WPR validation checked trace existence, hash, duration, attribution, Present metadata, and ETW loss.

5 Same-Day Empirical Evidence

The same-day evidence bundle documents measurable system-state changes, short-session benchmark variability, live telemetry, symptom relief, and trace-integrity checks, while stopping short of causal frame-time claims.

  • Background state and exploratory microbenchmarks: Process count fell by 97 (318 to 221), while immediately available memory rose by 7.69 GB (16.59 to 24.28 GB) after the intervention-plus-reboot case.The snapshots followed 46.16 hours and 0.14 hours of uptime, so they do not isolate startup cleanup or an LLM effect.
  • Background state and exploratory microbenchmarks: The three-run microbenchmark artifact preserved machine-readable observations but did not establish why throughput changed or whether game latency changed.CPU frequency, temperature, warm-up, uptime, file cache, and storage caching were insufficiently controlled, and the disk script was not a physical-media throughput benchmark.
  • Same-state repeatability pilot: A ten-run same-state probe measured median CPU SHA-256 throughput of 2425.43 MB/s with 3.27% variation and memory-copy throughput of 16.47 GB/s with 3.64% variation.One CPU run reached 2194.47 MB/s while the other nine ranged from 2351.40 to 2454.07 MB/s.
  • Same-state repeatability pilot: The one-minute repeatability probe showed a moving background environment, with process count changing from 283 to 266 and available memory reaching 17.61 GB.It supports session-level blocking and state manifests but does not strengthen the causal intervention claim.
  • Live-match telemetry and symptom report: During the 29.93-minute optimized session, GPU utilization was 45% at the median and 66% at P95, with no evidence of sustained GPU saturation, extreme temperature, or VRAM exhaustion.The telemetry cannot identify a frame-time cause, and its interpretation is limited by two-second sampling and absent independently logged throttling thresholds.
  • Live-match telemetry and symptom report: The operator reported that gunfight stutter was no longer observed, but the bundled intervention, online workload, adaptation, and other factors remained plausible explanations.The study therefore reports symptom change without treating it as controlled performance evidence.
  • WPR local-Bot integrity preflight: The study reports no P99 estimate, confidence interval, or condition effect because the PresentMon artifact lacked integrity and matched control, and no B0–B4 comparison was attempted.The WPR checks reduced collector risk but did not establish unchanged workload, no collector overhead, frame-to-present mapping, or per-frame intervals.
  • WPR local-Bot integrity preflight: Two local-Bot WPR traces of 90.69 and 85.85 seconds passed limited integrity checks for cs2.exe attribution, DxgKrnl Present metadata, and zero lost ETW buffers or events.The traces were retained as integrity evidence, not frame-time sessions.

6 Safety and Failure Analysis

The study preserved concrete safety and measurement failures, showing why performance actions and collectors require shared evidence gates, exact postconditions, and human stopping authority.

  • Over-broad registry mutation: The agent deleted the machine-wide Windows Run key instead of one named startup value, and the postcondition check detected the over-broad mutation.Previously enumerated startup entries were reconstructed.
  • Over-broad registry mutation: Four guards require exact typed targets, rejection of container deletion, complete original-state snapshots, and comparison of intended and neighboring state.Human approval remains necessary but cannot compensate for a misleading target.
  • PresentMon incompatibility: PresentMon-family attempts repeatedly made CS2 non-responsive and recorded ETW losses of 72,814 to 324,890 events, so none met the publication gate.A short CSV did not override failures in responsiveness, completeness, pairing, or minimum-frame requirements.
  • Crash during measurement: A July 25 bugcheck occurred during replacement-collector preflight, but the available evidence does not establish whether the collector, CS2, PowerShell, NTFS, or earlier optimization caused it.The correct response was to halt that measurement family, preserve evidence, and avoid causal wording.
  • Safety implications: The control plane stops on over-broad scope, missing snapshots, focus changes, anti-cheat warnings, hangs, event loss, invalid traces, crashes, or failed postconditions.Rejected or neutral interventions remain experimental observations rather than being silently retried.

7 Discussion

The case supports LLM assistance as evidence integration and experiment construction under deterministic policy and human authorization, while emphasizing that collector failures can dominate optimization attempts.

  • Where LLM assistance helped: The workflow integrated game configuration, registry state, startup inventories, driver descriptions, power settings, scripts, benchmarks, telemetry, and crash logs into hypotheses and auditable test designs.The observed value was integration rather than autonomous optimization.
  • Study scope: The same-day case asks whether a guarded workflow can operate on one real machine while distinguishing scoped actions, invalid captures, and loss-free traces.It cannot establish persistence across days, thermal states, or software updates.
  • Measurement is part of the system: Failed collector pilots exposed the reproducibility mistake of naming familiar tools without qualifying their observation point, configuration, data loss, and compatibility.If no backend passes the gate, the result remains an experience report about safe action and observer effects.
  • From prototype to controlled agent: The artifact is a research prototype whose next controller must validate schemas, resolve allowlisted targets, capture original state, enforce postconditions, orchestrate reboots, and emit atomic audits.Agent proposals should be declarative while a separate executor determines permissibility.

8 Threats to Validity

Threats to validity constrain causal, construct, external, safety, and temporal conclusions: the study lacks controlled frame-time evidence, broad replication, denominator-based safety rates, and frozen software conditions.

  • Internal validity: Bundled changes, uncontrolled thermal and software conditions, no concurrent live-match control, and unblinded observation prevent causal attribution of game-performance effects.Baseline and post-reboot measurements also differ sharply in uptime.
  • Construct validity: Process counts, free memory, microbenchmarks, and short GPU telemetry do not measure CS2 frame intervals, while WPR integrity does not create them; P99 remains unobserved.The measurements characterize background state or capture integrity rather than the target endpoint.
  • External validity: Evidence comes from one Windows gaming laptop, one user, one game, one environment, and one day, so no claims extend to other machines, games, systems, or models.Multimachine replication and multi-day persistence remain future work.
  • Safety-evaluation validity: One documented registry incident and several instrumentation failures reveal failure modes but cannot estimate a failure probability or support a denominator-based safety rate.Some actions predate the structured audit schema, making reconstruction weaker than contemporaneous append-only traces.
  • Reproducibility and temporal validity: Changing CS2, Windows, drivers, and agent models threaten reproducibility, requiring frozen versions, preserved invalid traces, exact prompts and policies, and sanitized manifests.A new environment stratum is needed after unavoidable updates.

9 Conclusion

Spike-Killer demonstrates both the feasibility and limits of safety-constrained LLM assistance on one gaming laptop, with actionable failure evidence but no controlled frame-time claim.

  • Conclusion: The study produced verifiable background-state changes, exploratory benchmarks, a 10-run repeatability pilot, live telemetry, and subjective symptom relief, but no valid controlled frame-time evidence.Accordingly, it makes no P99 improvement claim.
  • Conclusion: An over-broad registry mutation, PresentMon incompatibility, and an unexplained temporally associated kernel crash support exact target resolution, snapshots, postconditions, collector gates, and human stopping authority.These failures are treated as actionable results rather than discarded anomalies.
  • Conclusion: A causal P99 claim would require a valid workload, linkage from Present events to frame intervals, and a repeated controlled study beyond this same-day submission.The paper should therefore be read as a single-machine experience report about safe evidence collection and collector failure modes.

Generative AI Use Statement

Generative AI tools assisted with hypothesis organization, drafting, and implementation scaffolding, while human authors retained review, decision-making, and responsibility.

  • Generative AI tools assisted with hypothesis organization, draft text, and implementation scaffolding.
  • Human authors reviewed all code and evidence and made all research and writing decisions.
  • The human authors take full responsibility for the submitted work, and no generative AI system is an author.

Data Availability Statement

The manuscript is accompanied by validation descriptions and aggregated results, while release plans separate public artifacts from private machine-specific data and approval-sensitive actions.

  • The manuscript is accompanied by executable validation descriptions and aggregated results.
  • A de-identified artifact package is being prepared for public release, while raw Windows ETL traces and machine-specific logs remain private.
  • The published artifact will distinguish inspection-only scripts from actions requiring explicit human approval on a real machine.
Loading 2608.21069v1…