PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic AI for operating scientific instruments for nanoscale characterization

Zahra Ayar, Marcos Penedo, Mahdi Mehdikhani, Nahid Hosseini, Prabhu Prasad Swain, Georg E. Fantner

arXiv:2608.26198v1cs.AIphysics.ins-det

TL;DR

Operating an AFM requires continuous expert decisions, while existing automation covers only parts of the workflow. This paper presents a guarded, MCP-based agentic system for command execution, image-based tuning, and post-processing, achieving performance comparable to AFM operators in live experiments.

  • Problem

    AFM imaging requires continuous parameter adjustment and artifact-specific correction, but image-driven coordination of execution, tuning, and post-processing largely remains human.

  • Method

    Three MCP-based agents translate instructions into validated commands, tune imaging parameters from AI-vision assessments, and apply transparent post-processing from approved tools.

  • Results

    In live experiments, AFM Pilot achieved performance comparable to AFM operators under matched detuned conditions.

  • Takeaways & Limitations

    The framework supports agentic operation of an AFM while leaving experimental intent human-defined and delegating execution, tuning, and post-processing to AI agents.

  • Takeaways & Limitations

    The system does not yet decide which experiment to perform, so experimental intent remains human-defined.

Abstract

from arXiv · show

Operating a scientific instrument such as an atomic force microscope (AFM) requires continuous expert decision-making. A trained user defines the experimental intent, translates it into instrument commands, assesses incoming data, adjusts imaging parameters, and post-processes the final image. Existing automation usually addresses only parts of this workflow through hard-coded routines, task-specific controllers, or trained machine-learning models. Here we present an agentic-AI framework that operates the executable part of the AFM workflow using a general-purpose, tool-augmented large language model connected to instrument functions through the Model Context Protocol (MCP). The framework consists of 3 MCP-based agents: AFM Messenger converts natural-language instructions into checked instrument commands; AFM Pilot assesses image quality through a large language model (LLM) and, if necessary, adapts imaging parameters; and AFM Doctor diagnoses image artifacts and applies transparent post-processing from a pre-approved tool set. Because the language model performs image assessment rather than a fixed scalar objective or external optimizer, the same strategy can be applied across sample types and imaging modes without specific retraining. Safe hardware operation is enforced through an ambiguity check layer before execution. Benchmarking against fine-tuned and off-the-shelf tool-using models shows that this guarded execution layer, rather than model capability alone, reduces wrong-command execution to zero. In live experiments on different samples, AFM Pilot matched expert operators in image quality, iteration count, and tuning time, with no significant difference. These results demonstrate a safe route to agentic operation of scientific instruments, where experimental intent remains human-defined while command execution, image-based tuning, and post-processing are delegated to AI agents.

1. Introduction

The introduction frames AFM as a demanding, continuously adaptive workflow and presents an MCP-based agentic-AI system that delegates command execution, image-based tuning, and artifact handling while retaining human-defined intent.

  • Motivation: AFM image quality requires continuous adjustment as tip, sample, and feedback conditions change, while diverse artifacts demand different corrective actions and trade-offs.Relevant artifacts include parachuting, feedback ringing, hysteresis, creep, double-tip, and loss of surface tracking.
  • Method: MCP externalizes tool definitions, parameters, unit requirements, and clarification rules, enabling reuse across models without retraining.The tool descriptions can be prepared in hours and reused when models or instrument software change.
  • Contribution: The proposed system uses an LLM to translate natural-language instructions into instrument actions, assess AFM images and experimental data, and adjust parameters without fixed scalar optimization.The approach uses multimodal vision-language capabilities for image assessment rather than a conventional control model or fixed scalar metric.
  • Contribution: Three cooperating agents divide the workflow among hardware communication, closed-loop imaging optimization, and artifact diagnosis with transparent post-processing.AFM Messenger, AFM Pilot, and AFM Doctor provide these respective functions.
  • Evaluation and safety: The work benchmarks the execution layer, tests live agentic imaging, and compares operation with expert practice while addressing risks from incorrect or ambiguous commands.The execution layer checks each request before acting because errors could damage the probe, sample, or instrument.

2. Results

The agentic AFM framework delegates executable instrument operation to AI while retaining experimental intent with the human operator. Results show that guarded tool access enabled reliable command execution, image-driven tuning comparable to experts, and transparent artifact correction across samples and imaging modes.

  • Framework: The framework preserves human control over experimental intent while assigning command execution, image assessment, parameter tuning, and post-processing to the agentic system.This division targets instrument operation rather than experimental goal selection.
  • Command reliability: 25.1 ± 1.1% was the error rate for untuned Claude Sonnet 4.6 with structured tool access, compared with 28.4 ± 1.5% for fine-tuned GPT and 23.3 ± 0.7% with function calling.Models made different types of errors: unit or value errors comprised 1.8% for Sonnet with tools, while ambiguity or missing information comprised 64.9%.
  • Command reliability: 0 incorrect command executions were observed with the final interactive MCP configuration, which combined structured tool access, ambiguity checking, and interactive clarification.The deployed MCP workflow connected models to 129 structured tools and withheld under-specified requests until missing information was provided.
  • AFM Pilot: Across five matched pairs, human operators and AFM Pilot showed no statistically significant differences in iterations, tuning time, parachuting severity, or ringing severity.The reported p-values were 0.063, 0.625, 1.00, and 0.188, respectively; AFM Pilot also maintained feedback stability while improving tracking across calibration, biological, and KPFM samples.
  • AFM Doctor: AFM Doctor used AI vision to diagnose scars, height offsets, and background tilt, then selected corrections from a pre-approved set of transparent processing tools.It explained likely physical causes and applied artifact-removal processing to produce an accepted image.

3. Discussion

The framework unifies safe command execution, image-based tuning, and post-processing through three MCP-based agents, moving LLMs from workflow coordination toward direct AFM operation. Its guarded, interactive architecture supports reliable human-supervised use while leaving experimental intent human-defined and exposing limits in timing, parameter scope, and available imaging channels.

  • Framework contribution: The framework combines AFM Messenger, AFM Pilot, and AFM Doctor to execute safe commands, tune parameters from images, and post-process AFM data without task-specific training.This unifies workflow components usually treated separately through MCP-based agents.
  • Framework contribution: AFM Pilot assesses height, error, scan-line information, and artifact severity together, identifies likely imaging problems, and selects bounded parameter changes using general scientific and visual knowledge.The approach replaces fixed objectives or external optimizers with LLM-based measurement assessment and action selection.
  • Safe execution: Structured tool access reduced command errors, while the ambiguity check layer prevented execution of under-specified prompts and physically inappropriate values.The discussion emphasizes that syntactically valid commands can still be unsafe for physical instruments.
  • Human-supervised operation: Interactive clarification keeps humans in the loop, while specialized tool sets, agent selection, and session history support coordinated operation during longer experiments.Operators can provide missing information when requests are ambiguous, and prior commands and decisions remain available as context.
  • Human-supervised operation: AFM Pilot is positioned as an expert assistant for non-specialists rather than a replacement for experienced operators, particularly in research groups using AFM as a supporting technique.The comparison with human experts suggests practical assistance where users need reliable imaging without deep expertise in parameter tuning.
  • Limitations and future directions: The current system tunes only setpoint and integral gain, incurs variable inference and data-transfer costs, and uses mainly height and error channels, while broader autonomous experiment selection remains future work.Extensions require carefully defined parameter interactions and safe ranges, local deployment, additional imaging channels, and integration with active-learning or hypothesis-driven methods; experimental intent remains human-defined.

4. Conclusion

A general-purpose, tool-augmented LLM can operate the executable part of an AFM workflow without AFM-specific retraining. Three MCP-based agents translate instructions into validated commands, adjust imaging parameters, and apply transparent post-processing using pre-approved tools.

  • Framework: The framework operates the executable AFM workflow with a general-purpose, tool-augmented LLM without AFM-specific retraining.This capability is implemented through structured tools and three MCP-based agents.
  • AFM Messenger: AFM Messenger translates natural-language instructions into validated instrument commands.Its role is command translation within the executable workflow.
  • AFM Pilot: AFM Pilot evaluates acquired images and selects bounded parameter adjustments.The agent performs image-based assessment and constrained tuning.
  • AFM Doctor: AFM Doctor applies transparent post-processing from a pre-approved tool set.Post-processing is constrained to the approved tools and remains transparent.

5. Methods · 5.1 Instrument and software interface · 5.2 Data preparation

The methods used a custom open-hardware AFM controlled through the RePySPM Python API and prepared a screened command-to-function dataset from real and synthetic scenarios. The dataset was partitioned into fixed training, validation, and test sets for model evaluation.

  • 5.1 Instrument and software interface: Experiments used a custom open-hardware AFM platform controlled through RePySPM, an open-source Python API communicating with Python-based microscope software.RePySPM provided high-level access to z-control, scan parameters, scan control, motors, signals, lasers, image acquisition, and utility operations.
  • 5.1 Instrument and software interface: RePySPM organized instrument functions into modules covering control, configuration, acquisition, and utility operations.The modules included z-control, scan-parameter configuration, scan control, motors, signals, lasers, image acquisition, and utility operations.
  • 5.2 Data preparation: An experienced AFM user authored a command list of 234 prompts/commands and paired natural-language instructions with corresponding RePySPM Python function calls.The pairings connected operator language to executable instrument functions.
  • 5.2 Data preparation: An experienced AFM user screened all scenarios to create a dataset of 2747 scenarios containing prompts and expected answers.The datasets were provided in a GitHub repository.
  • 5.2 Data preparation: The dataset combined real operator sessions with synthetic examples covering rarely used instrument functions.Synthetic examples supplemented real sessions for less frequently exercised functions.
  • 5.2 Data preparation: The dataset was randomly divided into training (76%), validation (8%), and test sets (two sets of 8%).Three fixed datasets were used for all tests in all models, including dataset 1 with 225 test cases and dataset 2 with 200 randomly shuffled scenarios sampled from dataset 1 using a fixed random s…

5.3 Fine-tuned text-generation system · 5.4 AFM Messenger for safe command execution and ambiguity checking

Sections 5.3–5.4 describe a fine-tuned text-generation baseline and AFM Messenger, an MCP-based agent that translates natural-language instructions into instrument commands with ambiguity checks before execution. The system uses structured tool discovery and evaluates command correctness against expected calls or selections across complete and underspecified commands.

  • 5.3 Fine-tuned text-generation system: The fine-tuned model was based on gpt-4.1-2025-04-14, using batch size =3, epochs=3, and LR multiplier= 2.Training examples used short system, user, and assistant exchanges.
  • 5.3 Fine-tuned text-generation system: During inference, the fine-tuned model generated Python code as free text at temperature 0, without schema constraints or output validation.It received the same system prompt used for the fine-tuned setup.
  • 5.3 Fine-tuned text-generation system: FT-GPT-FC evaluated the same fine-tuned model with function calling, without further training or the interactive ambiguity check.Holding the model constant isolated the contribution of structured tool selection and exposed errors targeted by the ambiguity layer.
  • 5.3 Fine-tuned text-generation system: A Flask chatbot with an HTML front end connected user commands to the GPT model and executed generated code.This lightweight interface operationalized the text-generation system for command handling.
  • 5.4 AFM Messenger for safe command execution and ambiguity checking: AFM Messenger converted each public RePySPM API method into an MCP tool listing its module, function, parameters, and unit information.A discovery script stored the resulting tools in a JSON registry loaded through FastMCP57; the registry could be regenerated after RePySPM changes.
  • 5.4 AFM Messenger for safe command execution and ambiguity checking: The ambiguity check distinguished SET and GET operations, routed shared parameters to the correct module, enforced unit conversions, and separated configuration from scan execution.Missing values, uncertain units, unclear axes, or ambiguous targets were returned to the operator before any tool was invoked.
  • 5.4 AFM Messenger for safe command execution and ambiguity checking: Benchmarking used three datasets containing complete and underspecified commands, with correctness assessed by exact match at temperature 0.Failures were manually classified into unit or value errors, ambiguity or missing information, extra unintended action, wrong tool or module, and other errors.

5.5 AFM Pilot for closed-loop imaging

AFM Pilot performs closed-loop AFM parameter tuning by combining AI-vision assessment of tracking and feedback artifacts with operator-defined safety constraints. A separate deterministic detector independently cross-checks artifact severity without controlling parameter updates.

  • Closed-loop assessment: AFM Pilot retrieves forward/backward height and error data, renders diagnostic panels with instrument state, and uses AI vision to assess tracking and feedback stability.The assessment scores six imaging problems, including parachuting and feedback ringing.
  • Closed-loop assessment: Operator-defined scan-rate limits, minimum setpoint, and parameter-priority order constrain tuning until tracking and feedback reach an acceptable state.Observed artifact severity and height/error data indicate whether further tuning is required.
  • Independent verification: A Python detector independently computes severity scores between 0 and 1 for three artifact classes from forward/backward height and error arrays.It provides a quantitative cross-check of AI-vision assessment but does not participate in tuning or determine parameter updates.

5.6 AFM Doctor for artifact diagnosis and post-processing

AFM Doctor uses a guarded, vision-assessed workflow to diagnose AFM image artifacts and apply transparent corrections from a fixed, pre-approved tool set. It can re-inspect processed images and uses background-region masking to avoid bias from sample topography.

  • Approved processing: The agent’s approved tools include plane leveling, row alignment, scar removal, cropping, background subtraction, and file export.The fixed tool set limits processing to predefined operations.
  • Artifact diagnosis: AFM Doctor diagnoses artifacts from rendered images, explains likely physical causes, and selects corrections from an approved processing tool set.Each processing step is rendered for AI-vision assessment before the agent chooses a correction.
  • Guarded correction: Restricting AFM Doctor to transparent tools reduces unsupported artifact introduction, while re-inspection determines whether further approved corrections are needed.Human reference processing was performed in Gwyddion open-source software.
  • SPM correction operations: Established SPM operations include line flattening, background leveling, scar removal, masked substrate-only leveling, multi-point leveling, FFT destriping, baseline zeroing, and color-range setting.Corrections derive line offsets and polynomial backgrounds only from masked flat-background regions, not sample topography.

5.6 Human-operator comparison · 5.7 Statistical Analysis

AFM Pilot was compared with five human operators under matched detuned conditions, shared adjustable parameters, safety bounds, and image-quality measures. Statistical analysis used GraphPad Prism with Python, reporting mean ± standard deviation and testing group differences by one-way ANOVA with Tukey post hoc comparisons.

  • 5.6 Human-operator comparison: Five human operators and AFM Pilot began from matched detuned conditions for expert comparison.The comparison used the same adjustable parameters, safety bounds, and image-quality measures.
  • 5.6 Human-operator comparison: The comparison used identical adjustable parameters, safety bounds, and image-quality measures for humans and AFM Pilot.
  • 5.6 Human-operator comparison: Parachuting severity scores of approximately 0.30–0.35 were considered acceptable by experienced AFM users.The threshold was based on visual assessment.
  • 5.6 Human-operator comparison: Ringing severity scores of approximately 0.20–0.25 were considered acceptable by experienced AFM users.These ranges indicated acceptable image quality for the expert comparison.
  • 5.6 Human-operator comparison: For AFM Pilot, reaching the acceptable severity ranges indicated that further tuning was not required unless another condition applied.The supplied passage truncates the condition following “unless another.”
  • 5.7 Statistical Analysis: Data analysis used GraphPad Prism and an accompanying Python analysis layer.SPM visualization and processing with human were performed using Gwyddion.
  • 5.7 Statistical Analysis: Results were reported as mean ± standard deviation unless otherwise stated.
  • 5.7 Statistical Analysis: Group differences were assessed using one-way ANOVA followed by Tukey post hoc multiple-comparison tests.

Supplementary Data

Supplementary experiments show AFM Pilot tuning across collagen fibrils and irregular salt residues, while a guarded MCP rule set enforces clarification outside the model. A representative session documents gain-limited ringing, improved tracking with lower setpoint, and residual image advisories.

  • AFM Pilot across samples: AFM Pilot applied its tuning strategy to collagen fibrils and heterogeneous salt residues, including non-periodic and irregular surface features.The collagen experiment began from a deliberately detuned state; a local instability remained in the final image and could not be removed by either operator or agent.
  • Guarded execution: MCP disambiguation rules request clarification for SET/GET confusion, module ambiguity, unit uncertainty, and missing values before execution.The rules are enforced outside the model, so they apply identically across model backbones without retraining.
  • Representative session: The final operating point used setpoint 0.22 V and i_gain 0.015, just below the ringing ceiling, with residual parachuting limited to 2–4 px rail touches at steep feature edges.This was described as approximately the floor for the tip/sample without pushing into ringing.
  • Advisories and limitations: The only flagged advisory was a cosmetic ~300–500 nm background ramp from creep or tilt, while numpy hysteresis detections were not confirmed by AI on the patterned sample.Recommended handling was plane-fit or tilt correction before exporting or quantifying the image; the hysteresis discrepancy was attributed to a known numpy false-positive.
  • Representative session: At setpoints 0.27, 0.25, and 0.22, frame-wide ringing began near i_gain 0.016–0.022, indicating a gain-limited ceiling that lowering setpoint did not substantially raise.Dropping 0.27→0.22 reduced parachuting 0.55→0.35 and cleared ringing at fixed gain, while the gain ceiling stayed put.
  • Representative session: After the prompt rewrite, AI detected frame-wide ringing at iteration 4 and milder line-correlated ringing at iterations 6 and 9, matching the operator’s assessment.AI vision drove decisions, while numpy served only as a guide; numpy parachuting varied 0.00–0.60 frame-to-frame compared with AI’s steadier 0.35–0.55.

2. Iteration Table

The run progressed from nontracking to stable collagen-fibril tracking through setpoint adjustments, while AI distinguished genuine banding from ringing and cleaned slight parachuting by increasing integral gain. Operator assessment agreed that the resulting image was good.

  • Initial engagement: Initial manual settings were free amp ~2.1 V and setpoint 90%, but the AI reported NOT TRACKING in free air on a blank region.The run began with operator-set parameters before automated adjustments.
  • Tracking recovery: 1.65 produced TRACKING after lower setpoint values, resolving a large collagen fibril with real banding.At 1.75, the scan still showed a smooth ~700 nm bow with Error railed; setpoint was reduced before tracking succeeded.
  • Artifact assessment: i_gain ×3 was applied twice while AI judged the observed wiggles and periodic banding to be genuine structure rather than ringing.The AI specifically assessed the banding as real rather than an artifact.
  • Final tuning: 0.45 ringing was reported after re-engagement, while the operator judged the image good with slight parachuting; i_gain increased 0.009→0.011 and parachuting became clean.The gain adjustment trimmed the remaining parachuting without changing the reported image-quality assessment.

3. Advisory Section

The AI misclassified genuine collagen D-band banding as feedback ringing despite naming it correctly, while the operator confirmed the image was good. It consequently prescribed reducing i_gain, a mistake not prevented by the sample-context note.

  • AI assessment: 0.45 frame-wide ringing score at i_gain 0.009 and 0.011 contrasted with numpy’s 0.00–0.06 throughout.The AI assigned the higher score at both tested i_gain values, whereas numpy remained in the stated range.
  • Operator validation: The operator confirmed that periodic Error-signal banding was genuine collagen Dband structure rather than feedback ringing, and judged the image good.The live-screen confirmation directly contradicted the AI’s artifact interpretation.
  • Failure mode: Although the AI named the banding correctly, it still scored the periodic signal as ringing and prescribed cutting i_gain.The sample-context note did not prevent the mis-score or the resulting parameter recommendation.
Loading 2608.26198v1…