PaperlayerView exact revision on arXiv

Source-linked AI summary

Agents for self-driving laboratories applied to quantum computing

Shuxiang Cao, Zijian Zhang, Mohammed Alghadeer, Simone D Fasciati, Michele Piscitelli, Mustafa Bakr, Peter Leek, Alán Aspuru-Guzik

arXiv:2412.07978v2cs.AIquant-ph

TL;DR

Laboratory calibration requires substantial knowledge to set parameters, interpret results, and manage experiments as superconducting systems grow more complex. k-agents organizes laboratory knowledge in LLM-based agents and uses an execution agent to coordinate staged experiments. On a superconducting quantum processor, it demonstrated gate calibration and GHZ-state generation and characterization, while retaining limits on interruptible execution.

  • Problem

    Superconducting-circuit calibration requires scientist customization, validation, monitoring, and adjustment, and drifting parameters can make recalibration time-consuming and repetitive.

    Source: §II · p. 6

  • Method

    k-agents uses LLM-based knowledge agents and an execution agent to coordinate laboratory knowledge and automate experiments.

    Source: §I · p. 2

  • Results

    k-agents demonstrated single- and two-qubit gate calibration and GHZ-state generation and characterization on a quantum processor, orchestrating experiments with multimodal interpretation and closed-loop feedback.

    Source: §III · p. 9

  • Takeaways & Limitations

    The authors suggest k-agents is a valuable tool for research groups working with superconducting quantum processors.

    Source: §III · p. 9

    The current execution agents do not support fine-grained, interruptible execution; external signals or emergencies can terminate only the entire program.

    Source: §III · p. 10

Abstract

from arXiv · show

Fully automated self-driving laboratories are promising to enable high-throughput and large-scale scientific discovery by reducing repetitive labour. However, effective automation requires deep integration of laboratory knowledge, which is often unstructured, multimodal, and difficult to incorporate into current AI systems. This paper introduces the k-agents framework, designed to support experimentalists in organizing laboratory knowledge and automating experiments with agents. Our framework employs large language model-based agents to encapsulate laboratory knowledge including available laboratory operations and methods for analyzing experiment results. To automate experiments, we introduce execution agents that break multi-step experimental procedures into agent-based state machines, interact with other agents to execute each step and analyze the experiment results. The analyzed results are then utilized to drive state transitions, enabling closed-loop feedback control. To demonstrate its capabilities, we applied the agents to calibrate and operate a superconducting quantum processor, where they autonomously planned and executed experiments for hours, successfully producing and characterizing entangled quantum states at the level achieved by human scientists. Our knowledge-based agent system opens up new possibilities for managing laboratory knowledge and accelerating scientific discovery.

I. MAIN

Laboratory automation is difficult because it depends on complex, multimodal knowledge that is costly to encode and maintain. The paper introduces k-agents, which coordinates knowledge and execution agents to automate experiments, and demonstrates the framework on superconducting quantum processors.

Motivation

  • Laboratory automation requires detailed knowledge to set experiment parameters, interpret results, and execute workflows, while growing apparatus complexity increases the effort of translating and maintaining that knowledge.Laboratory knowledge is also multimodal and complex, challenging automated programs to perform at a human-comparable level.Source: §I · p. 1

Framework

  • k-agents uses LLM-based knowledge agents to hold laboratory operations, procedures, and methods for inspecting experiment results, without fine-tuning.The agents are selectively activated to support scaling as their number grows.Source: §I · p. 2

Translation benchmark

  • On a test set of 80 instructions from eight experiments, k-agents outperformed standard RAG in selecting the correct experiment class and supported heterogeneous translation agents working together.The comparison used 17 code translation agents.Source: Figure 2 · p. 3

Execution

  • Its execution agent decomposes multi-step procedures into agent-based state machines, using inspection reports and natural-language transition rules to guide closed-loop progress.Unlike rigid deterministic rules, agents determine state transitions; this approach minimizes the experimental history loaded into LLMs.Source: Figure 1 · p. 2, §I · p. 3

Demonstration

  • k-agents demonstrated automated single- and two-qubit gate calibration and GHZ-state generation and characterization on a superconducting quantum processor.The system orchestrated experiments, interpreted multimodal results, and guided execution with closed-loop feedback.Source: §I · pp. 2–3

A. Knowledge agents

Knowledge agents receive and transfer laboratory knowledge, reducing reliance on prompt histories or manual tool organization. k-agents supports them with indexed procedural memory and separate agents for translating instructions and inspecting results.

Knowledge agents

  • Knowledge agents are AI agents whose performance is measured by their ability to receive and transfer knowledge.The paper describes a strong agent as accepting natural-language knowledge and answering related queries correctly in natural language.Source: A. Knowledge agents · p. 3

Knowledge transfer

  • Transferring knowledge in natural language facilitates more rigorous testing of agents and avoids treating the system as a black box.The paper also links this transferability to improved trustworthiness of AI systems.Source: A. Knowledge agents · p. 4

Related work

  • Prior systems often embed instructions and entire toolsets in prompts, limiting scalability for complex toolsets and long-horizon tasks as input length grows.The cited approaches are generally limited to tasks whose execution context fits within the prompt window.Source: A. Knowledge agents · p. 4

Knowledge architecture

  • k-agents uses a scalable memory architecture and agent-based retrieval to access indexed procedural knowledge and experiments, with agents reflecting on and selecting retrieved information.The design supports more complex and long-duration tasks.Source: A. Knowledge agents · p. 4

Translation agents

  • Code translation agents apply documented experiment interfaces to convert natural-language instructions into code, while procedure translation agents use stored examples to generate code for multistep workflows.A laboratory “experiment” can be an instrument action, tool call, or another agent; procedure code calls an execution agent.Source: A. Knowledge agents · p. 4

Inspection agents

  • Inspection agents use prompts and optional example figures to analyze experiment plots with multimodal LLMs and return text-based reports.Users attach the inspection knowledge through decorators on functions that produce figures.Source: A. Knowledge agents · p. 4

B. Execution agent

The execution agent turns procedures into staged workflows, translating and running each instruction before using inspection reports to choose the next step. The paper also illustrates visual-inspection benchmarks and a single-qubit recalibration workflow.

Visual inspection

  • Providing example figures significantly improved visual-inspection accuracy, and combining reports from multiple inspection agents improved accuracy in some cases.The benchmark compared zero-shot visual agents, few-shot visual agents, fitting agents, and a combined setup for determining experiment success.Source: Figure 3 · p. 5

Execution agent

  • The execution agent decomposes a procedure into stages, each holding one experiment or sub-procedure and a natural-language rule for choosing the next stage.The next stage can depend on the previous experiment’s result.Source: B. Execution agent · p. 5
  • At each stage, the execution agent activates context-relevant translation agents, runs selected code, gathers inspection reports, and uses reports and transition rules to select and update the next stage.This stage-by-stage design avoids monitoring the entire experimental history when deciding what to do next.Source: B. Execution agent · p. 5

Single-qubit recalibration

  • In single-qubit recalibration, k-agents repeated a Ramsey experiment with increased experiment time after finding that the first run lacked enough oscillations to estimate frequency.Subsequent experiments passed the agents’ success criteria before randomized benchmarking characterized gate fidelity.Source: Figure 4 · p. 6

II. RESULTS

The k-agents framework was evaluated on superconducting-processor calibration and experiments, from single-qubit benchmarking to two-qubit gate discovery and GHZ-state tomography. The agents achieved 97% code-translation accuracy with GPT-4o and autonomously identified working gate parameters and carried out tomography from natural-language instructions.

II. RESULTS

  • 97% accuracy was achieved by GPT-4o in code translation, and inspection agents performed best when constructed with example figures.Code-generation failures could be retried with prompts.Source: §II · p. 7
  • The agent calibrated a single qubit from deliberately perturbed initial settings, transitioned between calibration and benchmarking, and validated fidelity through randomized benchmarking.It adjusted frequency, amplitude, and DRAG parameters, retrying failed steps and reverting to an earlier stage after repeated failures.Source: §II · p. 7
  • The agent autonomously identified working parameters for a siZZle gate, whose simultaneous off-resonant drives alter the ZZ interaction between fixed-frequency qubits.Pulse duration was then fine-tuned programmatically by measuring the ZZ interaction rate.Source: §II · p. 7
  • Two LLM-based agents used empirical parameter-selection knowledge and experiment history to suggest frequency and amplitude settings for successive tests.The search considered regions below, between, and above the qubits’ transition frequencies.Source: §II · p. 7
  • After a 3-hour search capped at 100 experiments and 20 frequencies, the agent identified parameters of frequency 4726MHz and amplitude 0.3049.Figure 5 depicts the search space, accepted parameter locations, and failed attempts.Source: §II · pp. 8–9, Figure 5 · p. 8
  • On a 16-qubit processor’s adjacent three-qubit subset, the agent generated a GHZ state from natural-language instructions, performed tomography, and reported state fidelity.The workflow generated a density-matrix plot and a natural-language report.Source: §II · pp. 7, 9, Figure 6 · p. 9

III. DISCUSSION

The authors report that k-agents automated quantum-gate calibration and GHZ-state characterization with closed-loop feedback, and outperformed standard RAG in natural-language instruction translation accuracy. They also discuss practical strengths and limitations, including documentation requirements, latency, interruptibility, and safety scope.

III. DISCUSSION

  • K-agents calibrated single- and two-qubit gates and generated and characterized a GHZ state on a quantum processor, while outperforming standard RAG in translation accuracy.The authors also report that multimodal LLMs analyzed experiment figures, especially when given example figures.Source: §III · p. 9
  • Unlike methods that tune specific parameters, k-agents targets entire experimental procedures using multimodal prompts and LLMs.The authors contrast this with optimization approaches requiring manual tuning and crafted reward functions, and with rigid rule-based systems.Source: §III · p. 9
  • The framework is designed to work with routine laboratory documentation, though initial onboarding and training remain necessary.The authors compare setup effort to preparing manuals or protocols for new laboratory staff.Source: §III · pp. 9–10
  • K-agents differs from ChemCrow and Voyager by targeting long-horizon quantum-laboratory workflows and building on existing laboratory codebases.The authors describe ChemCrow as focused on relatively straightforward, short-horizon chemistry tasks and Voyager as relying on extensive Minecraft-related training data and simulated environments.Source: §III · p. 10
  • Full automation depends on clean codebases and structured procedure documents, while runtime LLM planning and translation add latency compared with traditional automation.Caching repeated LLM responses can mitigate the latency.Source: §III · p. 10
  • Execution agents lack fine-grained interruption, and the low risk of retrying failed code on superconducting processors may not apply to other applications.The authors identify interactive monitoring and intervention as beneficial in quantum systems and safety control as future work.Source: §III · p. 10
  • During a three-hour two-qubit gate parameter search, LLM use cost less than 5 US dollars, and overall efficiency was reported as comparable to humans.The run used 1,373,207 input tokens and 168,039 output tokens; inspections took seconds per result, while code generation was more efficient than human code generation.Source: §III · p. 10

1. Definition of knowledge agents

The paper defines knowledge agents by their ability to acquire knowledge and apply it by transferring knowledge to other agents. It regards bare LLMs as weaker agents for laboratory knowledge than agents designed to accept laboratory documentation.

1. Definition of knowledge agents

  • A knowledge agent’s learning ability means internalizing knowledge from related inputs.The paper bases this concept on Zhang et al. [16].Source: §1 · p. 10
  • Knowledge agents acquire knowledge and apply it by transferring that knowledge to other agents.Source: §1 · p. 10
  • The paper treats bare LLMs as weak knowledge agents because laboratory documents and code are difficult for them to internalize.The proposed agents are distinguished by their ability to accept knowledge from laboratory documentation.Source: §1 · p. 10

2. Translation Agents

Translation agents convert instructions into code or procedures, activating agents based on instruction similarity. The system uses analysis and self-reflection to screen code candidates and limits generalization to translations consistent with agent knowledge.

2. Translation Agents

  • K-agents uses code translation agents for executable code and procedure translation agents for procedures that may contain multiple instructions.Source: §2 · p. 11
  • Code translation agents are built from experiment classes, while procedure translation agents are built from structured procedure documents.A document translation agent can generate code that invokes an execution agent to run a procedure.Source: §2 · p. 11
  • Translation agents are selectively activated using embedding similarity between incoming instructions and instructions associated with each agent.The agent score is the maximal inner product between the incoming instruction embedding and the agent’s instruction embeddings.Source: §2 · p. 11
  • For code translation, an LLM analyzes the instruction, checks whether the experiment class fits, and generates a candidate snippet only when appropriate.The analysis and self-reflection steps are intended to improve accuracy and avoid hallucinations.Source: §2 · p. 11
  • Procedure translation agents adapt a matching procedure title to the instruction and submit code that calls an execution agent.Source: §2 · p. 11
  • Both translation types generalize by creating code or instructions from inputs, while rejecting translations that deviate too far from agent knowledge.The authors say this controlled generalizability mitigates unsuitable knowledge from public corpora in private experimental settings.Source: §2 · p. 11

B. Details on execution agents

Execution agents decompose procedures into stages with transition rules, translate and run each stage, then use inspection reports to choose the next state. Translation candidates can be gathered from selected agents and synthesized before execution.

B. Details on execution agents

  • The execution agent decomposes a procedure into single-experiment stages and attaches rules for selecting the next stage from each stage’s results.This agent-based state machine includes initial, COMPLETE, and FAILED states.Source: B. Details on execution agents · pp. 11–12
  • Stage generation separates independent experiment instructions from progression-control sentences, then assigns indices and transition rules.The workflow adds COMPLETE and FAILED stages.Source: B. Details on execution agents · pp. 11–12
  • For each stage, the execution agent activates translation agents ranked by instruction-match score and gathers their code candidates.Algorithm 1 initializes the agent set and activation limits, then scores agents and selects the top-ranked candidates.Source: B. Details on execution agents · p. 12
  • If no code candidate is produced, the algorithm increases the number of activated agents by two until reaching the maximum; otherwise, an LLM synthesizes the final code.The synthesis uses the instruction and candidate code as context.Source: B. Details on execution agents · p. 12
  • After running translated code, inspection agents summarize experiment results, and the execution agent uses the report and transition rule to select the next stage.Execution continues until COMPLETE or FAILED, after which the agent summarizes the experiment history.Source: B. Details on execution agents · p. 12
  • Users can program a variable table to pass variables from one experiment to other experiments.Source: B. Details on execution agents · p. 12

C. Quantum processor hardware

The processor hardware was a 16-qubit coaxmon square lattice operated through LeeQ and QubiC; 17 built-in experiments were indexed for k-agents.

C. Quantum processor hardware

CODE AVAILABLILITY

This section describes how k-agents encodes experiment and procedure knowledge and uses agents to translate and execute laboratory instructions. It also presents benchmarks of code translation and of judging experimental success from fitting reports, plots, or both, using synthetic quantum-computing results.

CODE AVAILABLILITY

1. Representing knowledge of available experiments

  • Experiment classes document laboratory actions and provide visual or text inspection agents whose reports execution agents summarize to choose the next stage.Visual inspection prompts encode domain-specific figure-analysis knowledge; text inspection supports analyses such as fitting.Source: §1 · pp. 16–17

2. Representing knowledge of experiment procedures

  • Laboratory procedures use a structured Markdown format with Background, Steps, and Results sections, where Steps is required and the other sections inform agents’ interpretation and reporting.Procedure translation agents match incoming instructions to a procedure title and adapt its Steps; Results knowledge supports execution reports.Source: §2 · pp. 17–18

1. Code translation agent

  • Code translation agents retrieve experiment classes using embedding similarity, generate candidate code, and let execution agents select a candidate for the instruction.The Appendix describes prompts for generating possible instructions and code candidates from experiment documentation.Source: §1 · pp. 18, 21

2. Procedure translation agent

  • Procedure translation agents match instructions to procedure titles, rewrite them without changing parameter specifications, map available variables to required inputs, and wrap the result as an experiment for execution.The wrapped experiment is sent to a new execution agent as a code candidate.Source: §2 · p. 19

3. Execution agent (Stage generation)

  • Execution agents generate stages by extracting standalone instructions from procedure descriptions, then generate transition rules and store instruction numbers as key-value maps for later parameter changes.Stage extraction is intended to preserve the description without expanding it or including what to do next.Source: §3 · pp. 19–20

4. Execution agent (Stage transition)

  • Execution agents choose the next stage using inspection-report summaries, current-stage execution history, and transition rules, with a retry limit to prevent endless loops.They can also modify variables in the next stage when inspection suggests parameter changes.Source: §4 · p. 20

1. Benchmarking translation

  • The translation benchmark uses 80 experiment–instruction pairs and 17 experiment classes to compare agent-based candidate selection with standard RAG using two retrieved classes.Correctness checks whether the code calls the correct experiment with a valid call format; input values are not verified. The same text-embedding-3-large model is used to control embedding choice across LLMs.Source: §1 · pp. 20–21

2. Comparison with long-prompt apporach

  • The long-input benchmark duplicated descriptions of 17 experiments in random order to create inputs of about 11,000 tokens, and activated nine translation agents per instruction.Source: §2 · p. 21
  • Agent-based translation with a subset of agents outperformed long-context counterparts in some cases, and activating more agents could improve accuracy over prompting all tools (Figure 7).GPT-4o and Gemini 1.5 Pro achieved high translation accuracy with long inputs, but Claude 3 Opus declined notably.Source: §2 · p. 22, Figure 7 · p. 22

3. Benchmarking visual inspection

  • The benchmark compared fitting-report-only, figure-only, and combined inspection, using few-shot examples of successful and failed plots to guide visual judgments.Source: §3 · p. 22
  • Inspection agents were tested on four common quantum-computing plot types, with 100 successful and 100 failed synthetic examples per experiment, including cases difficult to judge from fitting results alone.The Gaussian-mixture readout cases included small leakage or substantial population in higher excited states, complicating SNR-based assessment and natural-language success criteria.Source: §3 · pp. 22, 24
  • Because LLaVA 1.6 cannot include multiple images in a conversation, its visual-inspection performance was demonstrated only with zero-shot prompting.Source: §3 · pp. 25–26

Appendix D: Built-in experiments

The appendix illustrates built-in calibration experiments and a siZZle-gate workflow that uses agent inspection to assess results and guide parameter tuning. Hamiltonian and repeated gate tomography provide distinct ways to characterize the ZZ interaction and calibrate the entangling gate.

Appendix D: Built-in experiments

  • For Ramsey qubit Q1, the measured oscillation was 0.102+/-0.000 MHz versus the expected 0.100 MHz, and 3.562+/-0.011 oscillations were observed, within the acceptable range (greater than 3 and less than 10).Source: Appendix D · pp. 26–27
  • Rabi oscillations estimate a rough single-qubit pulse amplitude, while repeated gate sequences in pingpong calibration refine that amplitude until it stabilizes around iteration five.The Rabi example is judged successful with about 5.1 oscillations; the pingpong example reports amplitude convergence and successful fine calibration.Source: Appendix D · pp. 27–28
  • The DRAG example identifies an optimal coefficient of -0.0061 within the central half of the sweep and meets the experiment’s success criteria.The fitted value is -0.0061392677674084704; the analysis also reports acceptable residuals with some deviations, particularly among red data points.Source: Appendix D · pp. 29–30
  • Randomized benchmarking estimates infidelity per Clifford at 0.00240+/- 0.00023 and infidelity per gate at 0.00131+/-0.00013.The example’s analysis reports a clear exponential decay in the |0⟩ population and judges the experiment successful.Source: Appendix D · pp. 30–31

Appendix E: Two-qubit siZZle gate

  • Manual siZZle calibration iterates over drive frequency and amplitude using Hamiltonian tomography and repeated gate tomography; once dynamics appear stable, code determines the gate duration.Figure 8 distinguishes continuous Stark-pulse-width variation for Hamiltonian tomography from discrete pulses for repeated gate tomography, which more precisely calibrates accumulated ZZ interaction per block.Source: Appendix E · p. 31, Figure 8 · p. 32
  • Hamiltonian tomography produced a fitted ZZ interaction strength of 0.16543846027754651 MHz, with clear symmetric Fourier peaks and a stable control-qubit expectation.The inspection agent judged the measurement successful based on the Fourier plot and fit.Source: Appendix E · p. 35
  • Repeated gate tomography yielded a fitted ZZ interaction strength of -0.1170390305626309 MHz, with clear symmetric Fourier peaks and stable control-qubit expectation.The inspection agent judged the experiment successful because the ground- and excited-state peak difference exceeded 25% of its maximum.Source: Appendix E · p. 36

Appendix F: Experimental Hardware and Setup

The experiments use three selected qubits from a 16-qubit superconducting processor arranged in a four-by-four lattice. The setup operates near 15 mK, with microwave control and resonator readout through a dilution-refrigerator hardware chain.

Appendix F: Experimental Hardware and Setup

  • The selected qubits are part of a 16-qubit, four-by-four transmon processor, with the three qubits used in this work identified on the chip image.The qubits are statically coupled through capacitive elements.Source: Appendix F · p. 36
  • Table I characterizes the three selected qubits using resonator internal Q factor, external decay rate, and randomized-benchmarking single-qubit gate fidelity.The fidelity measure uses Clifford gates decomposed into physical X and Y gates.Source: Table I · p. 36
  • The qubits operate in a 3He/4He dilution refrigerator at approximately 15 mK, with control pulses synthesized by QubiC and reflected readout signals amplified, down-converted, and digitized.ADCs connected to an FPGA capture the readout signals for measurement and further analysis.Source: Appendix F · p. 37

Related papers

Loading 2412.07978v2…