Machine learning for long-distance quantum communication

I. INTRODUCTION

• Projective-simulation agents rediscover teleportation, entanglement swapping, entanglement purification, and quantum repeaters, and outperform symmetric human-designed strategies in asymmetric settings.The rediscovery is guided by the challenge design, unlike the original human achievement of conceiving the protocols.Source: §I · pp. 1–2
• Previously learned protocols become block-action primitives, while sub-agents optimize shorter-distance actions for larger-scale use.Separately learning block actions for different initial fidelities supports asymmetric situations.Source: §I · p. 2
• The agent must react to stochastic measurement outcomes and sometimes abort and restart, as in entanglement purification.The paper reports that projective simulation can handle these probabilistic tasks.Source: §I · p. 2

II. PROJECTIVE SIMULATION FOR QUANTUM COMMUNICATION TASKS

• The paper frames protocol design as trial-and-error reinforcement learning: operations are actions, protocol states are perceived, and task-specific rewards guide learning.Protocol search resembles navigation through a network of states connected by operations.Source: §II · p. 2
• Projective simulation makes decisions through deliberation in an adjustable episodic and compositional memory network.The paper describes this as a physically motivated learning-and-decision framework.Source: §II · p. 3

III. LEARNING ELEMENTARY PROTOCOLS

• A successful protocol is a sequence of actions that reaches the task goal during a trial.Trials end unsuccessfully if the action limit is exceeded or no actions remain.Source: §III · p. 4

1. Basic protocol

• For teleportation, the agent must transmit an unknown state without directly sending the quantum system, using shared entanglement and local gates.The task challenges it to find a shortest operation sequence among exponentially many possibilities as sequence length grows.Source: §1 · pp. 4–5
• The task uses a sender-held input qubit and a shared Bell pair, and tests success using Jamiołkowski fidelity to require correct action for all input states.The available gate sets are Clifford or universal, with single-qubit Z-basis measurements allowed.Source: §1 · p. 5
• The agent finds the standard teleportation protocol, using measurement outcomes to select among correction operations of different lengths.The learned protocol is identical to the well-known quantum teleportation protocol.Source: §1 · p. 5
• The validated teleportation channel can also perform entanglement swapping when the input qubit is entangled with another system.The agent's different correction sequences reflect the stochastic measurement outcomes.Source: §1 · p. 5

2. Variants without pre-distributed entanglement

• When the input may be operated on immediately, the agent finds one-bit teleportation, using one provided qubit and a single measurement-dependent Pauli-Z correction.The protocol is described as conceptually and resource-wise simpler than standard teleportation.Source: §2 · p. 5
• With no operations on the input allowed before sending a qubit, the agent finds standard teleportation by first creating and distributing a Bell pair.The longer action sequence takes significantly more trials to discover.Source: §2 · pp. 5–6

B. Entanglement purification

• For recurrent purification, the agent must choose success or failure for each measurement branch and reward protocols for both success probability and fidelity increase.The setup has four branches from two single-qubit measurements, and recurrent use requires identifying which outputs to reuse.Source: B. Entanglement purification · p. 6
• The learned purification protocol is equivalent to DEJMPS for an even number of recurrence steps, but uses a shorter action sequence for the provided gate set.The successful protocols begin with bilateral CNOTs and measurements; the full-reward variant applies −iX to both output qubits before the next step.Source: B. Entanglement purification · p. 7

C. Quantum repeater

• Purification alone has a distance limit because accumulated noise can make initial states too poor to purify; repeaters divide channels into segments and purify before swapping.For sufficiently noisy but still purifiable short links, swapping alone can yield a state that cannot be purified.Source: C. Quantum repeater · p. 7
• The agent can use previously learned teleportation, swapping, and purification protocols as elementary actions to distribute an entanglement pair across two channel segments.The environment supplies noisy neighboring pairs and evaluates resource use rather than action count.Source: C. Quantum repeater · pp. 7–8
• For longer repeater settings, a main agent can delegate a subsystem solution to a sub-agent and reuse its action sequence as one block action.The sub-agent can itself delegate smaller subsystems at larger scales.Source: Figure 5 · p. 8
• Purifying short-distance pairs a few times before entanglement swapping is beneficial in the two-link task, despite requiring more agent actions, and matches the established repeater strategy.The task minimizes initial pairs while reaching fidelity at least 0.9 from pairs with initial fidelity 0.75.Source: C. Quantum repeater · pp. 7–8

IV. SCALING QUANTUM REPEATER

• As distance grows, the action space and solution sequences become unmanageable, and ordinary RL learns for a particular problem size rather than a transferable scheme.The repeater's scaling behavior becomes evident only at distances beyond two links.Source: §IV · p. 8
• A single action lets the agent instruct multiple sub-agents to find short-distance entanglement-distribution solutions, choose the best sequence, and reuse solutions as elementary actions in the larger setting.Source: §IV · pp. 8–9
• The scalable-protocol objective is to distribute an entangled pair over long distances with sufficient fidelity while minimizing resource use.The imperfect-memory setting considers an asymmetric length-8 repeater and varies memory decoherence time.Source: §IV · p. 9, Figure 6 · p. 9

A. Symmetric protocols

• For symmetric links with initial fidelity 0.75, the agent finds solutions with desirable scaling beyond eight links using only slightly more resources than repeated purification followed by swapping.The comparison strategy purifies all pairs to a chosen working fidelity before swapping.Source: A. Symmetric protocols · p. 9

B. Asymmetric setup

• For asymmetric link fidelities, the learned solution is preferable to a strategy that ignores the asymmetry.The example uses fidelities (0.8, 0.6, 0.8, 0.8, 0.7, 0.8, 0.8, 0.6) and CNOT reliability p = 0.99.Source: B. Asymmetric setup · p. 9

C. Imperfect memories

• Purification outcomes must be communicated before the output pair can be reused, so both qubits are stored for t_epp = l/c; swapping by-products can be accounted for retroactively.The latter follows because the protocols use Clifford operations and purification communication spans a longer distance than swapping communication.Source: C. Imperfect memories · p. 10
• With imperfect memories, required resources increase sharply at decoherence time τ = 1/30 s, and the agent finds no protocol at τ = 1/31 s.The authors note this may reflect reaching a threshold or needing more actions than the agent could find in a few thousand trials; the demanding requirement is tied to this setup's low starting fidelities.Source: C. Imperfect memories · pp. 9–10

D. Choosing location of repeater stations

• The placement example evaluates combinations of 1, 2, or 3 repeater stations across seven candidate locations for a 20 km asymmetric setup.The agent’s protocol is used to assess the resources required for the best combinations under the specified channel-noise and memory-error model.Source: D. Choosing location of repeater… · p. 10
• The authors present this station-placement analysis as a proof of principle that the agent can help choose locations, with other setups and error models left for future repetitions.The example uses the same length-dependent channel-noise and imperfect-memory model as Section IV C, with τ = 0.1 s.Source: D. Choosing location of repeater… · p. 10

V. SUMMARY AND OUTLOOK

• The authors report that reinforcement learning can retrieve basic protocols and optimize relevant figures of merit when the task environment and reward are suitably structured.They also state that framing the challenges as reinforcement-learning problems offers advantages over optimization techniques.Source: §V · pp. 10–11
• Researchers supply key parts of task formulation by asking the right questions and identifying relevant resources; the agent’s role is assisting exploration once a task is properly formulated.The authors distinguish this researcher input from implementation details that reflect practical constraints such as computational runtime.Source: §V · p. 11
• Beyond rediscovering existing protocols, the agent finds adapted solutions when channel qualities lack symmetries assumed by basic protocols, and delegation helps it handle larger problems.Source: §V · p. 11

Appendix A: Introduction to projective simulation

• Projective simulation uses episodic and compositional memory organized as clips encoding percepts, actions, or sequences, connected in a two-layer network.The first layer represents environment states; edges to action clips carry weights that determine action-selection probabilities.Source: Appendix A · p. 14
• The agent’s decisions form a one-step random walk, and a trial of n interactions is an n-step walk defined by the network weights.The weights are updated after each interaction according to a learning rule.Source: Appendix A · p. 14
• The learning rule distributes reward through an edge trace according to that edge’s contribution to the rewarded action sequence; the trace decays at rate η.A separate damping parameter γ helps the agent forget, which is useful when the environment changes.Source: Appendix A · p. 14

1. Environment description

• In the teleportation environment, the goal is to transfer A′’s state to B, measured by Jamiołkowski fidelity, and reaching the goal gives reward 1 and ends the trial.Source: §1 · p. 14
• The teleportation task provides 10 actions spanning a universal gate set, CNOT, and destructive Z-measurements that remove further operations on the measured qubit.The setup initializes A′ as part of an entangled state for the fidelity test, while A and B begin in a Bell state.Source: §1 · p. 14
• The agent uses previous actions in the current trial as percepts, with different Z-measurement outcomes represented by different percepts.Source: §1 · p. 14

2. Discussion

• Entanglement is a limited resource: an irreversible operation can destroy it and make the goal unreachable, unlike recoverable mistakes in a simple maze.A reset action could keep the goal reachable after a wrong irreversible action, but would require a different class of percepts.Source: §2 · p. 15
• For teleportation, once the common four-action portion of a solution is found, only outcome-dependent correction operations remain to be learned.The shortest solution for one measurement-outcome combination takes four actions, regardless of the required correction operation.Source: §2 · p. 15
• Brute force would require at least 714 > 6.7 × 10^11 action sequences for the most complicated universal-gate-set solution, versus a few hundred thousand agent trials.The passage counts at least 14 actions in that solution and excludes measurement actions from the sequence count.Source: §2 · p. 15

3. Environment variants without pre-distributed entanglement

• In the no-pre-distributed-entanglement variants, the agent transfers an input state to B without sending it directly; one variant restricts initial operations on A′ and allows only one qubit to be sent.In variant 1, the agent found a protocol using fewer actions than standard teleportation by exploiting a task loophole.Source: §3 · p. 15

1. Environment description

• The purification task asks for a protocol that improves Bell-pair fidelity when applied repeatedly to two copies of its prior output.The reward evaluates ten recurrent applications because one step may raise fidelity while later applications undo the gain.Source: §1 · pp. 15–16
• The reward combines accepted-branch success probability with fidelity increase after ten steps, scaled so known protocols receive reward 1.Source: §1 · p. 16
• Purification evaluation tracks separate branches and their probabilities so the agent must choose actions for every measurement outcome and is rewarded for the whole protocol.The next purification step uses the average density matrix of the branches the agent keeps.Source: §1 · p. 16

• The DEJMPS starting state is diagonal in the Bell basis, with coefficients λij describing its components.The procedure applies bilateral CNOTs, measures A2 and B2, and postselects on coinciding results; the success probability is N.Source: §2 · p. 16
• Without intervention, recurrent purification amplifies both λ00 and λ10, so DEJMPS adds iX before the CNOTs and measurements.The iX operation exchanges λ10 and λ11.Source: §2 · p. 16
• Applying the agent’s map twice has the same effect as applying DEJMPS twice, making the protocols equivalent for an even number of recurrence steps.The DEJMPS procedure uses iX on both copies to exchange λ10 and λ11 and prevent unwanted amplification of λ10.Source: §2 · p. 16
• A second agent-found protocol applies H⊗H before each purification step and succeeds, but performs slightly worse.H⊗H exchanges λ10 and λ01.Source: §2 · p. 17

3. Automatic depolarization variant

• The variant depolarizes the state after each purification step while preserving its fidelity.The resulting state is a mixture of |Φ+⟩ and the other three Bell states.Source: §3 · p. 17
• Protocols accepting two relevant measurement branches are equivalent to the entanglement-purification protocol in.Some protocols accept only one branch, halving success probability and reward.Source: §3 · p. 17
• Successful protocols apply bilateral CNOTs and measure qubits A2 and B2 in the computational basis.Optional local operations that do not change fidelity can be undone by the automatic depolarization.Source: §3 · p. 17

1. Environment description

• The repeater environment starts neighboring stations with entangled pairs of fidelity 0.75 and targets an end-to-end pair above Fth = 0.9.Available actions include one-step purification of a pair and swapping at the middle station; the setup uses protocol for computational convenience.Source: §1 · p. 17

1. Environment description

• The agent can delegate smaller problems to sub-agents and apply each found sequence as a block action.As the link count grows, purification and swapping actions, including blocks at different positions, enlarge the action space.Source: §1 · pp. 17–18

a. Symmetric variant

• The symmetric variant restricts the agent to purifying all pairs, swapping at every second active station, and using symmetrically obtained shorter block actions.The available-action restrictions force symmetric protocols.Source: a. Symmetric variant · p. 18

2. Additional results and discussion

• For symmetric initial fidelities of 0.7, the unrestricted agent finds solutions close to, but not equal to, the working-fidelity strategy.For some gate reliabilities it does slightly better by changing operation order or omitting a purification step while still reaching the fidelity threshold.Source: §2 · p. 18, Figure 9 · p. 18
• With initial fidelities (0.95, 0.9, 0.6, 0.9, 0.95, 0.95, 0.9, 0.6), the agent’s advantage over the working-fidelity strategy is more pronounced.This eight-link case has two especially noisy links.Source: §2 · p. 18, Figure 9 · p. 18

3. Repeater stations for setups with memory errors

• For memory-error setups, station distances matter; the 78.9 km scenario uses seven intermediate stations and yields asymmetric initial fidelities (0.8, 0.6, 0.8, 0.8, 0.7, 0.8, 0.8, 0.6).A scaled-down 20 km scenario provides seven possible repeater locations.Source: §3 · pp. 18–19, Table II · p. 19, Table III · p. 19

4. Working-fidelity strategy

• For 2^k links, the working-fidelity strategy swaps at every second active station, halving the number of links at each round.The target fidelity and resource-minimizing choice of Fw are applied through repeated rounds until one pair remains.Source: §4 · p. 19
• The working-fidelity strategy chooses Fw, purifies pairs until F ≥ Fw, swaps at the station with the smallest combined neighboring distance, and repeats until one pair remains.Fw is optimized to minimize resources, and the strategy leads to resource requirements per repeater station that grow logarithmically with distance.Source: §4 · p. 19

Appendix F: Computational Resources

• The calculations ran on four 16-core AMD Opteron 6274 CPUs, and memory requirements were insignificant for all scenarios.Table IV provides scenario runtimes.Source: Appendix F · p. 19, Table IV · p. 20