Source-linked AI summary
Proton Irradiation Characterization of an Open-Source ML Accelerator on a Zynq UltraScale+ MPSoC
Saad Memon, Rafal Graczyk, Jan Swakoń, Leszek Grzanka, Sebastian Kusyk, Mike Papadakis
TL;DR
Commercial accelerator opacity limits verifiable radiation-mitigation work, motivating an empirical baseline for an open-source Tensil accelerator. The paper performs a monitored proton campaign on a Zynq UltraScale+ system running ResNet-20 and observes both workload interruptions and silent output corruption. Its results show that execution liveness and nominal timing do not guarantee correct inference, while field-based associations remain non-localized under confounded exposure conditions.
Problem
Few empirical radiation-response baselines exist for open-source RTL-accessible neural-network accelerators, limiting evidence for verifiable mitigation strategies.
Method
The study irradiated an unmitigated open-source Tensil accelerator on a Zynq UltraScale+ SoC while monitoring Linux-managed ResNet-20 inference and endpoint-specific recovery behavior.
Results
Two output-corruption events returned incorrect classes while execution continued, including 39 consecutive absent-class outputs at normal cadence; seven other events interrupted the workload.
Takeaways & Limitations
Liveness and nominal timing do not establish inference correctness, so Linux-managed accelerators require end-to-end content-aware checks and recovery reaching persistent corrupted state.
Takeaways & Limitations
The cross sections apply only to this hardware, workload, and protocol, while field size, run order, and dose prevent isolating individual components or attributing a fault location.
Abstract
from arXiv · showhide
As spaceborne computing systems increasingly rely on neural network (NN) accelerators, the opacity of commercial, black-box architectures severely restricts the development of verifiable radiation mitigation strategies. Open-source, register-transfer level (RTL)-accessible accelerators resolve this limitation by enabling user-defined instrumentation, yet few have empirical radiation-response baselines. This work establishes a foundational system-level proton-irradiation baseline for an unmitigated open-source Tensil NN accelerator deployed on a Zynq UltraScale+ SoC executing ResNet-20 inference. Under 20 to 58 MeV proton irradiation, we delivered $4.29 \times 10^{10}$ p/cm$^{2}$ within monitored operational windows. Seven workload interruptions required two restarts of the notebook process, four reboots or board resets, and one power-cycle sequence. Two output-corruption events returned incorrect CIFAR-10 classes without loss of service. In the longer event, the accelerator returned a class absent from the ten-image CIFAR-10 pool for 39 consecutive inputs at normal cadence. The process remained alive, while the kernel log, limited memory test, and sampled power showed no anomaly. Observation of the stuck-class sequence ended with scheduled bitstream reconfiguration. All nine onsets occurred under the nominal 4 cm beam, which exposed the SoC, LPDDR4, and additional board circuitry; none occurred under the 2 cm SoC-centered field. This pattern shows a field association but does not establish LPDDR4 as the cause because field size was confounded with run order and dose. Linux-managed accelerators require end-to-end content checks and recovery that reaches the state in which corruption can persist. This baseline documents availability loss and silent output corruption, supporting future software hardening of COTS FPGA-SoCs for neural-network inference in space systems.
I. INTRODUCTION
The paper frames radiation response as a system-level problem spanning Linux software, programmable logic, and external memory, then establishes monitored proton-irradiation evidence for both service interruptions and silent wrong outputs.
- System-level motivation: Every inference crosses the Linux processing system, programmable logic accelerator, and external DRAM, so radiation can disturb any part of the stack.A stopped service and a running service that returns a wrong answer are operationally distinct failure modes.
- Research gap: Prior Linux-managed campaigns reported crashes and corrupted inferences but did not report corrupted-state duration or monitor behavior during events.The paper positions these omissions as gaps in system-level reporting.
- Experimental approach: The campaign ran an open-source Tensil TCU executing ResNet-20 on ten CIFAR-10 images in 100-inference blocks, reloading the PL at block boundaries.The study retained predictions in execution order and monitored progress, kernel logs, reachability, power, and a userspace memory test.
- Key observation: One block produced 61 correct predictions followed by 39 consecutive bird predictions, despite nominal availability indicators and a scheduled reload boundary.Bird was absent from the ten-image input pool, and outputs arrived at the usual cadence.
- Contributions and scope: The paper reports seven Linux-level interruptions, silent constant-class corruption, endpoint-specific cross sections, and a matched-energy field comparison confounded by run order and dose.It does not identify a physical mechanism or evaluate a mitigation.
- Reporting practices: The reporting recommendations are to count events by onset, match denominators to endpoint monitors, and report recovery without inferring fault origin.These practices address persistent corrupted states and monitor blind spots.
B. Attribution Under Whole-System Exposure
Whole-system proton exposure makes component attribution difficult because the SoC, LPDDR4, interfaces, memory, and board circuitry participate together in each inference. The platform and implementation details define the specific system-level baseline rather than an isolated device result.
- Attribution boundary: A wide beam exposes the SoC, memory package, traces, clocks, and power-delivery circuitry simultaneously, so workload observations cannot identify the initiating component.The study therefore uses field geometry only to test association at matched energies.
- Hardware platform: The DUT was an Avnet Ultra96-V2 with a Zynq UltraScale+ XCZU3EG MPSoC and adjacent 2 GB LPDDR4 without ECC.The board also used a microSD card for the boot image and root filesystem.
- Irradiation setup: The heat sink was removed, the SoC faced approximately normal to the beam, and the board operated in room-temperature air with 8.000 V supplied input.These conditions describe the physical irradiation setup.
- Software and monitoring: PYNQ Linux ran on the processing system while a laptop controlled the board through USB-gadget Ethernet and collected outputs and kernel messages.Because all Linux evidence traversed that link, service loss alone has limited diagnostic meaning.
- Accelerator architecture: The Tensil TCU used a 16×16 systolic array at 100 MHz, with FP16BP8 data paths and external LPDDR4 transfers through AXI4 and DMA interfaces.Each inference exercised software, interfaces, external memory, DMA, configuration state, and accelerator-local state.
- Implementation baseline: The routed design met timing with +0.523 ns worst-case slack and used 24,382 lookup tables, 18,907 registers, 180 block-RAM tiles, and 273 DSP48E2 blocks.Vivado marked 42.8% of configuration-bit positions as essential, but the paper did not use that count for normalization.
B. Inputs, Oracle, and Observability
The experiment used a fixed, validated ten-image CIFAR-10 pool and a class-only oracle while varying proton energy and nominal beam field. These choices define what inputs, outputs, and exposure conditions the campaign could observe.
- Inputs: The input pool comprised CIFAR-10 test-set indices 10–19, spanning six classes, with automobile, bird, cat, and deer absent.All ten images were classified correctly before irradiation, making their labels the online reference.
- Sampling and records: Each inference sampled one image uniformly with replacement, while the realized order was not reproducible from the sampling rule alone.The log retained each selected input index, reference class, and predicted class.
- Oracle and observability: The checker retained only the predicted class, defined by the largest output score, so it could detect class changes but not within-class numerical perturbations.It did not retain complete score vectors, intermediate tensors, or accelerator state.
- Beam energies: The campaign used nominal 20-, 40-, and 58-MeV proton settings, reporting incident energy because component-specific energy was not measured.Facility fluence and flux uncertainty was reported as 3%.
- Beam fields: The nominal 2-cm field primarily covered the SoC, while the nominal 4-cm field covered the SoC, LPDDR4, and additional board area.Both fields were used at 20 and 40 MeV; only the wider field was used at 58 MeV.
D. Block Protocol and Recovery
Each notebook block combined memory testing, monitored inference, output retention, and PL reconfiguration, while recovery escalated from process restart through power cycling after sustained progress loss. Event clustering and endpoint-specific accounting separated workload interruptions from output corruption and incorporated explicit exposure, uncertainty, and statistical limitations.
- Block protocol: Each block ran a 10-MB userspace memory test, 100 logged inferences, class-summary retention, and a final PL configuration download.The inference sequence took about 2.34 s, with outputs typically 23–24 ms apart; the memory test typically took 15–20 s.
- Monitoring: Operators monitored block summaries, terminal status, kernel messages, network reachability, and sampled power, with continuous supply current available only for run 1.Other runs contained two power-rail samples per block.
- Recovery: Recovery escalated from process restart to Linux reboot, board reset, and power cycle after sustained loss of progress.When multiple actions changed together or observation ended, the analysis reported a lower bound or full sequence rather than assigning causality to one action.
- Event definitions: An event onset began after a normal state or recovery action, while consecutive abnormal outputs, logs, or relaunch failures formed one cluster until restart, reboot, reset, or power cycle.Persistent corruption therefore counted once regardless of affected predictions, with clustering performed separately for each endpoint.
- Event definitions: Linux-SEFI classified workload-preventing F8/F9 onsets as observed symptoms requiring at least a process restart, not identified locations or mechanisms.Its cross sections are upper bounds on radiation-induced rates because interruptions may have any origin.
- Endpoints: Output and Linux-SEFI onsets were analyzed separately, while corrected-OCM reports remained telemetry rather than workload failures.Every onset began while the beam was on, so the authors called them radiation-associated under a temporal criterion that does not exclude beam-independent causes.
- Exposure accounting: Output-event exposure used an allocation approximation because classification evidence was retained for only 62 of 92 launched blocks.The approximation does not assert that verified and unverified blocks shared the same latent event history.
- Statistical analysis: Cross sections used clustered onset counts and endpoint-specific denominators, with condition-specific units because exposed area differed between fields.Nonzero counts used two-sided 95% Garwood intervals under a Poisson model, while zero counts used one-sided 95% upper limits.
A. Exposure and Data Completeness
The campaign delivered 6.19 × 10^10 p/cm2 overall, with 4.29 × 10^10 p/cm2 inside reconstructed monitored operational windows. Verification was incomplete: 92 blocks launched, 83 completed, but class records were retained for only 62 blocks.
- Exposure: 6.19 × 10^10 p/cm2 was delivered across the monitored runs, including exposure outside reconstructed operational windows.Reconstructed operational windows accounted for 4.29 × 10^10 p/cm2; the remainder occurred before monitoring, after monitoring, or during recovery.
- Exposure: 92 blocks launched and 83 completed, yielding 8300 inferences.Nine launched blocks did not complete, although failed relaunches within one cluster were not counted as separate Linux-SEFI onsets.
- Data completeness: Class records were retained for 6200 inferences across 62 blocks, so output verification was incomplete.Retention was highest at 20 MeV and thinnest at the energies where output events occurred.
- Data completeness: The monitoring record included workload progress and availability or telemetry indicators, but these indicators did not examine accelerator content.A content oracle was not used for the listed availability and telemetry indicators.
B. Silent Output Corruption
The campaign observed sustained silent output corruption: after 61 correct predictions, 39 changing inputs produced the same absent-from-pool class at normal cadence. Availability monitors remained nominal until scheduled reconfiguration ended observation, while another output event had incomplete ordering evidence.
- Observed corruption: 61 predictions were correct, followed by 39 consecutive bird outputs, although no bird image existed in the ten-image pool.The constant class differed from the last correct output and from the first-maximum argmax of an all-equal score vector, while persisting across changing inputs.
- Observed corruption: The corrupted state became effective between inferences 61 and 62 within one 23–24 ms inference interval.The transition occurred after model load and PL configuration, in state not refreshed between inferences.
- Monitor response: Every availability indicator remained nominal during the 39 incorrect outputs, including process liveness, cadence, kernel log, memory test, and sampled power.A watchdog using only these signals would not have detected the corruption.
- Recovery and censoring: The error persisted for about 0.9 s until scheduled PL reload, which rewrote configuration, reinitialized PL memories, and reloaded the model.The protocol, rather than the fault, right-censored the episode at 39 predictions.
- Other output event: The second output event comprised three incorrect predictions in a completed block, but output order was not retained, supporting one to three possible onsets.The primary analysis used one onset, while the event was carried with uncertainty.
C. Availability Loss: Linux-SEFIs, Recovery, and Telemetry
Seven Linux-SEFIs spanned three recovery levels, from process restart through reboot or board reset to power cycle, while telemetry and memory records constrained but did not localize the underlying faults. The campaign also recorded corrected on-chip-memory errors and incomplete event attribution.
- Linux-SEFIs and recovery: Seven Linux-SEFIs spanned three recovery levels, including process restart, reboot or board reset, and power cycle.Two F8 events cleared with process restart while Linux remained reachable; the taxonomy labels describe observed behavior, not fault locations.
- Linux-SEFIs and recovery: One reboot-tier event was documented by a null-pointer kernel oops, but a software-only cause could not be excluded for another termination.The photographed trace passed through the clone/fork path while ipython was active.
- Counting and denominators: The reported cross sections used clustered operational onsets and endpoint-specific denominators, with Garwood intervals for nonzero counts and one-sided upper limits for zero events.Recovery-censored or multi-action episodes were reported as lower bounds or complete sequences.
- Endpoint comparison: The output-event endpoint did not distinguish the two fields because the small-field upper limit exceeded the wide-field estimate.The output-event cross sections used verified-block fluence rather than the operational-window denominator.
- Telemetry: The campaign reported corrected-OCM telemetry under both beam fields, including repeated reports at address 0xFFFEA4A0 in run 5.The repeated reports formed one grouped episode despite appearing across nine blocks and one reboot.
- Telemetry: 97 complete memtester records contained 1746 pattern results with no mismatch, but they did not map one-to-one onto the 92 blocks.The record could not distinguish a persistent cell from repeated upsets in the affected OCM region.
D. Cross Sections
Wide-field Linux-SEFI exposure produced a pooled point estimate, but the run 5 count and estimate varied with alternative interpretations of ambiguous recovery records. These results therefore remain sensitive to event attribution.
- Linux-SEFI cross sections: Seven wide-field Linux-SEFI onsets over 1.93 × 10^10 p/cm2 yielded 3.6 × 10^-10 cm2/system.The estimate had a two-sided interval of 1.5 × 10^-10 to 7.5 × 10^-10 cm2/system.
- Operational availability: Small-field runs maintained workload operation for 91–93% of the beam interval, compared with 75–76% for runs 3 and 5 and 21% for run 4.The small-field shortfall reflected beam time outside the block sequence rather than downtime.
- Sensitivity to attribution: Alternative interpretations reduced the 58-MeV run 5 Linux-SEFI count from four to three and its point estimate from 4.7 × 10^-10 to 3.5 × 10^-10 cm2/system.Applying the alternatives together left one or two possible events because the archive could not resolve the sequence.
E. Field Contrast
The matched-energy field comparison found all observed onsets in the wider field, but timing uncertainty and recovery interpretation substantially weaken the apparent contrast. The persistent output corruption also shows that reset-on-error protocols can miss corrupted state before recovery.
- Matched-energy field test: 0.016 was the primary one-sided exact p-value for all three matched-energy onsets occurring in the wider field.With only three events, this was the smallest attainable value; alternative readings and timing bounds produced p values from 0.016 to 0.46.
- Sensitivity: p ranges from 0.016 to 0.46 when timing bounds and alternative interpretations of run 4 recovery are considered.The run 4 wide-field window lasted about four minutes, with a ±150 s timing bound equal to 62% of its fluence.
- Sensitivity: [1.25, ∞) was the primary two-sided exact 95% interval for the wide-to-small rate ratio, decreasing to [0.47, ∞) under the second run 4 onset merge.Crediting all run 4 exposure outside the operational window produced p values of 0.066 and 0.10 under the reported alternatives.
- Persistent corruption: 39 consecutive wrong-class outputs persisted at usual cadence before scheduled reload, revealing behavior that reset-on-error protocols do not observe.The process remained operational, and availability indicators did not flag the event; the reload ended observation without establishing whether it cleared the condition.
- Monitoring implications: Content checks are needed alongside availability monitors because the top-1 oracle can miss corruption and the stock deployment had no content checker.A class-histogram check would have detected the absent class at its first occurrence, but the proposed measures were not evaluated.
B. Persistent State and Recovery Scope
The campaign shows that radiation-related observations can persist across monitored blocks and that recovery actions refresh different portions of the inference stack. Field-associated event counts support a system-level observation, not component attribution or susceptibility ranking.
- Persistent state: A corrected OCM address recurred across nine blocks spanning a reboot, but ECC corrected every read before software received the value.The record does not link these corrected-error reports to the output corruption.
- Recovery scope: A process restart leaves Linux, drivers, PS state, external memory, and PL configuration unchanged, whereas a Linux reboot clears PL configuration and reruns boot firmware.In this protocol, subsequent block launches also re-downloaded the PL and reloaded the model, refreshing more state than the recovery action alone.
- Recovery scope: Recovery should escalate according to the state that may remain corrupted, from process supervision to model reload or PL reconfiguration and further escalation if needed.This guidance follows from the refresh analysis rather than a comparison of recovery methods; no recovery strategies were tested against each other.
- Field association: All nine primary onsets occurred under the wider field, but field was confounded with run order and accumulated dose.The wider field received 45.0% of operational exposure, while no Linux-SEFI occurred in 44 completed small-field blocks and no output event occurred in 31 verified blocks.
- Interpretation boundary: The observed pattern does not identify the responsible circuitry, and counts are not susceptibility rankings across differing spectra, devices, workloads, monitors, clustering rules, recoveries, and denominators.The selected-campaign comparison summarizes differing experimental conditions rather than a common ranking basis.
D. Limitations and Threats to Validity
The findings are bounded to one hardware–software configuration and cannot localize the internal propagation path or establish component causation. The campaign nevertheless demonstrates that liveness and nominal timing do not establish inference correctness and motivates content-aware monitoring and staged recovery.
- Scope: The cross sections apply only to an Ultra96-V2 Tensil accelerator running ResNet-20 on ten fixed CIFAR-10 images.They do not quantify device-to-device variation, susceptibility across neural-network models, CIFAR-10 accuracy under irradiation, or flight qualification.
- Scope: The energy-stratified cross sections do not establish an energy dependence, and the field comparison remains exploratory.The two beam configurations did not isolate individual components.
- Instrumentation limits: The experiment retained neither internal propagation data nor local-console evidence sufficient to reconstruct output-event paths or distinguish notebook-service loss from USB-link loss.Unretained signals included logits, intermediate tensors, model checksums, DMA transactions, physical LPDDR4 addresses, PL readback, and a local console.
- Observed behavior: The longer output event produced 39 identical absent classes while availability monitors remained nominal, and scheduled reconfiguration did not establish whether the underlying condition cleared.The next launch failed, and the corrected OCM reports were not linked to output corruption.
- Causal limits: All nine event onsets occurred under the wider field, but field size was confounded with run order and accumulated dose, preventing localization or identification of LPDDR4 as the source.The result supports an association with the wider field, not component-level attribution.
- Implications: Liveness and nominal timing do not establish inference correctness, so future systems need end-to-end content checks and staged recovery that can refresh persistent corrupted state.The paper also recommends onset-based event counting, endpoint-matched denominators, explicit monitor blind spots, randomized field order, local-console access, and longer runs without scheduled reloads.