Source-linked AI summary
Application of curiosity driven exploration methods for hardware interference identification
Ludovic Matar, Clement Moulin-Frier, Pierre-Yves Oudeyer
TL;DR
Multi-core interference complicates execution-time assurance and comprehensive source identification because complex micro-architectural interactions produce difficult-to-cover behaviors. The paper applies curiosity-driven exploration to a simulator-based dual-core model, using behavior-space coverage to guide experiments. The approach achieves higher behavioral diversity than randomly generated programs, while remaining limited by aggregated observations and a simplified behavior space.
Problem
Existing interference-analysis approaches have limited ability to cover multi-core behavior spaces, whose complex micro-architectural interactions are sensitive to small program variations.
Method
The paper uses curiosity-driven automated discovery with a multi-core simulator to explore interference behavior spaces through performance-counter-based goals.
Results
The proposed exploration achieves higher behavioral diversity and better discovery of possible behaviors than randomly generated programs in a simplified simulated dual-core architecture.
Takeaways & Limitations
Automated discovery can efficiently explore a simulated hardware behavior space and supports investigating non-trivial interference patterns.
Takeaways & Limitations
The behavior space uses aggregated, non-internal information, limiting characterization of interference sources, and its performance can decrease in high dimensions.
Abstract
from arXiv · showhide
The transition from single-core to multi-core architectures in safety-critical embedded systems introduces significant challenges due to inter-core interference caused by contention for shared hardware resources. Such interference affects execution times and complicates the verification of strict temporal requirements, particularly in domains such as avionics where standards require comprehensive identification of interference sources. Existing interference analysis approaches, whether manual or model-based, struggle to capture the full range of behaviors arising from the complex interactions among micro-architectural components. In this paper, we frame multi-core interference analysis as the exploration of a complex system behavior space. We propose the use of curiosity-driven exploration algorithms from artificial intelligence to systematically and efficiently cover the space of possible interference behaviors. Using a simulator-based environment, we show that the proposed approach achieves broader and more uniform behavioral coverage within a limited experimental budget compared to traditional pseudo-random program generation methods.
1 Introduction
Multi-core systems create interference through contention for shared hardware resources, complicating execution-time assurance and interference-source identification. The paper frames this challenge as complex-system behavior exploration and applies curiosity-driven algorithms to improve coverage over pseudo-random generation.
- Motivation: Multi-core architectures introduce inter-core interference when parallel tasks compete for shared resources, affecting execution time and strict temporal assurance.In avionics, AMC 20-193 requires identifying and addressing all interference sources during parallel execution.
- Interference sources: Interference can arise from cache, memory, bus, and I/O contention among cores or processes.These mechanisms can evict data, limit bandwidth, or delay transfers and operations.
- Limitations of existing approaches: Existing manual, empirical, and model-based approaches remain limited in covering the full behavior space of multi-core processor systems.Complex non-linear interactions among micro-architectural components make behavior sensitive to small program variations, while pseudo-random generation is unlikely to cover the entire space.
- Proposed approach: Curiosity-driven exploration algorithms are proposed to efficiently cover complex-system behavior spaces and discover interference patterns in multicore processors.The approach uses a multi-core processor simulator to avoid the complexity of real hardware during prototyping.
- Result: The proposed method achieves better discovery of possible system behaviors than pseudo-random program generation under a limited experimental budget.The paper reports this comparison as its main introductory result.
2 Related works
Related work applies machine learning to execution-time estimation and automated interference generation, but the paper targets a different objective: discovering interference sources and patterns through automated exploration.
- Machine-learning approaches: Machine-learning studies have estimated worst-case execution times for both single-core and multi-core platforms.These studies address execution-time prediction rather than the paper’s specific source-discovery objective.
- Automated interference generation: Kryptonite synthesizes a maximally interfering environment for a program under test by arranging resource-targeting gadgets.It combines greedy gadget selection with reinforcement learning to increase the program’s execution time.
- Relation to this work: The paper adapts the broader idea of automated discovery on multi-core platforms to explore interference patterns rather than optimize one interfering environment.The related approach motivates repeated system interaction, although its objective differs from source discovery.
3 Description of our dual-core model
The paper models interference with a simplified dual-core simulator containing private L1 caches, shared memory components, and timing-sensitive cache and DDR mechanisms. Programs issue scheduled memory operations whose interactions determine delays.
- System architecture: The simulator contains two cores with private L1 caches, a shared L2 cache, DDR memory with a controller, and an interconnect.These shared components provide the resources on which interference can arise.
- Core and program model: Core execution is simplified to memory-access instructions, with RD and WR operations issued at scheduled cycles.Programs are represented as cycle-to-operation-and-address mappings, while non-memory instructions are abstracted away.
- Address and program representation: The simulator maps virtual addresses to physical DDR locations and models programs as executable pairs that do not exchange data.The representation focuses on instruction-driven interference rather than application data values.
- Cache model: Cache behavior is configurable by size, line size, associativity, and write policy, while the shared L2 cache has no coherence mechanism.Eviction is managed through a binary-tree structure that selects cache rows based on memory accesses.
- DDR and controller model: DDR banks use row buffers and timed state machines, making same-bank row changes and simultaneous intrabank accesses more delaying than interbank accesses.The memory controller reorders requests through command, read-data, and write-data queues with prioritization rules.
4 Overview of the exploration strategy
The exploration strategy represents simplified assembly-program pairs as inputs and interference-related performance-counter outcomes as behaviors, then uses IMGEP to pursue diverse target behaviors. It iteratively reuses prior experiments to sample goals, select promising programs, and expand coverage of the behavior space.
- Exploration overview: IMGEP treats the simulated hardware platform as a complex system whose input pairs of independent programs map to interference behaviors.Each experiment executes both programs in parallel and separately, with system state cleared between executions.
- Exploration overview: The discovery agent samples target behaviors and updates goal-generation and goal-achievement models using an experiment database H.The database stores previous independent experiments and supports reuse during subsequent exploration.
- Parameter space: The parameter space Θ consists of pairs of simplified assembly programs, using modeled memory instructions and distinct address sets for the two cores.The programs are constrained to a reasonable cardinality so the exploration results remain interpretable.
- Behavior space: The behavior space B is built from performance-counter differences between non-isolated and isolated executions, capturing resource conflicts associated with interference.Counters include clock cycles, row misses, instruction types, branch mispredictions, and stalls.
- Behavior space: For nbanks banks and nrows rows, the behavior-space dimension is n = 3 + 4 · 2 + 2 · 2 · nbanks · nrows, yielding B ⊂ R59 when nrows = 3 and nbanks = 4.The Euclidean representation permits distance calculations, but large dimensions can reduce exploration performance through the curse of dimensionality.
- Goal sampling: Curiosity is implemented by sampling goals in a region extending beyond already discovered outcomes, while boundary estimates are periodically updated from H.This targets nearby unexplored regions and uses uniform sampling between per-feature minima and maxima.
5 Results
The evaluation measures behavioral diversity and interference coverage produced by IMGEP in a simulated dual-core system. IMGEP achieves broader diversity and identifies more interference cases than random or operator-combination exploration, while aggregated observations limit source characterization.
- Experimental comparison: IMGEP is compared with random exploration and operator-combination exploration using shared warming data and the same mixing and mutation operators.The comparison tests whether IMGEP’s diversity advantage comes only from the operators.
- Diversity evaluation: The study evaluates dataset diversity using filled bins in multidimensional behavior spaces, including execution time, hit/miss outcomes, and 59-feature outcomes.The diversity measure counts non-empty histogram bins across selected outcome dimensions.
- Diversity evaluation: For every tested k, IMGEP produces higher diversity than operator-combination exploration across miss ratios, the full behavior space, L2 and DDR hit/miss spaces, and execution-time goals.The diversity measurement also increases faster for IMGEP on the full behavior space and the specified goal subspaces.
- Interference coverage: IMGEP produces interference in about 7 of 10 cases, compared with about 4 of 10 for randomly generated programs.Interference is identified when absolute non-isolation and isolation execution times differ.
- Interference coverage: The IMGEP behavior cloud spreads across both core-specific execution-time differences, indicating more observed interference cases than the comparison exploration.Any nonzero coordinate represents interference on the corresponding core.
- Analysis and limitations: Aggregated exploration data limits direct characterization of interference sources, although the results suggest that repeated requests to shared DDR locations yield more varied delays.The analysis also states that matching access counts alone are insufficient to characterize interference.
6 Conclusion
The work applies automated discovery algorithms to simulated SoC interactions, using a behavior and parameter space to explore interference-related behaviors. Results show successful interaction with a simplified dual-core model and higher behavioral diversity than random program exploration, while future work targets distinct interference mechanisms.
- 6 Conclusion: The method uses simple machine-learning models to structure an autonomous agent interacting with SoCs through automated discovery algorithms.The approach establishes both behavior and parameter spaces for exploration.
- 6 Conclusion: Automated discovery algorithms successfully interact with a simplified simulated dual-core architecture, supporting exploration of a real SoC.
- 6 Conclusion: The behavior space, represented by performance counters, achieves higher diversity with IMGEP than with randomly generated programs.
- 6 Conclusion: Future work will target distinct interference mechanisms rather than maximizing diversity without identifying why interferences occur.The planned objective is to maximize the number of distinct interference types.
A Description of our dual-core model
This section presents a table selecting the simulator parameters used in the experiments.
- A Description of our dual-core model: Table 4 lists the simulator parameters selected for the experiments.
- A Description of our dual-core model: The table concerns parameters of the simulator used to evaluate the dual-core model.
- A Description of our dual-core model: The listed parameters define the experimental configuration for the dual-core model.
A.1 Complementary information on the DDR and its controller
The simplified DDR2 model and its controller jointly process memory operations cycle by cycle. Controller scheduling accounts for request state, latency constraints, prioritization, and callbacks for completed reads.
- A.1 Complementary information on the DDR and its controller: The simplified DDR2 model implements ACT, PRE, and REF operations for memory-row activation, deactivation, and cell refreshment.
- A.1 Complementary information on the DDR and its controller: The DDR and memory controller operate in coupled cycle-by-cycle execution.
- A.1 Complementary information on the DDR and its controller: The controller completes due requests, signals completed RD accesses through callbacks, and uses callbacks to permit subsequent core instructions.
- A.1 Complementary information on the DDR and its controller: For queued requests, the controller checks DDR executability and minimum-delay constraints before prioritizing row hits, WR over RD, and FIFO arrival order.
- A.1 Complementary information on the DDR and its controller: The model is inspired by previously described mechanisms.
A.2 Interconnect model
The interconnect model represents memory-request handling with a FIFO queue, latency and bandwidth constraints, and deferred processing for requests that cannot be served immediately. Core and interconnect clocks are synchronized by default.
- A.2 Interconnect model: The interconnect maintains a FIFO queue and executes requests in arrival order after minimum access latency, with a random two-cycle delay.
- A.2 Interconnect model: Each clock cycle, executable queued requests are selected according to priority and limited by bus bandwidth.
- A.2 Interconnect model: Requests that cannot be processed are moved to a queue for treatment in subsequent cycles.
- A.2 Interconnect model: The interconnect and cores use the same clock frequency by default, although distinct clocks can be introduced in the code.
B Intrinsically Motivated Goal Exploration Process
The exploration uses parameterized IMGEP settings and operators that mix and mutate simplified assembly instruction sequences. Mutation changes instruction content or sequence structure while respecting available cycles and instruction limits.
- Parameterization and representation: The experiments use selected IMGEP parameters to configure exploration and mix sequences represented as cycle-to-(type, address) dictionaries.The parameter selection is documented in Table 5, while the sequence representation supports operator-based exploration.
- Mutation operator: The mutation operator modifies instruction sequences by adding, deleting, or modifying instructions within specified mutation, cycle, address, and length constraints.It copies the input sequence, tracks unused cycles, and applies a bounded number of mutations.
- Mutation operator: Mutation selects among add, delete, and modify operations when deletion is possible, otherwise choosing between add and modify.An addition uses an available cycle and assigns a new instruction type and address.
- Mutation operator: Instruction modification can change the access type, address, or both, while deletion releases the removed cycle for reuse.Access types switch between read and write; address changes are sampled within the configured range.
B.1 Diversity Evaluation
Diversity is evaluated with histogram occupancy, while pairwise squared Euclidean distances provide intuition but are not a formally additive diversity measure. The latter increases for clouds with larger perimeters and pairwise separations.
- Pairwise-distance measure: The pairwise-distance measure sums squared Euclidean distances between all outcome pairs to approximate the spread of an outcome cloud.Small clouds with short pairwise distances yield smaller values, whereas larger cloud perimeters yield larger values.
- Pairwise-distance measure: This pairwise-distance measure is not additive over disjoint sets because D(H1∪H2) ≠ D(H1)+D(H2) in general.The non-additivity makes it ill-defined as a positive diversity measure, despite its interpretive usefulness.