Source-linked AI summary
Toward Methods and Tools for Critical Examination of Disparate Systems for Discovery of Hidden Dependencies in Engineering of Autonomous and Semi-Autonomous Systems
Assaf Marron, David Harel
TL;DR
Engineers developing autonomous and semi-autonomous systems must apply system-wide considerations, while domain knowledge and interactions may remain unarticulated. The paper proposes adversarial-like examination of videos of disparate systems and reports valuable review points from both AI-assisted and unassisted reviews. Initial results show that repeat AI Skill reviews added points, while human observers sometimes noticed points LLMs missed.
Problem
Developers of autonomous and semi-autonomous systems must account for software, hardware, people, related systems, and physical environments, including when tasks seem encapsulated.
Source: §I · p. 1Method
The methodology uses adversarial-like examination of videos of disparate systems to identify implicit risks, faults, interactions, and dependencies relevant to a target system.
Source: §I · pp. 1–2Results
LLMs reported typically 10 to 20 valuable RPs with and without an AI Skill; repeat reviews using the Skill added RPs, while human observers sometimes reported RPs LLMs missed.
Source: §V · p. 4Takeaways & Limitations
The authors conclude that video critique can provide low-risk practice in system-wide perspectives and may help engineers identify target-relevant issues more quickly.
Source: §VII · p. 5Resource-saving fixed-frequency video-frame sampling caused LLMs to miss some issues, which higher sampling rates at designated intervals detected.
Source: §V · p. 4
Abstract
from arXiv · showhide
We present a methodology, an initial AI Skill package, and emerging results, for enhancing software engineers' acquisition and application of domain knowledge and system-wide engineering perspectives towards the development of an autonomous or semi-autonomous system in a domain that may be new to them. At the core of the approach is a process involving proactive adversarial critiquing of videos of disparate systems in related environments. Among other things, the engineers and automated AI tools look for interactions, dependencies, and environment assumptions manifested in the observed systems, which are applicable also to the target system, but that otherwise might remain unarticulated in project artifacts. The approach and its supporting tools can enhance the quality of the target system, the individual engineer's skills and the organization's engineering practices.
I. INTRODUCTION AND OVERVIEW
Engineers developing autonomous and semi-autonomous systems need system-wide domain knowledge, yet relevant interactions and assumptions may remain unarticulated. The paper proposes critical examination of videos of disparate systems to discover and generalize such considerations for target-system engineering.
I. INTRODUCTION AND OVERVIEW
- The approach complements requirements-engineering learning as engineers must consider software, hardware, people, related systems, and physical environments, even for seemingly encapsulated tasks.The authors also describe diverse domain information as material that modern AI tools can digest and organize.Source: §I · p. 1
- The proposed approach extends domain learning with proactive examination of other systems to uncover interactions and dependencies that may otherwise remain unarticulated.It focuses on systems other than the target and does not claim that examined videos provide sufficient evidence for validating or debugging those systems.Source: §I · pp. 1–2
- Video can expose interactions, be replayed for close inspection, and help observers immerse themselves in the depicted environment.The authors present video examination as complementary to other materials and hands-on learning.Source: §I · pp. 1–2
- Drill-down critical examination (DDCE) uses adversarial-like inspection to identify risks, functional faults, and challenging questions, including ones implicit in a video.DDCE is not reverse engineering and does not involve testing with varied inputs.Source: §I · p. 2
- Comparing reviewers’ review points (RPs) aims to elicit insights from differences in their observations, rather than evaluate their performance on the same videos.RPs can be generalized to identify interactions, dependencies, cause-and-effect relationships, and extreme scenarios, enriching target-system engineering and training.Source: §I · p. 2
II. MOTIVATION
Examples from video examinations illustrate how non-experts can raise system-wide engineering questions. The observations span assumptions, monitoring, actuation, environmental diversity, alternatives, control, and holistic system behavior.
II. MOTIVATION
- Observers questioned tacit design assumptions, noticed a robot’s shaking before it fell, and asked whether actuator actions were confirmed as successful.The examples raise questions about human-like constraints, self-monitoring, and reassessing environmental state after an action.Source: §II · p. 2
- Examiners considered environmental diversity, missing alternatives, control modes, human-robot handover, and why maneuvers were performed sequentially rather than in parallel.Examples include crop and fire conditions for agricultural robots, signaling on a delivery vehicle, and how a robot determines when to release an object.Source: §II · p. 3
III. METHODOLOGY OUTLINE
The methodology applies drill-down critical examination to videos of systems related to, but distinct from, the target. It compares reviews, iterates as useful, and applies relevant findings to engineering work.
III. METHODOLOGY OUTLINE
- Select videos of diverse systems and environments related to the target, designate human and computerized examiners, and document their issues and questions.Examinations may be repeated with and without the specialized AI Skill to combine current domain knowledge with broader, experience-informed review.Source: §III · p. 3
- Compare reviews across reviewers or related systems, explain differences and missed issues, optionally repeat examinations, and apply relevant RPs to target plans and organizational practices.The examined systems’ specifications and intended logic are not required.Source: §III · p. 3
IV. DDCE GUIDELINES FOR ENGINEERS AND AI SKILLS
The guidelines direct human and AI examiners to critically inspect videos without interactively testing or reverse engineering the depicted system. They organize attention around direct observations, relationships, scale, and emerging risks.
IV. DDCE GUIDELINES FOR ENGINEERS AND AI SKILLS
- Adopt an independent critical perspective, report observation uncertainties, and focus on overlooked points without reverse engineering, interactively testing, or fixing the depicted system.The guidelines assume the depicted system was rigorously developed and tested.Source: §IV · p. 3
- Document participating entities, environment elements, sensors, actuators, control modes, life-cycle concerns, suspicious events, and whether actions are verified.The guidelines also call attention to conflicts, surprising behavior, and unusual desirable solutions.Source: §IV · p. 3
- Examine interactions and dependencies among systems, objects, people, and ambient conditions, including interventions, workarounds, bystander reactions, and silent assumptions.Also report effects of the system on its surroundings.Source: §IV · p. 3
- Consider diverse deployment contexts, time scales, abstraction levels, event frequencies, and quantitative factors such as numbers of people or objects, weights, and distances.The guidance ranges from sub-second responses to system-wide multi-interaction tasks.Source: §IV · pp. 3–4
- Look for emergent patterns, omissions, early warning signs, risks, self-imposed constraints, and behaviors that could become problematic if continued or repeated.Examiners are encouraged to anticipate unhandled conditions and consider alternatives without needing to create a full test plan.Source: §IV · p. 4
V. EMERGING RESULTS
The authors compared their video examinations with and without an AI Skill. LLMs produced relevant review points, and the skill and higher frame-sampling rates yielded additional findings, while human observers also identified points the LLMs missed.
V. EMERGING RESULTS
- The authors examined several videos themselves and compared their findings with LLM examinations using and not using the AI Skill.The videos were self-explanatory, and no general descriptions were added.Source: §V · p. 4
- LLMs reported typically 10 to 20 valuable review points with and without an AI Skill, and none were outright incorrect, inapplicable, or about extremely unlikely scenarios.Some points were ones human observers had not noticed initially.Source: §V · p. 4
- Reviewing the same video again with an AI Skill led LLMs to report additional review points beyond an examination without the skill.Source: §V · p. 4
- LLMs reasoned about elements not shown in the videos, but human observers sometimes identified review points the LLMs missed, even when using the skill.Source: §V · p. 4
- Fixed-frequency frame sampling sometimes missed issues; prompting for a higher sampling rate at designated intervals enabled their detection.Examples included a package briefly blown away and a robot joint rotating more than 360 degrees.Source: §V · p. 4
- LLM review points often raised domain-specific details for software engineers to follow up, including delayed checks for plant heat damage and confirmation of package delivery.Source: §V · p. 4
VI. FUTURE PLANS
The authors plan to develop automated support for the methodology, evaluate its impact and cost effectiveness in real-world system development, and investigate additional applications and cognitive skills.
VI. FUTURE PLANS
- Planned software development includes agents and AI Skills for finding videos, identifying cross-domain applicability of review points, and drawing conclusions from differences between examiners' outputs.Source: §VI · p. 4
- The authors plan to evaluate the methodology's impact and cost effectiveness during development of real-world target systems.Source: §VI · p. 4
- Additional applications to investigate include examining the target system itself and examining other systems for competitive analysis or acquisition comparisons.Source: §VI · p. 4
- The authors propose investigating whether DDCE's mental activities are included in Algorithmic and Computational Thinking or reflect distinct skills warranting special attention.Source: §VI · p. 5
VII. CONCLUSION
The proposed methodology supports engineers in developing a holistic system view by critically examining videos of systems other than the target. Its initial results demonstrate promise, while the authors argue that its component combination can benefit engineers and automated tools and, through them, future systems.
VII. CONCLUSION
- Critiquing videos of other systems offers engineers low-risk practice in applying system-wide perspectives, alongside studying texts and interacting with domain experts.Source: §VII · p. 5
- The authors argue that the approach may strengthen engineers' domain knowledge and perspectives and help identify target-relevant issues more quickly, enriching further study.Source: §VII · p. 5
- The methodology combines concise examination instructions for humans and AI, video-based examination, focus on systems other than the target, and a process for applying findings to the target.Source: §VII · p. 5
- The authors argue that the methodology's combination can benefit human engineers and automated tools and, through them, enhance future system quality.Source: §VII · p. 5