PaperlayerView exact revision on arXiv

Source-linked AI summary

Trust and Safety

S. K. Devitt, R. Horne, Z. Assaad, E. Broad, H. Kurniawati, B. Cardier, A. Scott, S. Lazar, M. Gould, C. Adamson, C. Karl, F. Schrever, S. Keay, K. Tranter, E. Shellshear, D. Hunter, M. Brady, T. Putland

arXiv:2104.06512v1cs.ROcs.HC

TL;DR

Increasingly autonomous and digitally connected robots challenge existing safety and assurance processes, while regulation struggles to keep pace with technological change. This chapter examines trust and safety frameworks for RAS-AI and argues for adaptive regulation, novel assurance approaches, and automated monitoring to support safe operations.

  • Problem

    Increasingly autonomous and digitally connected robots challenge existing safety and assurance processes, while regulation cannot keep pace with technology.

    Source: Introduction · p. 3, Regulating trust and safety · p. 3

  • Method

    The chapter examines existing and emerging trust, safety, regulatory, and assurance frameworks and methods for robotics and RAS-AI.

    Source: Introduction · p. 3, Conclusion · p. 26

  • Results

    Future trust and safety will combine standards with iterative, adaptive, and responsive regulation and assurance, including novel technical and social approaches for diverse RAS-AI applications.

    Source: Conclusion · p. 26

  • Takeaways & Limitations

    Automated assurance such as ASsurance-as-a-Service, stakeholder engagement, and continued framework development are proposed to support safe operation of future robotics systems.

    Source: Conclusion · p. 26

    Assurance must balance evaluator convenience and reliability, because higher evaluator requirements reduce the pool of qualified evaluators and hinder regular retesting.

    Source: Land · p. 15

Abstract

from arXiv · show

Robotics in Australia have a long history of conforming with safety standards and risk managed practices. This chapter articulates the current state of trust and safety in robotics including society's expectations, safety management systems and system safety as well as emerging issues and methods for ensuring safety in increasingly autonomous robotics. The future of trust and safety will combine standards with iterative, adaptive and responsive regulatory and assurance methods for diverse applications of robotics, autonomous systems and artificial intelligence (RAS-AI). Robotics will need novel technical and social approaches to achieve assurance, particularly for game-changing innovations. The ability for users to easily update algorithms and software, which alters the performance of a system, implies that traditional machine assurance performed prior to deployment or sale, will no longer be viable. Moreover, the high frequency of updates implies that traditional certification that requires substantial time will no longer be practical. To alleviate these difficulties, automation of assurance will likely be needed; something like 'ASsurance-as-a-Service' (ASaaS), where APIs constantly ping RAS-AI to ensure abidance with various rules, frameworks and behavioural expectations. There are exceptions to this, such as in contested or communications denied environments, or in underground or undersea mining; and these systems need their own risk assessments and limitations imposed. Indeed, self-monitors are already operating within some systems. To ensure safe operations of future robotics systems, Australia needs to invest in RAS-AI assurance research, stakeholder engagement and continued development and refinement of robust frameworks, methods, guidelines and policy in order to educate and prepare its technology developers, certifiers, and general population.

Introduction

As RAS-AI becomes more autonomous, connected, and widespread, existing trust and safety processes face increasing challenges. The chapter examines emerging frameworks, regulatory approaches, and interdisciplinary methods for addressing these challenges.

  • Increasingly autonomous and networked RAS-AI challenges existing safety and assurance processes across industries.Source: Introduction · p. 3
  • The chapter examines gaps between current trust and safety practices and those required as robotics becomes more autonomous.Source: Introduction · p. 3

Regulating trust and safety

Designing democratic AI

  • The chapter proposes designing for trust through transdisciplinary systems methods that combine humanities and STEM perspectives.Source: Human values · p. 4

Safety and assurance

Robotics safety builds on established standards, system safety, and organisational safety management, but increasingly complex systems require attention to interactions across disciplines and throughout system evolution. Existing processes improve safety outcomes, while automation- and autonomy-specific guidance remains an active research area.

  • Established standards use risk variables to shape control-system architecture, with higher risks requiring redundant controls and fault detection.Source: Safety and assurance · p. 5
  • System safety applies engineering and management to identify and manage hazards while balancing safety and operability across the system life cycle.Source: Safety and assurance · pp. 5–6
  • Safety management systems extend system safety to organisational structures, accountabilities, policies, procedures, and safety culture.Source: Safety and assurance · p. 6
  • Although system safety and safety management processes improve safety outcomes, standards and practices specific to automation and autonomy remain underdeveloped and actively researched.Source: Safety and assurance · p. 6
  • Systems-of-systems safety requires considering interactions among operational safety, cybersecurity, resilience, change management, emergency management, and occupational health and safety.Source: Safety and assurance · p. 7

Systems integration testing

  • Model-based integration testing can alleviate some difficulties of testing complex systems in the field.Source: Safety and assurance · p. 7

Emerging Issues in Assuring Autonomous Robots

Assuring autonomous robots is difficult because uncertainty affects their sensing, decision-making, software, hardware, and interactions with physical environments. The chapter therefore frames assurance as a multi-level problem requiring approaches that address changing systems and complex interactions.

Algorithms

  • Algorithm assurance must address uncertainty while providing useful guarantees about solution quality and computational requirements such as time and memory.Source: Algorithms · p. 8

Hardware

  • Soft and compliant robots create new assurance difficulties because their safety-oriented hardware dynamics depend on additional factors.Source: Hardware Components · p. 8

Integrated Robotic System

  • Reliable assurance must account for erroneous components and complex system interactions, including seemingly minor issues such as low battery states.Source: Integrated Robotic System · p. 9

Regulation in Air, Maritime and Land

Australia regulates RAS-AI differently across air, maritime, and land domains, with each facing distinct gaps between existing rules and autonomous operations. Current responses emphasize collaboration, risk-based approaches, and reforms that support safe testing and deployment.

Air

  • CASA applies existing remotely piloted aircraft requirements broadly to unmanned systems, without specified autonomous-aircraft content or graduated requirements by autonomy level.A human-controlled RPA and a fully autonomous aircraft can differ substantially in communication, visual capability, intent, and safety risks.Source: Regulation in Air, Maritime and Land · p. 10

Maritime

  • Maritime operators must often seek exemptions because existing vessel requirements were written for traditional manned ships and lack tailored autonomous-vessel standards.This creates administrative burdens and delays for operators and AMSA.Source: Maritime · p. 11
  • The Autonomous Vessel Forum identified collaboration, assured technology, risk controls, safe failure, flexible regulation, changing seafarer roles, and cyber-attack response as central to safe maritime autonomy.The forum brought together 135 experts from industry, academia, Navy, and government.Source: Maritime · pp. 12–13
  • AMSA is pursuing collaborative regulatory reform intended to streamline assurance and accreditation, better match regulatory oversight to risk, and reduce barriers to maritime technology uptake.The approach involves stakeholders working on autonomous-system assurance and accreditation pathways.Source: Maritime · p. 13

Land

  • Land-transport reform must address unknown deployment timing, applications, technology mixes, ownership changes, and shifting responsibility at higher automation levels.The NTC proposes nationally consistent, internationally aligned, technology-neutral, outcomes-based reforms with safety as the key outcome.Source: Public land · p. 17

Robotics standards and regulation

Existing accreditation frameworks and standards do not adequately cover diverse autonomous systems, particularly software, learning systems, and systems operating across inherited sectoral legal frameworks. The chapter therefore points toward broader regulatory re-evaluation and continuous assurance approaches.

Framework gaps

  • Systematic and efficient accreditation frameworks for RAS-AI do not yet exist, while existing mechanisms may assume constant human control or physical separation from people.These assumptions are poorly suited to increasingly autonomous systems.Source: Robotics standards and regulation · p. 18

Standards

  • ISO/TC 299 standards provide useful guidance for industrial robots but do not cover more diverse robotic industries, robotic software, or learning systems.Functional-safety software standards include IEC61508, IEC62061, and ISO 13849-1.Source: Robotics standards and regulation · p. 18

Regulatory complexity

  • Sector-specific legal definitions and inherited regulatory frameworks make assurance frameworks more complex for different autonomous-technology types.The Australian industry must align assurance with societal acceptance and practical compliance requirements.Source: Robotics standards and regulation · p. 18

Continuous assurance

  • ACcreditation-as-a-Service uses an API to request operational status updates about what a robot is doing and why, supporting oversight of rapidly changing systems.A robust communications network could enable faster intervention when operations depart from authorised parameters.Source: Robotics standards and regulation · p. 19

Adaptive regulation

  • The chapter suggests that autonomous robotics may require a complete re-evaluation of existing regulatory approaches rather than only discrete new interventions.This aligns with movement from prescriptive inspection toward risk-based, responsive, and anticipatory regulation.Source: Robotics standards and regulation · p. 19

New ways of thinking about regulation

Australia’s robotics regulation must adapt to rapidly evolving autonomous systems while preserving safety, community trust, and effective oversight. The chapter highlights anticipatory, collaborative regulation alongside computational and democratic approaches to AI governance.

Conclusion

Trust and safety in increasingly autonomous RAS-AI will require adaptive assurance approaches alongside standards, including automated monitoring and stakeholder preparation.

Conclusion

  • Trust and safety work must address society’s expectations, safety management systems, system safety, and emerging issues in autonomous robotics.The chapter situates these needs within Australia’s history of safety standards and risk-managed practices.Source: Conclusion · p. 26
  • Adaptive, responsive regulation and assurance should complement standards across diverse RAS-AI applications.The chapter also calls for novel technical and social approaches, especially for game-changing innovations.Source: Conclusion · p. 26
  • Frequent user updates can make pre-deployment assurance and time-intensive certification impractical, motivating automated assurance such as ASaaS.ASaaS would use APIs to continually check systems against rules, frameworks, and behavioural expectations.Source: Conclusion · p. 26
  • Contested, communications-denied, underground, and undersea environments require distinct risk assessments and imposed limitations.Self-monitors are already operating within some systems.Source: Conclusion · p. 26
  • Australia needs sustained investment in assurance research, stakeholder engagement, and robust frameworks, methods, guidelines, and policy.These efforts should educate and prepare technology developers, certifiers, and the general population.Source: Conclusion · p. 26

Appendices

The appendices list standards and projects covering industrial, service, mobile, collaborative, medical, and autonomous robotics safety and performance.

Appendices

  • Several entries are drafts, working drafts, or committee documents, indicating ongoing development of the robotics standards portfolio.Examples include ISO/DIS, ISO/CD, ISO/WD, and ISO/FDIS documents listed throughout the appendix.Source: Appendices · pp. 29–30
  • The appendix also lists standards for personal care, collaborative, and medical robotics, including robotically assisted surgical and rehabilitation equipment.These include ISO 13482, ISO/TS 15066, IEC/TR 60601-4-1, IEC 80601-2-77, and IEC 80601-2-78.Source: Appendices · p. 30

APPENDIX B: AUSTRALIA’S AI ETHICS PRINCIPLES28

The appendix presents AI ethics principles centered on wellbeing, human-centred values, fairness, privacy protection, and security.

APPENDIX B: AUSTRALIA’S AI ETHICS PRINCIPLES28

  • AI systems should benefit individuals, society, and the environment throughout their lifecycle.Source: Appendix B · p. 31
  • AI systems should respect human rights, diversity, and individual autonomy throughout their lifecycle.Source: Appendix B · p. 31
  • AI systems should be inclusive and accessible and should not result in unfair discrimination.Source: Appendix B · p. 31
  • The listed principles also include privacy protection and security.Source: Appendix B · p. 31
Loading 2104.06512v1…