PaperlayerView exact revision on arXiv

Source-linked AI summary

A Practical Evaluation of Commercial Industrial Augmented Reality Systems in an Industry 4.0 Shipyard

Oscar Blanco-Novoa, Tiago M Fernandez-Carames, Paula Fraga-Lamas, Miguel Vilar-Montesinos

arXiv:2402.00925v1cs.HC

TL;DR

Shipyards need practical evidence on how Industrial AR tools perform in complex industrial settings. The paper reviews relevant applications, develops a fog-computing IAR architecture, and evaluates devices, SDKs, and markers in Navantia scenarios, finding substantial performance differences influenced by lighting.

  • Problem

    Shipbuilding’s complex processes motivate Industry 4.0 automation, but practical evaluations of generic IAR tools in realistic shipyard scenarios were lacking.

  • Method

    The paper reviews IAR applications and tools, presents Navantia’s fog-computing architecture, and evaluates three devices, two SDKs, and multiple markers in shipyard scenarios.

  • Results

    The evaluation found substantial performance differences among IAR tools, with lighting affecting recognition and the best combinations depending on the shipyard scenario.

  • Takeaways & Limitations

    Binary markers with ARToolKit provided the best recognition distances, while ARToolKit offered longer reading distances than Vuforia under normal lighting.

Abstract

from arXiv · show

The principles of the Industry 4.0 are guiding manufacturing companies towards more automated and computerized factories. Such principles are also applied in shipbuilding, which usually involves numerous complex processes whose automation will improve its efficiency and performance. Navantia, a company that has been building ships for 300 years, is modernizing its shipyards according to the Industry 4.0 principles with the help of the latest technologies. Augmented Reality (AR), which when utilized in an industrial environment is called Industrial AR (IAR), is one of such technologies, since it can be applied in numerous situations in order to provide useful and attractive interfaces that allow shipyard operators to obtain information on their tasks and to interact with certain elements that surround them. This article first reviews the state of the art on IAR applications for shipbuilding and smart manufacturing. Then, the most relevant IAR hardware and software tools are detailed, as well as the main use cases for the application of IAR in a shipyard. Next, it is described Navantia's IAR system, which is based on a fog-computing architecture. Such a system is evaluated when making use of three IAR devices (a smartphone, a tablet and a pair of smart glasses), two AR SDKs (ARToolKit and Vuforia) and multiple IAR markers, with the objective of determining their performance in a shipyard workshop and inside a ship under construction. The results obtained show remarkable performance differences among the different IAR tools and the impact of factors like lighting, pointing out the best combinations of markers, hardware and software to be used depending on the characteristics of the shipyard scenario.

I. INTRODUCTION

The paper frames Industrial Augmented Reality as a promising Industry 4.0 technology for modernizing complex shipbuilding processes, while addressing limited practical evaluation in realistic shipyard settings.

  • IAR applications span design, assembly, training, maintenance, factory planning, and quality control across manufacturing contexts.
  • Industrial IAR is characterized by hardware and software suited to factories through properties such as robustness, ruggedness, accessibility, and battery life.
  • Navantia’s Shipyard 4.0 project evaluates IAR as a way to provide shipyard operators with user-friendly process information and interfaces.
  • The paper’s stated contributions combine an IAR literature review, a shipyard-oriented system design, and practical evaluation of diverse hardware and software.
  • The paper reviews IAR research for shipbuilding and smart manufacturing and identifies a lack of practical evaluations in comparable shipyard scenarios.

B. IAR FOR SHIPBUILDING

Shipbuilding IAR research addresses demanding operational settings through applications for welding, training, maintenance, design, and broader commercial shipbuilding workflows.

  • Shipbuilding-specific IAR research has been developed for tough environments such as shipyards and ships under construction.
  • Welding: Welding applications include augmented helmets, robot interaction interfaces, and IAR-based welder training.
  • Training: IAR training systems have used AR glasses and a paint-gun interface to let operators practice painting virtual structures and immediately view results.
  • Maintenance: Maintenance systems replace paper or electronic documents with tablets that provide step-by-step instructions for ship-related tasks.
  • Design: Design-oriented IAR can compare ship CAD models with actual construction to detect discrepancies such as misalignments or collisions.
  • Commercial applications: Commercial providers offer IAR applications for shipbuilding construction, safety, training, operations, maintenance, and utilities.

C. IAR HARDWARE AND SOFTWARE

Industrial AR hardware offers diverse wearable and display options but remains constrained by cost and usability, while software SDKs provide recognition, tracking, and interaction capabilities.

  • Hardware: Commercial industrial AR interfaces mainly use smart glasses, with helmets and other head-mounted devices also available.
  • Hardware: US $500 to US $5,000 prices make current IAR hardware too expensive for massive deployment.
  • Hardware: No perfect wearable device has been found because desirable properties include wide field of view, low weight, full-day battery life, and optical projection.
  • Interaction: Voice interaction is recommended for hands-free operation, but noisy industrial environments still challenge voice processing.
  • Software: Available SDKs include open-source, free, and commercial options, so selection depends on the environment and development platform.
  • Software: Industrial AR SDKs implement some combination of fast rendering, recognition and tracking, and speech or gesture recognition.

III. IAR SYSTEM DESIGN

Shipyard 4.0 identifies IAR use cases spanning plant information, quality control, manufacturing assistance, asset location, installation visualization, warehouse management, predictive maintenance, collaboration, and block assembly. These applications connect contextual information, spatial guidance, monitoring, and reporting to shipyard operations.

  • A. SHIPYARD’S IAR USE CASES: Shipyard 4.0 selects IAR use cases across information access, quality, manufacturing, tracking, maintenance, collaboration, logistics, and block assembly.The selected applications target processes in workshops and ships, including pipe identification, quality inspection, operator guidance, asset location, infrastructure visualization, warehouse operations, predictive maintenance, reporting, and alignment.
  • A. SHIPYARD’S IAR USE CASES: Plant information uses markers as asset identifiers so applications can display contextual properties such as a pipe’s material, size, and destination.The example shows Vuforia displaying pipe information on a smartphone.
  • A. SHIPYARD’S IAR USE CASES: Quality control can superimpose Navantia’s 3D CAD model on a real piece and automatically identify deviations after 3D scanning and reconstruction.The workflow combines visual comparison with 3D cameras and reconstruction software.
  • A. SHIPYARD’S IAR USE CASES: IAR can locate pipes through interaction with an active UHF RFID positioning system and present their locations on tablets or smart glasses.UHF RFID was selected after considering shipyard conditions including metals, liquids, interference, reading distance, temperature, mobility, and cost.
  • A. SHIPYARD’S IAR USE CASES: IAR can overlay ship infrastructure models to reveal concealed piping or wiring and support maintenance, fault repair, and real-time infrastructure monitoring.The HoloLens example displays a monitoring view of Navantia’s Fene shipyard.
  • A. SHIPYARD’S IAR USE CASES: Warehouse IAR can guide operators to locate and store items faster while displaying shelf contents, and collaboration tools can share operator views and enrich reports with audio and video.Block-assembly applications can also project 3D block models to guide hull-piece alignment and pipe positioning.

B. COMMUNICATIONS ARCHITECTURE

Navantia’s IAR system uses a three-layer fog-computing architecture connecting IAR devices, local gateways, and cloud services. Local fog nodes reduce latency, distribute computation and storage, and provide localized data, while shipboard connectivity remains challenging.

  • B. COMMUNICATIONS ARCHITECTURE: Local fog nodes minimize latency by supplying IAR devices with localized data and responding faster than remote cloud services.The fog layer can act as a proxy caching server for IAR data.
  • B. COMMUNICATIONS ARCHITECTURE: Fog services distribute computational power and storage to resource-constrained IAR devices, supporting lighter hardware and longer battery life.Demanding tasks and large file storage can be delegated to local gateways.
  • B. COMMUNICATIONS ARCHITECTURE: The fog architecture supports physically distributed, low-latency, QoS-aware applications while reducing network traffic and cloud computational load.Small, inexpensive gateways also make the system flexible and scalable.
  • B. COMMUNICATIONS ARCHITECTURE: Navantia’s architecture comprises node, fog, and cloud layers linking IAR devices, local single-board gateways, sensor and RFID networks, and cloud services.The fog layer exchanges data with the shipyard IIoT ecosystem and RFID readers, while the cloud stores data and runs compute-intensive services.
  • B. COMMUNICATIONS ARCHITECTURE: IAR devices communicate with the fog through deployed IEEE 802.11 b/g/n WiFi, but metal structures challenge electromagnetic propagation inside ships.PLC is being evaluated, although electrical interference from high-current tools affects network speed.
  • B. COMMUNICATIONS ARCHITECTURE: The reported experiments evaluate marker recognition and tracking, not the communications performance of the fog architecture.This scope boundary follows the unresolved connectivity challenges inside ships.

IV. IMPLEMENTATION

The implementation combines three IAR devices with ARToolkit and Vuforia within a fog-computing architecture designed for shipyard conditions. Marker recognition depends on the chosen SDK, marker type, hardware, lighting, and viewing conditions.

  • A. SELECTED HARDWARE: Three devices—a smartphone, rugged tablet, and smart glasses—were tested as the IAR system’s node layer.The devices were the UMI Super, Panasonic FZ-A2mk1, and Epson Moverio BT-2000.
  • A. SELECTED HARDWARE: The proposed architecture uses fog computing to move computational and communication capabilities closer to sensor nodes for lower-latency AR interaction.
  • A. SELECTED HARDWARE: Smartphone and tablet users point at markers through touch interfaces, whereas the glasses provide see-through projection with gesture, touch, and voice interaction.
  • B. SELECTED SOFTWARE: ARToolkit and Vuforia were selected because they support complementary recognition capabilities, including ARToolkit’s long-distance binary-marker recognition and Vuforia’s image-feature processing.
  • B. SELECTED SOFTWARE: Lighting, reflections, shadows, marker material, marker size, and electrical interference can substantially affect recognition and tracking performance.
  • B. SELECTED SOFTWARE: Markerless systems remain constrained by mapping, memory, computational, positioning, and ambient-light requirements in shipyard environments.

B. EXPERIMENTAL SETUP

The experimental setup evaluates three IAR devices with applications, recognition methods, and marker types tailored to shipyard asset information. Markers were selected for either natural-feature or binary-code recognition and adapted for environmental exposure.

  • B. EXPERIMENTAL SETUP: The experiments evaluate the three IAR devices with varied markers and applications based on ARToolkit or Vuforia.
  • B. EXPERIMENTAL SETUP: The applications associate markers with pipes and display work-order, type, diameter, material, pallet, thickness, and weight information.
  • B. EXPERIMENTAL SETUP: Natural Feature Tracking stores and recognizes visual patterns, while binary-code detection is an ARToolkit-specific method optimized for fast recognition with binary markers.
  • B. EXPERIMENTAL SETUP: The marker set included custom square, square QR, rectangular QR, and binary markers with or without BCH error correction.
  • B. EXPERIMENTAL SETUP: Markers were printed on different materials, with some laminated for water and dust exposure, although reported results use non-laminated laser-printed paper markers.

C. WORKSHOP TESTS

Workshop tests compare marker recognition across devices, SDKs, marker geometries, and lighting conditions. ARToolkit with optimized binary markers achieved the longest distances, while device and marker geometry affected performance.

  • C. WORKSHOP TESTS: The workshop tests varied marker type, lighting, and reading angle, with Table 4 comparing maximum recognition and tracking distances across devices and SDKs.
  • C. WORKSHOP TESTS: Under improved lighting, ARToolkit recognized optimized binary markers at distances up to 26 m, compared with 6–10 m around 220 lx.The 26 m result enabled detection of 190 mm-wide wall markers from the workshop center.
  • C. WORKSHOP TESTS: Vuforia recognized the custom square marker at less than 2 m and did not detect ARToolkit-optimized binary markers because they provide few natural feature points.
  • C. WORKSHOP TESTS: Vuforia performed best with QR markers, generally below 2 m, while ARToolkit struggled with QR markers but achieved similar distances when detection succeeded.
  • C. WORKSHOP TESTS: Smartphone and tablet results were similar, whereas the Epson Moverio glasses did not reach the detection distances of the other devices.
  • C. WORKSHOP TESTS: The longest recognition distance was obtained by the Panasonic FZ-A2mk1 with ARToolkit detecting binary marker C, reaching 28 m.

BCH (13,9,3)

The BCH (13,9,3) binary marker is included in the detection-rate evaluation, and the supplied passages note that markers can remain detectable when partially hidden.

  • BCH (13,9,3): Table 5 reports detection rates for binary markers, while the experiments also consider recognition when markers are partially hidden.

D. EVALUATION OF THE IAR SYSTEMS IN A SHIP

Tests inside an offshore patrol vessel showed that lighting strongly affected marker recognition, with low-light dining-room conditions producing widespread failures and bridge results varying with reflections and viewing angle.

  • Dining-room tests: Under low illumination, most recognition and tracking tests failed, and measured distances generally decreased because marker white pixels appeared darker.The dining-room experiments covered three lighting scenarios, including very low illumination.
  • Dining-room tests: Camera characteristics strongly influenced low-light performance: the UMI Super performed worst, while Epson Moverio glasses adjusted sensitivity best and the tablet was intermediate.
  • Dining-room tests: Vuforia tolerated poor lighting better than ARToolKit but delivered shorter recognition distances, whereas ARToolKit and selected devices exceeded 10 m with some markers.
  • Bridge tests: In the bridge, ambient light enabled recognition of more markers, but window reflections required measurements at multiple angles.
  • Bridge tests: Backlighting behind markers made detection difficult, while ARToolKit performed better across bridge test cases and smartphone and glasses performance was similar.

E. KEY FINDINGS

The key findings identify lighting, camera dynamics, marker-recognition algorithms, and software openness as major determinants of shipyard IAR performance.

  • ARToolKit generally achieved longer recognition and tracking distances than Vuforia, exceeding 25 m with binary markers in some scenarios.BCH error-correction codes increased binary-marker detection rates, while NFT recognition stayed below 2 m.
  • Lighting markedly affected marker-based IAR: low light made many systems unusable, while normal light shifted the main differences toward software algorithms and reflections.
  • Under low luminosity, camera dynamic range was especially important because marker-edge detection requires high contrast across lighting conditions.
  • Potential lighting remedies include dedicated marker lamps or helmet-mounted directional lights, though their feasibility requires further study.
  • Laminated markers slightly reduced maximum tracking distance through reflections but offered greater durability against dust, water, and grease.
  • ARToolKit’s open-source design enabled adaptation to Epson glasses, whereas proprietary Vuforia could not be similarly extended during development.

VI. CONCLUSIONS

The study validated an IAR technology selection for Navantia’s Industry 4.0 shipyard and confirmed that lighting, viewing angle, and recognition algorithms shape system performance.

  • The experiments validated selected IAR hardware and software for shipyard applications while confirming the effects of lighting and reading angle.
  • ARToolKit produced substantially longer reading distances than Vuforia under normal lighting, and its recognition and tracking algorithms were important performance factors.
  • Marker-based IAR solutions still require adaptation to low-light conditions, while markerless platforms require further study for Shipyard 4.0 requirements.
Loading 2402.00925v1…