Source-linked AI summary
Adaptive Training for Nautical Rules of the Road
Amit Dutta, Sushil J. Louis
TL;DR
Students often struggle to master COLREGs, and safe navigation requires practice in recognizing collision risks and selecting appropriate actions. The paper compares adaptive and non-adaptive simulation trainers under controlled instructional conditions. Adaptive training produced higher post-test scores and more favorable engagement and feedback ratings, though generalization beyond 30 university students remains unresolved.
Problem
Students often find COLREGs difficult to master, while real-world navigation practice can be costly and dangerous.
Method
The study compares RAFT and RoR, which share instructional content, scenario framework, and interface but differ in performance-based difficulty control and feedback delivery.
Results
Students trained with RAFT achieved significantly higher post-test scores than RoR students, with p < 0.0001 and d > 2, while survey ratings favored RAFT.
Takeaways & Limitations
The findings provide evidence that adaptive training supported better learning outcomes and a more engaging training experience in this nautical simulation study.
Takeaways & Limitations
The sample consisted of 30 university students, so generalization to other populations and operational settings requires additional studies.
Abstract
from arXiv · showhide
Knowledge of the nautical rules of the road is essential for safe ship navigation and collision avoidance. We evaluated adaptive and non-adaptive versions of a ship-driving simulation trainer designed to assess and improve students' knowledge and application of these rules. We randomly assigned 30 university students to an adaptive or non-adaptive training condition and measured learning using pretest and post-test scores. Students who received adaptive training achieved significantly higher post-test scores than those who received non-adaptive training (p < 0.0001). After the post-test, all students experienced both versions of the trainer and compared them in a survey. Of the 30 students, 73% judged the adaptive trainer more effective, and 22 rated it "very engaging," compared with 9 who gave the non-adaptive trainer the same rating. These findings provide evidence that adapting scenario difficulty and providing immediate, context-sensitive feedback can improve both learning outcomes and student engagement in simulation-based training.
1 Introduction
COLREGs are essential for safe navigation but difficult for students to master, while simulation offers controlled practice. This paper compares adaptive RAFT with non-adaptive RoR and reports better learning and engagement with adaptive training.
- COLREGs knowledge and correct application are essential for preventing collisions and reducing associated harms.
- Simulation provides a controlled setting for practicing collision-risk recognition, rule application, and safe decision making without real-world hazards.
- One-size-fits-all training can assign tasks that are too easy or too difficult, slowing learning and reducing engagement.
- RAFT adapts scenario difficulty and feedback to student performance, whereas RoR uses the same instructional content and interface without adaptation.
- The user study found significantly higher post-test scores and stronger engagement for adaptive RAFT than non-adaptive RoR.
- The paper contributes a controlled comparison using objective learning measures and student assessments of effectiveness, engagement, and feedback.
2 Prior Work
Prior research reports benefits of adaptive training across several domains, but outcomes vary by task, population, and adaptation method. Existing maritime studies motivate RAFT’s performance-based approach to navigational decision making.
- Adaptive training modifies content, difficulty, or feedback in response to learner performance across educational, medical, industrial, and military applications.
- A digital reading-game study found improvement under all three conditions but no significant cognitive or non-cognitive differences between them.
- Adaptive and non-adaptive laparoscopic trainers produced no significant effectiveness difference, although participants strongly preferred the adaptive version.
- Adaptive training nearly doubled X-ray screening performance, but the study lacked a non-adaptive comparison and could not isolate adaptation’s effect.
- Related maritime systems adapted task difficulty and feedback for submarine and target-angle tasks, whereas RAFT addresses a broader navigational task.
3 Rules of the Road Adaptive Fleet Training (RAFT)
RAFT trains students to analyze simulated collision risks and apply COLREGs using adaptive difficulty and immediate, context-sensitive feedback. Its continuous performance-based model maps student scores to increasingly challenging scenario conditions.
- RAFT generates crossing, head-on, and overtaking scenarios in which students identify collision risks and apply COLREGs.
- The Captain’s Report structures analysis of target location, motion, collision risk, applicable rule, and recommended action.
- RAFT scores each report, updates its difficulty variable, generates the next scenario, and provides immediate feedback on correct and incorrect responses.
- RAFT uses continuous difficulty δ ∈[0, 1], while RoR uses one of three fixed levels that do not respond to performance.
- Difficulty controls time of day, time to closest point of approach, and number of ships.
- As performance improves, RAFT increases δ and produces later-day or nighttime scenarios, denser traffic, and less time before closest approach.
- RAFT computes scenario scores from 16 weighted report components before updating difficulty.
- The update rule maps higher normalized scores to larger difficulty increases, after which RAFT maps difficulty to TCPA, TOD, and NSHIP.
4 Experimental Design
The study randomly assigned 30 university students to adaptive RAFT or non-adaptive RoR training, using pre-tests, assigned practice, post-tests, and a cross-over survey.
- 30 university students were randomly assigned, with 15 students in each training condition.
- Students completed a common pre-test, practiced with either adaptive RAFT or non-adaptive RoR, and then completed a common post-test.
- After the post-test, each group used the trainer it had not used during initial practice.
- The cross-over exposure enabled all students to compare both systems in the final survey, while learning effectiveness was assessed from pre-test and post-test results.
5 Results
RAFT progressively individualized scenario difficulty and produced stronger learning outcomes than RoR, while students generally preferred RAFT for effectiveness, engagement, and feedback. Both trainers received generally positive satisfaction ratings, and survey responses did not isolate which RAFT feature caused perceived differences.
- Adaptive difficulty: RAFT increased difficulty according to each student’s performance, producing individualized difficulty trajectories across training scenarios.
- Adaptive difficulty: All 15 RAFT students progressed from novice to intermediate difficulty, ending between δ = 0.4 and δ = 0.6.
- Learning outcomes: The post-test weighted intermediate-level questions twice as heavily as novice-level questions to account for their different difficulty.
- Learning outcomes: 57.1% versus 62.3% were the mean pre-test scores for RAFT and RoR; post-test means were 74.3% and 61.4%, respectively.
- Learning outcomes: p < 0.0001 and d > 2: RAFT achieved significantly higher post-test scores than RoR, with a large between-condition effect.
- Response times: 91.9 versus 90.0 seconds were the groups’ mean pre-test response times; post-test means were 79.5 seconds for RAFT and 99.4 seconds for RoR.
- Student preferences: 73% of students selected RAFT as more effective, while 22 selected RAFT as “Very engaging,” compared with 9 for RoR.
- Student preferences: RAFT received the highest feedback rating from 21 students versus 10 for RoR, while both trainers generally satisfied students with overall quality and question difficulty.
6 Conclusions and Future Work
RAFT combined performance-based difficulty adaptation with immediate, context-sensitive feedback, while RoR used fixed difficulty and quiz-end feedback. In a study of 30 students, RAFT produced better learning and engagement outcomes, though generalization beyond university students and this setting remains uncertain.
- RAFT adjusted continuous scenario difficulty from student performance and provided immediate, context-sensitive feedback; RoR used fixed levels and quiz-end feedback.Both trainers otherwise shared the same instructional content, scenario-generation framework, and user interface.
- RAFT students achieved significantly higher post-test scores than RoR students, with p < 0.0001 and large effect sizes (d > 2).The study randomly assigned 15 of 30 university students to each condition; RAFT students also answered post-test questions significantly faster.
- 22 of 30 students (73%) judged RAFT more effective, while 22 rated it “Very engaging” compared with 9 for RoR.Survey responses also favored RAFT’s feedback and guidance, rated “Very well” by 21 students versus 10 for RoR.
- Because the sample comprised 30 university students, additional studies are needed to determine whether the findings generalize to other populations and operational settings.
- RAFT illustrates how a fixed-level trainer can become performance-based by continuously updating difficulty and mapping it to variables controlling the next scenario.The approach increases scenario difficulty as student performance improves.
- Future work will test bidirectional difficulty adaptation, isolate the effects of difficulty adaptation and immediate feedback, and adapt multiple scenario variables independently.These studies are intended to extend RAFT beyond a single difficulty value.