Source-linked AI summary
Cybersickness in Virtual Reality Questionnaire (CSQ-VR): A Validation and Comparison against SSQ and VRSQ
Panagiotis Kourtesis, Josie Linnell, Rayaan Amir, Ferran Argelaguet, Sarah E. MacPherson
TL;DR
Cybersickness can impair cognitive and motor performance, while SSQ and VRSQ have limitations as VR-specific measures. This study validates paper-and-pencil and VR versions of CSQ-VR against those questionnaires and evaluates performance decline and pupil size. CSQ-VR showed stronger psychometric performance, supported assessment during immersion, and pupil size predicted cybersickness intensity.
Problem
SSQ and VRSQ have limitations for measuring cybersickness, including omitted nausea items, post-exposure-only assessment, and difficult score interpretation.
Method
The study validates paper-and-pencil and VR CSQ-VR versions against SSQ and VRSQ in participants exposed to repeated VR rides with cognitive and psychomotor testing.
Results
CSQ-VR showed better internal consistency and psychometric properties than SSQ and VRSQ for detecting temporary performance decline due to cybersickness.
Takeaways & Limitations
CSQ-VR supports valid cybersickness assessment during VR exposure, and pupil size predicted cybersickness intensity.
Takeaways & Limitations
The sample was limited to young adults, excluded participants with high MSSQ scores, and assessed only working memory and psychomotor functions.
Abstract
from arXiv · showhide
Cybersickness is a drawback of virtual reality (VR), which also affects the cognitive and motor skills of the users. The Simulator Sickness Questionnaire (SSQ), and its variant, the Virtual Reality Sickness Questionnaire (VRSQ) are two tools that measure cybersickness. However, both tools suffer from important limitations, which raises concerns about their suitability. Two versions of the Cybersickness in VR Questionnaire (CSQ-VR), a paper-and-pencil and a 3D-VR version, were developed. Validation and comparison of CSQ-VR against SSQ and VRSQ were performed. Thirty-nine participants were exposed to three rides with linear and angular accelerations in VR. Assessments of cognitive and psychomotor skills were performed at baseline and after each ride. The validity of both versions of CSQ-VR was confirmed. Notably, CSQ-VR demonstrated substantially better internal consistency than both SSQ and VRSQ. Also, CSQ-VR scores had significantly better psychometric properties in detecting a temporary decline in performance due to cybersickness. Pupil size was a significant predictor of cybersickness intensity. In conclusion, the CSQ-VR is a valid assessment of cybersickness, with superior psychometric properties to SSQ and VRSQ. The CSQ-VR enables the assessment of cybersickness during VR exposure, and it benefits from examining pupil size, a biomarker of cybersickness.
1. Introduction
Cybersickness limits VR applications and can temporarily impair cognitive and motor performance. The study develops and evaluates CSQ-VR against SSQ and VRSQ, including its use during immersion and pupil-size prediction.
- 1.1. Cybersickness, Cognition, and Motor Skills: Cybersickness can temporarily reduce cognitive and motor performance, threatening VR applications that require intact abilities.Prior studies reported slower reaction speed and effects on attentional and visual-processing performance.
- 1.2. Cybersickness Questionnaires: SSQ and VRSQ have limited suitability for cybersickness assessment because they omit nausea-related items, assess symptoms only after exposure, and produce difficult-to-interpret scores.The VRSQ factor analysis rejected all seven nausea items, although nausea is a frequent cybersickness symptom.
- 1.2. Cybersickness Questionnaires: CSQ-VR addresses these shortcomings with two questions each for nausea, disorientation, and oculomotor symptoms on a 7-point scale, producing interpretable total and sub-scores.Scores are calculated by adding the corresponding item responses.
- 1.3. Research Aims: The study validates paper-and-pencil and VR CSQ-VR versions against SSQ and VRSQ for evaluating cybersickness and detecting temporary cognitive or motor decline.The VR version is intended to support ongoing assessment while users remain immersed.
- 1.3. Research Aims: Pupillometry is also examined as a potential predictor of cybersickness intensity during VR exposure.
2.1. Virtual Environment Development
The study uses an immersive VR environment with touch-based interaction, eye-tracking, and repeated five-minute rides. The rides combine forward linear and angular acceleration patterns to induce cybersickness.
- 2.1. Virtual Environment Development: The virtual environment was implemented in Unity3D with SteamVR interactions designed to be ergonomic and require touching rather than button presses.
- 2.1. Virtual Environment Development: Instructions were delivered through video, audio, and written formats, while eye-tracking supported pupillometry during the experiment.
- 2.1.1. Linear and Angular Accelerations in VR: The VR ride lasted five minutes and was repeated three times, using linear and angular accelerations to induce cybersickness efficiently.The total ride exposure was 15 minutes per participant.
- 2.1.1. Linear and Angular Accelerations in VR: Figure 1 depicts examples of the ride’s linear acceleration on the left and angular accelerations in the centre and right.
2.2. Cognitive and Psychomotor Skills' Assessment
The experiment uses immersive VR adaptations of established cognitive and psychomotor tests. These assessments measure working memory, visuospatial memory, and reaction-time components before and after VR rides.
- 2.2. Cognitive and Psychomotor Skills' Assessment: Immersive VR versions of established tests were developed to repeatedly assess cognition and motor skills during VR exposure.The task development followed guidelines for cognitive assessment in immersive VR.
- 2.2.1. Working Memory: The Backward Digit Span Task measures working memory by requiring participants to recall heard digit sequences in reverse order using a VR keypad.
- 2.2.2. Visuospatial Working Memory: The Backward Corsi Block Test presents sequences across nine visible boxes positioned within a 27-box three-dimensional layout to assess visuospatial working memory.
- 2.2.3. Psychomotor Skills: The Deary–Liewald tasks assess simple and choice reaction time through 20 and 40 touch-response trials, respectively.
- 2.2.3. Psychomotor Skills: Eye-tracking separates choice reaction performance into overall Reaction Time, Attentional Time, and Motor Time.These scores represent psychomotor speed, attentional processing speed, and movement speed.
- 2.2. Cognitive and Psychomotor Skills' Assessment: Figure 2 shows the Digit Span, Corsi Block, and Deary–Liewald reaction-time tasks used in the assessment battery.
2.3. Cybersickness Questionnaires
The study describes limitations of SSQ and VRSQ and presents paper-and-pencil and VR versions of CSQ-VR designed to assess cybersickness more comprehensively, including during VR exposure.
- SSQ was developed for simulator sickness, whereas VRSQ omits nausea despite its frequency in cybersickness and has limited validation evidence.
- Both SSQ and VRSQ use 4-point scales, were not designed according to Likert-scale guidelines, and produce scores that are difficult to interpret.
- 2.3.1. Cybersickness in VR Questionnaire: CSQ-VR was developed from the VRISE section of VRNQ to address inverse scoring and insufficient coverage of oculomotor symptoms.
- 2.3.1. Cybersickness in VR Questionnaire: CSQ-VR uses six questions covering nausea, disorientation, and oculomotor symptoms, with two questions for each symptom type.
- 2.3.1. Cybersickness in VR Questionnaire: Each CSQ-VR item uses a 7-point response scale combining numbers and text from absent feeling to extreme feeling.
- 2.3.1. Cybersickness in VR Questionnaire: A 3D-VR CSQ-VR version was developed with an interface allowing users to select responses by touching numbers or sliding along a slider.
- 2.3.1. Cybersickness in VR Questionnaire: The VR version continuously measured average right- and left-eye pupil size while users answered the questionnaire.
- 2.3.1. Cybersickness in VR Questionnaire: The VR interface incorporated eye-tracking targets that were not visible to users during questionnaire completion.
2.4. Participants and Procedures
Thirty-nine participants were recruited through university mailing lists and social media, screened for motion-sickness susceptibility, trained on the VR equipment, and assessed before and during VR exposure.
- Thirty-nine participants were recruited by opportunity sampling through University of Edinburgh mailing lists and social media advertisements.The sample comprised 22 females and 17 males, with a mean age of 25.28 years.
- Participants scoring above the 75th MSSQ percentile were excluded to reduce the likelihood of severe cybersickness during VR exposure.
- Before the experiment, participants learned to wear the headset and use the controllers, then completed demographic and experience questionnaires.
- The experiment used an HTC Vive Pro Eye with a 120Hz eye tracker and reported tracking accuracy of 0.5°–1.1°.
- Before VR exposure, participants completed the paper CSQ-VR, SSQ, and VRSQ, then performed tutorial and experimental tasks while standing in the VR area.
2.5. Statistical Analyses
The analysis plan combined descriptive statistics, reliability testing, and correlations to evaluate CSQ-VR consistency and validity against established questionnaires.
- Reliability analyses assessed CSQ-VR internal consistency using recommended Cronbach’s α thresholds.Adequate, good, and very good consistency were defined as α ranges of 0.6–0.7, 0.7–0.8, and 0.8–0.95.
- Pearson correlations examined the validity of both CSQ-VR versions against post-exposure SSQ and VRS scores.
- Descriptive statistical analysis provided an overview of the participant sample.
3. Results
The CSQ-VR showed strong reliability and validity across paper-and-pencil and VR versions, outperforming SSQ and VRSQ in detecting temporary reaction- and motor-speed decline. Pupil size significantly predicted cybersickness intensity.
- 3.1. Reliability and Validity: CSQ-VR demonstrated superior overall internal consistency compared with SSQ and VRSQ.Its Total and Vestibular scores showed very good internal consistency, while Nausea and Oculomotor scores showed good internal consistency.
- 3.1. Reliability and Validity: Both CSQ-VR versions were significantly correlated with SSQ and strongly associated with VRSQ scores, supporting their construct validity.Total scores showed especially strong associations, while Vestibular correlations were comparatively weaker but remained moderate or strong.
- 3.2. Detection of Temporary Decline due to Cybersickness: Eleven observations met the criterion for temporary cognitive or motor decline, including six reaction-speed and five motor-speed declines.All temporary declines were observed on the DLRT task.
- 3.2. Detection of Temporary Decline due to Cybersickness: The CSQ-VR versions showed the best sensitivity and specificity for detecting temporary reaction-time and motor-speed decline compared with SSQ and VRSQ.Only the total scores of both CSQ-VR versions met the criteria for detecting motor-speed decline.
- 3.2. Detection of Temporary Decline due to Cybersickness: Vestibular/Disorientation scores were the strongest CSQ-VR sub-scores for detecting temporary reaction- or motor-speed decline.The paper-and-pencil Vestibular score performed best, closely followed by the VR-version Vestibular score.
- 3.3. Mixed Model Regression Analysis: Pupil size significantly predicted cybersickness intensity, with smaller pupils associated with greater intensity and the model explaining 50% of variance.The mixed model included pupil size as a fixed effect and participants as random effects.
4. Discussion
The CSQ-VR showed strong validity and reliability, outperforming SSQ and VRSQ in measuring cybersickness and detecting temporary performance declines. Its VR version also supports repeated assessment during immersion, although the study’s sample and performance measures limit generalizability.
- Reliability and validity: Both CSQ-VR versions were highly reliable and valid, with robust correlations supporting convergent and construct validity across cybersickness symptom domains.Correlations were observed for total scores and corresponding Nausea, Vestibular, and Oculomotor sub-scores.
- Comparison with SSQ and VRSQ: The CSQ-VR’s shorter six-question design covers nausea, disorientation, and oculomotor symptoms, whereas SSQ and VRSQ retain structural and scoring limitations.CSQ-VR uses a 7-point scale with responses combining numbers and textual intensity labels.
- Performance detection: CSQ-VR total scores had superior psychometric properties to SSQ and VRSQ scores for detecting temporary declines in reaction speed and motor speed.Only CSQ-VR total scores met the reported ROC-AUC criteria for motor-speed decline and showed the best sensitivity and specificity.
- Repeated assessment: Because cybersickness intensity can change during exposure, repeated assessment can identify specific observations affected by cybersickness rather than excluding an entire participant’s data.The VR version can assess symptoms while the user remains immersed and may help identify compromised performance observations.
- Limitations and future studies: The study’s young-adult sample, symptom-related exclusion criterion, and limited working-memory and psychomotor tests constrain conclusions about broader populations and abilities.Future work should include more age-diverse participants, stronger symptom experiences, and more complex cognitive and fine-motor tasks.
5. Conclusions
The study concludes that both CSQ-VR versions are short, valid, and reliable measures with superior psychometric properties to SSQ and VRSQ. The VR version additionally enables assessment during immersion and benefits from pupil-size measurement as a predictor of cybersickness.
- Conclusions: CSQ-VR is a short, valid, and reliable cybersickness measure with superior psychometric properties to SSQ and VRSQ.
- Conclusions: The paper-and-pencil and VR versions were highly sensitive and specific for detecting temporary performance declines modulated by cybersickness.
- Conclusions: The VR version permits cybersickness assessment while users are immersed in the virtual environment.
- Conclusions: Pupil diameter predicted the presence and intensity of cybersickness, supporting pupillometry as a potential VR biomarker.