Source-linked AI summary
Ten Years Later: Replicating Two Color Discrimination Studies
Shadmaan Hye, Andrew M. McNutt, Katherine E. Isaacs
TL;DR
Visualization guidelines rely on perceptual color-discrimination findings that have rarely been replicated, and it remains unclear whether regular color practice affects discrimination. This paper replicates two studies from 10 years earlier and finds that their perceptual effects persist, while reported color practice shows no significant association with discrimination.
Problem
Few perceptual studies informing visualization color guidelines have been replicated, and it remains unclear whether color-related practice affects color discrimination.
Method
The authors replicated two crowdsourced color-discrimination studies with new implementations and participants, adding questionnaires about self-reported color-related practices.
Results
Across both experiments, the original perceptual effects were reproduced, while participants with and without reported color-related practices showed no significant discrimination differences.
Takeaways & Limitations
The findings suggest that visualization color-discrimination guidance may generalize across viewers regardless of their regular color practice.
Takeaways & Limitations
Remote experiments used uncontrolled display settings, calibration, resolutions, and viewing setups, which may explain discrepancies from the original studies.
Abstract
from arXiv · showhide
Color discrimination is a fundamental aspect of visualization as it influences how people interpret visual encodings. Many visualization guidelines are informed by perceptual studies, yet relatively few have been replicated. Acknowledging that the interaction between human perception, visual tasks, and display technology can change over time, we replicate two crowdsourced color discrimination studies conducted 10 years earlier. Specifically, we replicated a visualization-focused color discrimination task (N=144) and a more general perceptual discrimination task (N=394). In both studies, our results reproduced the original perceptual effects. We further use the replication to investigate whether color-related practice influences color discrimination. Specifically, we extended our replication studies by adding questions about participants' engagement with color practices. We then examined whether diverse color-related practices (e.g., artistic hobbies, knowledge of color theory, and cosmetic makeup use) influenced color discrimination. We found no significant difference between participants who reported engaging in color-related practices and those who did not, suggesting that design guidance regarding color discrimination may generalize across viewers regardless of their regular color practice.
1 INTRODUCTION
The paper replicates two color-discrimination studies from 10 years earlier and reproduces their original perceptual effects. It also finds no measurable effect of reported color-related practice on color discrimination.
- Motivation: Color discriminability determines how much visual information color encodings can convey and informs visualization guidelines and palette tools.Prior work examined factors including lightness variation, color axes, and stimulus properties.
- Replication studies: Experiment 1 replicated a scatterplot study of perceptual color differences across mark sizes (N = 144).The original task and analyses were reimplemented on reVISit with participants recruited through Prolific.
- Replication studies: Experiment 2 replicated a just-noticeable difference study comparing an oriented shape with its background (N = 394).The replication preserved the original task and analyses while using a new platform and participant pool.
- Replication studies: Across both experiments, the replications reproduced the original perceptual effects.The studies preserved the original tasks and analyses while changing the platform and participant pool.
- Color-related practice: Participants reporting artistic hobbies, color-theory knowledge, cosmetic makeup use, or other color-related practices showed no measurable effect on color discrimination.Questionnaires partitioned participants according to whether they reported engaging in such practices.
2 RELATED WORKS
Prior work treats color discriminability as central to reliable visual encoding and shows that perceived color differences depend on both viewing context and observer experience. Perceptual-learning research further indicates that repeated discrimination training can improve performance, often for trained hues specifically.
- Color discriminability in visualization: Reliable color distinction is necessary for using color effectively as a visual encoding in data visualization.
- Color discriminability in visualization: Early visualization research framed color selection as a perceptual design problem involving color distance, linear separation, and color category.
- Color discriminability in visualization: Standard color-difference models such as CIELAB can misestimate perceived differences when colors appear in realistic viewing conditions.
- Color-related experience and perceptual learning: Population-level differences in color naming and categorization, along with contextual effects, show that color judgments can vary across observers and conditions.
- Color-related experience and perceptual learning: Repeated color-discrimination training can improve perceptual performance, although gains are often specific to trained hues.
3 EXPERIMENT ONE: DISCRIMINATION IN SCATTERPLOTS
Experiment One replicated the scatterplot color-discrimination effects that larger marks and greater color differences improve performance. Self-reported color-related practice showed no significant association with discrimination ability, although replication patterns were noisier for the smallest marks.
- Method: The study replicated Szafir’s scatterplot experiment, measuring how mark size relates to color discriminability.Stimuli used two colored target marks among gray distractors, with diameters ranging from 0.25° to 2.0° of visual angle.
- Method: Participants judged whether two colored marks were the same or different in a binary forced-choice task after color-vision screening and tutorial training.The sample included N = 144 participants recruited from Prolific in the United States.
- Results: Participants discriminated colors better as target differences increased and mark sizes grew, reproducing the original study’s principal findings.Within each CIELAB axis, 50% thresholds generally decreased as point diameter increased, indicating easier detection of small color differences with larger marks.
- Results: p = 0.97: participant-level accuracy showed no significant difference between Known Color Practice and Unknown Color Practice groups.The groups were analyzed using Welch’s two-sample t-test after participants reported color-related hobbies, color-theory education, color knowledge, and makeup use.
- Replication Differences: Replication differences were largest on L* (dz = −1.01) and a* (dz = −0.95), while the smallest mark size, 0.25◦, produced noisier thresholds.Larger mark sizes showed the closest convergence with Szafir’s results; the study also lacked controlled remote display settings, calibration, and resolution.
4 EXPERIMENT TWO: DISCRIMINATION IN LANDOLT-C
Experiment Two replicated Reinecke et al.’s web-based Landolt-C color differentiation task with 394 participants and reproduced its main perceptual patterns. Color-practice groups showed no significant accuracy difference, although the replication’s median discrimination volume was larger than the original’s.
- Method: The experiment replicated Reinecke et al.’s web-based color differentiation test as a general perceptual task involving visual-orientation identification.Participants identified the direction of a colored Landolt-C across eight cardinal directions while an iterated binary search estimated discrimination thresholds.
- Participants: 394 participants remained after six exclusions from the 400-person Prolific sample.The recruited sample comprised 200 males and 200 females; approximately 375 participants were required for 85% power.
- Results: Approximately 63% of participants produced relatively small discrimination volumes, while approximately 37% produced larger volumes, yielding a positively skewed distribution.Smaller volumes indicate higher sensitivity, whereas larger volumes indicate lower sensitivity.
- Results: Trial accuracy did not differ significantly between Known Color Practice and Unknown Color Practice groups (p = 0.674).Welch’s t-tests found no statistically significant group differences, and both groups had comparable distributions.
- Replication Differences: The replication reproduced the original distribution’s overall shape, but participants’ median ellipsoid volume was larger than in the original study.The larger median suggests participants often required larger color differences to discriminate colors; differences may reflect samples, displays, or viewing environments.
- Limitations: Recruitment and sample-size differences may have contributed to the shift in discrimination threshold.The original used a large LabintheWild sample, whereas this study used a screened, paid Prolific sample two orders of magnitude smaller.
5 CONCLUSION
The study replicated two color-discrimination studies 10 years later, reproduced their core perceptual patterns, and examined whether self-reported color-related practices affect discrimination. No significant practice-related differences emerged, while display and viewing conditions appeared more influential.
- Replication and extension: 10 years after the original publications, the replications reproduced the studies’ core perceptual patterns using new implementations and participants.The replication also extended the original studies by investigating self-reported color-related practice.
- Replication discrepancies: The key discrepancies involved distinguishing the smallest scatterplot marks in Exp 1 and increased ellipsoid volumes in Exp 2 on uncontrolled monitors.The differences may reflect changes over 10 years in displays, hardware, screen resolutions, and participant populations.
- Color-related practice: No significant differences in color-discrimination performance were found between participants reporting color practices and those reporting none across both experiments.The null result suggests regular color practice might not significantly change the perceptual thresholds evaluated by these tasks.
- Implications for replication: Display variation, adaptive brightness, and viewing settings appeared to affect observed thresholds more than individual color practices in both replications.The findings point to limited collection of display-hardware and viewing-condition information in visualization studies as an opportunity for future replication research.