PaperlayerView exact revision on arXiv

Source-linked AI summary

What Do We Mean by "Accessibility Research"? A Literature Survey of Accessibility Papers in CHI and ASSETS from 1994 to 2019

Kelly Mack, Emma McDonnell, Dhruv Jain, Lucy Lu Wang, Jon E. Froehlich, Leah Findlater

arXiv:2101.04271v4cs.HC

TL;DR

Accessibility research lacked a broader survey of work across its major venues. This paper analyzes accessibility papers from CHI and ASSETS using qualitative and quantitative strategies, finding disproportionate focus alongside growth in the field and identifying areas for improvement.

  • Problem

    No broader survey existed of accessibility research appearing at ACM CHI and ASSETS.

  • Method

    The paper combines qualitative and quantitative analysis strategies to study accessibility research across CHI and ASSETS.

  • Results

    Accessibility research is increasing, even outpacing CHI growth in the past decade, while BLV users receive disproportionate attention and disabled and older adult groups have a median sample size of 13.

  • Takeaways & Limitations

    The findings identify beneficial practices, areas for improvement, and the need for better representation in the accessibility research community.

  • Takeaways & Limitations

    Varied language across papers affected the codebook and constrained the conclusions that could be drawn.

Abstract

from arXiv · show

Accessibility research has grown substantially in the past few decades, yet there has been no literature review of the field. To understand current and historical trends, we created and analyzed a dataset of accessibility papers appearing at CHI and ASSETS since ASSETS' founding in 1994. We qualitatively coded areas of focus and methodological decisions for the past 10 years (2010-2019, N=506 papers), and analyzed paper counts and keywords over the full 26 years (N=836 papers). Our findings highlight areas that have received disproportionate attention and those that are underserved--for example, over 43% of papers in the past 10 years are on accessibility for blind and low vision people. We also capture common study characteristics, such as the roles of disabled and nondisabled participants as well as sample sizes (e.g., a median of 13 for participant groups with disabilities and older adults). We close by critically reflecting on gaps in the literature and offering guidance for future work in the field.

1 INTRODUCTION

This paper surveys accessibility research at CHI and ASSETS from 1994 to 2019 to characterize its communities, methods, trends, and underserved areas. It combines qualitative coding of 506 papers from 2010–2019 with analysis of paper counts and keywords across 836 papers.

  • Research gap and questions: No broader survey of accessibility literature existed beyond reviews of specific areas such as autism and visual impairment.The paper addresses this gap by examining research at CHI and ASSETS from 1994 to 2019.
  • Research gap and questions: The study asks whom accessibility research focuses on, which goals and methods it uses, and how the field has evolved over time.The questions include sample size, study location, and use of participatory methods.
  • Approach: The authors created a dataset of 836 CHI and ASSETS papers, manually coding 506 papers from 2010–2019 and analyzing counts and keywords across 1994–2019.Manual coding covered research foci, contribution types, methodological decisions, and disabled and nondisabled participant roles.
  • Main findings: Over 43% of papers in the past decade addressed accessibility for blind and low vision people, indicating disproportionate attention to this community.The paper also identifies areas that are underserved.
  • Main findings: Accessibility research increased over the full period and outpaced CHI growth during the past decade, while blind and low vision work remained consistently popular.The study also identifies pairings between research issues and communities, including communication for d/Deaf and hard of hearing people and behavior change for autism-focused work.
  • Contributions: The paper contributes trend characterizations, historical patterns, reflections and recommendations, and an open-source dataset containing 506 coded papers and metadata for 836 papers.These resources are intended to support future meta-analyses by the HCI accessibility community.

2 BACKGROUND AND RELATED WORK

The background frames accessibility through disability history and the social model, then situates this survey among narrower literature reviews and related analyses. Prior work covers specific populations, application areas, methodological practices, and critical discourse, while this paper extends comparison across accessibility communities.

  • History and conceptual foundations: The social model shifts attention from individual impairment to social and environmental conditions, distinguishing impairment as biological or physical from disability as socially produced.This perspective builds on disability-rights critiques of disability as a socially and culturally constructed form of oppression.
  • History and conceptual foundations: US policy milestones, including the Rehabilitation Act, Section 508, and the ADA, helped establish accessible technology as a recognized area of concern.The ADA extended protections and required public accommodations to provide appropriate aids or services, although its application to computing technology has been inconsistent.
  • Research venues: ASSETS was founded in 1994 to study accessible digital technologies and develop solutions to real-world accessibility problems, while accessibility later became a key CHI area.The venues cover design, evaluation, use, and education related to computing for people with disabilities and older adults.
  • Related literature: Existing surveys examine narrower domains, including visual accessibility, autism, aging, children, and design with older adults, rather than accessibility literature as a whole.Critical discourse analyses have identified corrective and othering approaches in autism research and deficit-oriented discourses in aging research.
  • Related literature: Prior quantitative analyses report web browsing as the most common application area at 24.2% of papers, followed by education and mobility at 14.6% each.That related work found that 70.2% of papers included at least one quantitative empirical evaluation and that formative studies were becoming more common.
  • Positioning of the survey: This survey examines proxies, participatory and co-design methods, and participant inclusion to compare representation across user populations.It complements prior critical disability scholarship addressing representation, ableism, aging, and interdependence in assistive technology.

3 METHOD

The authors survey accessibility papers from CHI and ASSETS using qualitative coding of 2010–2019 papers and quantitative analysis of 1994–2019 metadata, counts, and keywords. They construct and refine datasets while acknowledging search, metadata, and selection limitations.

  • The review qualitatively analyzes 506 papers from 2010–2019 and examines quantitative trends across 836 papers from 1994–2019.
  • The final dataset includes only short and long technical papers from CHI and ASSETS, excluding posters, keynotes, and similar materials.
  • CHI papers were filtered using accessibility-related terms in titles, abstracts, or author keywords, while ASSETS papers were treated as accessibility papers by definition.
  • The authors note that ACM Digital Library query logic may have changed, some papers may be missing, and the dataset reflects the researchers’ own biases and commitments.
  • The 2010–2019 dataset contains 215 relevant CHI papers and 291 ASSETS papers after candidate screening and consensus review.
  • Researchers manually coded research foci, methodological decisions, and disabled and nondisabled participant roles using a codebook developed through iterative coding and consensus.

4 RESULTS

The results characterize where accessibility research invests attention and what research problems and contribution types it addresses. The analysis combines a recent coded snapshot with longer-term temporal and keyword trends.

  • The authors complement qualitative findings from 2010–2019 with programmatic analysis of temporal trends across 1994–2019.
  • The study summarizes communities, research problems, and contribution types represented in accessibility research.

4.1 Communities of Focus, Research Problems and Contributions

Accessibility research disproportionately focuses on blind and low vision people, while research aims and contribution types vary across communities. Most papers contribute empirical findings or artifacts, whereas methodological, dataset, and literature-survey contributions are uncommon.

  • Communities of Focus: 43.5% of papers focused on blind or low vision people, compared with 14.2% on motor or physical disabilities and 11.3% on deaf or hard-of-hearing people.
  • Communities of Focus: The remaining communities each accounted for under 10% of papers, including cognitive impairments, older adults, autism, and intellectual or developmental disabilities.
  • Communities of Focus: 94.5% of blind or low vision papers had a singular community focus, whereas 42.9% of intellectual or developmental disability papers co-occurred with another code.
  • Research Problems: 36.8% of papers aimed to increase digital access through technology innovation, followed by understanding user needs at 27.5% and increasing physical access at 20.8%.
  • Research Problems: 64.9% of deaf or hard-of-hearing papers addressed communication, while autism papers most commonly addressed behavior change and understanding users.
  • Contributions: Empirical contributions appeared in 60.3% of papers and artifact contributions in 55.5%, while literature surveys occurred in only 0.6%.
  • Contributions: Methodological contributions comprised 3.2% of papers, dataset contributions 1.4%, and literature surveys three papers.

4.2 Prevailing Research Methods

Among 506 accessibility papers with user studies, research most often used interviews, usability testing, and controlled experiments, while participatory design appeared in a smaller share. Studies commonly included disabled or older adult participants, but participant-group sample sizes were modest and study locations varied.

  • Overall methods: 94.3% (N=477) of 506 papers included user studies.The remaining papers used existing-content analyses, algorithmic or system analyses, literature reviews, or theoretical contributions.
  • User-study methods: Interviews (42.1%, N=201), usability testing (41.7%, N=199), and controlled experiments (34.6%, N=165) were the most common user-study methods.56.4% (N=269) used multiple user-study methods.
  • Study locations: 34.1% (N=138) of papers with specified locations studied participants at home, work, or frequently visited places, compared with 32.1% (N=130) at laboratory or on-campus sites.Remote participation accounted for 24.2% (N=98), and non-campus locations were sometimes selected to improve ecological validity, recruitment, or accessibility.
  • Participants: 90.1% (N=430/477) of user-study papers included participants with disabilities and/or older adults.Among the 47 papers without them, 38.3% (N=18/47) included specialists and/or caregivers in proxy roles.
  • Sample sizes: Participant groups with disabilities or older adults had a median sample size of 13, with medians ranging from 9 for autism to 28 for DHH groups.Overall paper-level participant counts had a median of 18, while caregiver and specialist groups had medians of 7.5 and 7.0.
  • Participatory methods: 10.3% (N=49/477) of user-study papers identified participatory design or co-design, and 95.9% (N=47/49) included disabled participants and/or older adults.Participatory-design studies ranged from one to 20 sessions and often included specialists, therapists, teachers, or caregivers.

4.3 The Role of Nondisabled Participants, Caregivers and Specialists

Nondisabled participants, caregivers, and specialists appeared mainly as stakeholders, proxies, comparison groups, or controls alongside research centered on people with disabilities. Their use varied by community and study purpose, with proxies especially common in autism, cognitive impairment, and IDD research.

  • Proxy roles: Specialists and caregivers commonly served as proxies for disabled individuals’ perspectives, behaviors, or experiences with technologies.Proxies were most often specialists (65.8%, N=25/38) and/or caregivers (44.7%, N=17).
  • Proxy roles: 22.6% (N=7/31) of autism-focused studies used proxies, followed by cognitive impairment (17.4%, N=8/46) and IDD (21.4%, N=3/14).Reasons included domain expertise and communication difficulties; no DHH-focused paper used proxies.
  • Comparative roles: 13.6% (N=65) of papers compared disabled and nondisabled participants.These comparisons addressed differing needs or experiences and sometimes established a baseline or control group.
  • Comparative roles: Controlled experiments were used in 63.1% (N=41) of papers comparing disabled and nondisabled participants, with nondisabled participants often serving as controls.Other comparisons supported collaboration or evaluated equitable experiences across ability levels.

4.4 Historical Context: Programmatic Analysis of 26-year Temporal Trends

Across 1994–2019, accessibility research expanded in volume and relative prominence at CHI and ASSETS, while its communities, technologies, and methods diversified. Blind and low vision remained the most represented community, alongside newer technology and method trends.

  • Paper counts: Accessibility papers increased from 22 in 1994 to 95 in 2019, with CHI accessibility papers overtaking ASSETS papers in 2018.Accessibility-paper growth also outpaced the growth of CHI itself.
  • Paper counts: 7.8% (55 of 702 papers) of CHI papers addressed accessibility in 2019, making “accessibility” the second-most common overall keyword.The proportion of CHI papers on accessibility had grown substantially over time.
  • Communities of focus: BLV received the highest proportion of community-focused attention, followed by motor/physical and DHH communities.Older adults and cognitive impairments appeared consistently from the earliest years, while autism, neurodiversity, color vision deficiency, and mental health appeared later.
  • Technology trends: Games, wearable computing, social computing, 3d printing and DIY, AR/MR/VR, and collaboration tools entered the top-20 technology keywords only during 2010–2019.These groups represented recent technology trends in accessibility and HCI more broadly.
  • Method trends: Interviews first appeared among method keywords in 2000–2004 and reached 10.9% in 2015–2019, while “usability” declined and “user experience” appeared after 2005.The keyword share for interviews was lower than the 42.1% of papers coded as employing interviews, and 34.1% of papers had no method keywords.

5 DISCUSSION

The survey reveals a sprawling accessibility field whose norms and research priorities are unevenly distributed across communities, problems, methods, and participant roles. It identifies underserved areas and methodological practices that warrant continued scrutiny and development.

  • Current Foci and Growth Opportunities in Accessibility Research: Blind and low vision people are by far the most common community of focus, while autism, IDD, cognitive impairment, mental health, and chronic illness receive less attention.The authors associate this skew with funding, public disability discourse, and the apparent concreteness of visual accessibility problems, while identifying underrepresented communities as opportunities for growth.
  • Current Foci and Growth Opportunities in Accessibility Research: Only 7.1% of papers included multiple communities of focus, limiting attention to intersections across disability communities.The survey also calls for greater consideration of intersections between disabled identities and race or gender.
  • Current Foci and Growth Opportunities in Accessibility Research: Communication appeared in 64.9% of DHH papers, whereas behavior change appeared in 32.3% of autism papers and 35.7% of IDD papers.The authors urge researchers to examine why particular problems are popular within some communities, especially where community members question behavior-change goals.
  • Engaging with Users in Accessibility Work: Participatory design appeared in 10% of user-study papers, but some papers used the term for only a single participant session.The authors encourage careful discussion of tradeoffs and tensions in participatory methods.
  • Engaging with Users in Accessibility Work: A median of 13 disabled and older adult participants reflects recruitment challenges, but small samples are not necessarily limitations in accessibility research.The authors note that repeatedly sampling small populations can burden participants, and that some disabilities make participation difficult.
  • Engaging with Users in Accessibility Work: Case studies appeared in 4% of papers and methods requiring no user participation in 6%, despite their potential to reduce participant burden.Content analyses and online ethnographies are identified as promising lower-engagement approaches, while reviewers are encouraged not to dismiss papers solely for small samples.
  • Engaging with Users in Accessibility Work: Ability-based comparisons and proxy participation can be useful but risk reinforcing normative, ableist beliefs and require researchers to question whether alternatives were considered.The authors distinguish proxy participation from disabled participants speaking for themselves and recommend engaging with both where appropriate.

6 CONCLUSION

The authors combine qualitative and quantitative analyses to characterize accessibility research at CHI and ASSETS across ten and twenty-six years. They use the findings to identify beneficial practices, areas for improvement, and questions for the field’s future.

  • CONCLUSION: The study combines qualitative and quantitative analysis to examine accessibility research over the past decade and broader community changes over 26 years.The paper analyzes coded research practices alongside longer-term publication trends.
  • CONCLUSION: The authors identify beneficial community practices while calling for better representation and greater awareness of how ableism can enter research norms.They encourage researchers to use the survey’s data and ideas to reflect on the future of accessibility research.
Loading 2101.04271v4…