PaperlayerView exact revision on arXiv

Source-linked AI summary

Ask the Experts: What Should Be on an IoT Privacy and Security Label?

Pardis Emami-Naeini, Yuvraj Agarwal, Lorrie Faith Cranor, Hanan Hibshi

arXiv:2002.04631v1cs.CYcs.CRcs.HC

TL;DR

Consumers lack readily available privacy and security information for IoT purchase decisions, while proposed labels provide little guidance on what content to include. The paper uses a three-round Delphi study with 22 experts and interviews with 15 IoT consumers to develop and assess a layered label prototype. Experts identified and organized label factors, and consumers understood the primary layer while still wanting unfamiliar secondary information included.

  • Problem

    Privacy and security information about IoT devices is not readily available to consumers making purchase decisions, and proposed labels do not specify their content.

  • Method

    The study combined a three-round expert Delphi process with 22 experts, consumer interviews, and iterative prototype label design.

  • Results

    Experts identified label factors and distributed them across primary and secondary layers; all 15 consumers understood the primary layer and wanted additional secondary information included.

  • Takeaways & Limitations

    A layered IoT privacy and security label can present glanceable information while retaining additional factors online for consumers seeking more detail.

  • Takeaways & Limitations

    The small-scale qualitative consumer study requires additional large-scale iterative design and testing to refine and validate the label.

Abstract

from arXiv · show

Information about the privacy and security of Internet of Things (IoT) devices is not readily available to consumers who want to consider it before making purchase decisions. While legislators have proposed adding succinct, consumer accessible, labels, they do not provide guidance on the content of these labels. In this paper, we report on the results of a series of interviews and surveys with privacy and security experts, as well as consumers, where we explore and test the design space of the content to include on an IoT privacy and security label. We conduct an expert elicitation study by following a three-round Delphi process with 22 privacy and security experts to identify the factors that experts believed are important for consumers when comparing the privacy and security of IoT devices to inform their purchase decisions. Based on how critical experts believed each factor is in conveying risk to consumers, we distributed these factors across two layers---a primary layer to display on the product package itself or prominently on a website, and a secondary layer available online through a web link or a QR code. We report on the experts' rationale and arguments used to support their choice of factors. Moreover, to study how consumers would perceive the privacy and security information specified by experts, we conducted a series of semi-structured interviews with 15 participants, who had purchased at least one IoT device (smart home device or wearable). Based on the results of our expert elicitation and consumer studies, we propose a prototype privacy and security label to help consumers make more informed IoT-related purchase decisions.

I. INTRODUCTION

Consumers want privacy and security information when purchasing IoT devices, but accessible information and label-content guidance remain limited. This paper uses expert and consumer studies to identify, layer, and prototype label content.

  • Motivation: IoT incidents and opaque data practices have exposed privacy and security concerns around connected devices.Reported examples include private audio disclosures, hacked baby monitors, and manufacturers failing to disclose device microphones or human audio review.
  • Motivation: 92% of surveyed IoT consumers were concerned about privacy and wanted control over personal information collected by smart devices.
  • Research gap: Consumers seek privacy and security information at purchase, yet little public information is available to inform IoT purchase decisions.
  • Approach: The study solicited expert opinions to determine which privacy and security information is most relevant and digestible for consumers with limited attention.
  • Findings: Experts identified factors for consumer information and additional factors intended mainly to support accountability for companies.
  • Label design: The authors partitioned important factors into primary glanceable content and secondary information requiring more space, then proposed a prototype label tested with 15 consumers.

A. Consumers’ Privacy and Security Concerns and Preferences

Prior work documents substantial IoT privacy and security concerns alongside limited pre-purchase information. This paper addresses that gap by combining expert elicitation with consumer interviews to develop a label covering both privacy and security.

  • Consumer concerns: Consumers’ IoT privacy concerns vary with the type, purpose, and retention of collected data.
  • Consumer concerns: Limited privacy and security information before purchase is identified as one possible explanation for the gap between consumers’ concerns and mitigating actions.
  • Label research: Privacy labels can help website users find important information faster and more accurately than traditional privacy policies.
  • Prior recommendations: Existing industry and policy reports largely emphasize security mechanisms, with few references to data privacy considerations.
  • This study: The study elicited label content from 22 experts through interviews and two surveys, then iterated the design through 15 non-expert consumer interviews.

2) Delphi Method:

The Delphi method structured expert input through anonymized, sequential rounds that shared summarized prior responses. Here, interviews generated candidate factors and two surveys examined their inclusion, layer placement, and rationale.

  • Delphi method: A Delphi process uses sequential questionnaires, summarized feedback, and anonymized responses to solicit judgments and develop expert consensus.
  • Study design: The study implemented three rounds consisting of expert interviews followed by two surveys.
  • Expert interviews: Open-ended interviews asked experts to define IoT privacy and security and propose information for an IoT label.
  • Survey rounds: The first survey collected experts’ rationales for including or excluding an extensive list of privacy, security, and general factors.
  • Survey rounds: The second survey presented prior reasons, asked experts to rate factor inclusion, and elicited layer placement and category rationales.
  • Label organization: Experts also assessed whether privacy and security should be separated or merged on the label.

6) Data Analysis:

The analysis combined iterative thematic coding, author discussion, and expert member checking, then used the resulting factors to design consumer-facing label prototypes and interview materials.

  • Expert data analysis: Thematic analysis summarized approximately 22 hours of expert interview recordings qualitatively and inductively.
  • Expert data analysis: The primary coder progressed from transcript notes and initial codes to broader themes, which authors reviewed and consolidated through discussion.
  • Iterative validation: The researchers revised themes after coding open-ended survey responses and presented them to experts in the second survey.
  • Expert findings: Saturation in new factors was reached after interviewing 20 experts, with no additional factors emerging in later interviews or surveys.
  • Consumer study design: Expert-study results informed primary and secondary labels placed on fictitious security-camera boxes and an online shopping website.
  • Consumer study design: Consumer participants were adults who had purchased at least one smart home or smart personal device and completed one-hour interviews.
  • Consumer study design: Participants examined labeled and unlabeled hypothetical security-camera boxes and discussed what the labels conveyed about privacy and security.

3) Risk Communication in Comparative Purchase Process:

The consumer study examined how participants compared IoT products using layered privacy and security labels, including their understanding of primary and secondary information. Interviews also elicited perceived risks, usefulness, and preferred label organization.

  • Comparative purchase process: Participants compared two same-price, same-feature security cameras with different privacy and security information to discuss risk and purchase choice.
  • Layered label access: The interviews explored reactions to secondary information accessed through a QR code or URL and the trade-offs between one-layer and two-layer labels.
  • Risk interpretation: Participants discussed concerns about the rejected product, risky privacy and security factors, exposed risks, and possible product improvements.
  • Label organization and behavior: The study also asked about section organization, misplaced factors, and online versus in-store purchase behavior.
  • Analysis: Researchers transcribed about 15 hours of interviews and analyzed them using structural coding with four structural codes and 13 subcodes.
  • Ethics: The study received IRB approval, and participants provided informed consent for participation, recording, and transcription.
  • Scope: The consumer study was small-scale and qualitative, so additional large-scale iterative design and testing is needed to refine and validate the label.

IV. RESULTS

The results combine a 22-expert elicitation with consumer testing to identify, explain, and assess privacy and security label content. Experts distinguished how privacy and security are understood, while consumers evaluated proposed organization and information.

  • Expert elicitation: 22 experts generated 47 privacy, security, and general factors for possible inclusion on an IoT label.
  • Expert rationale: 17 experts completed the first survey, producing an average of seven reasons for or against including each factor and two or three primary thematic reasons per factor.
  • Layer recommendations: 21 experts rated factor placement and yielded 12 factors recommended for the primary layer and 13 for the secondary layer.
  • Consumer testing: The authors used expert findings and discussions to design prototype labels, then iteratively improved them through interviews with 15 non-expert consumers.
  • Privacy and security concepts: Nearly all experts linked security definitions to confidentiality, integrity, and availability, while privacy definitions varied across experts.
  • Consumer understandability: 15/22 experts considered security information less tangible and understandable for consumers because it relies on technical mechanisms.
  • Measurability: 19/22 experts reported that security practices are easier to measure and assess than privacy practices because security is more objective and privacy is more subjective and context dependent.
  • Label consequences and organization: Experts associated labels with greater accountability and transparency, while consumers preferred the proposed separation into security mechanisms, data practices, and general information.

B. Factors to Include in the IoT Label

Experts recommended a primary layer containing highly relevant, glanceable privacy and security factors, but the authors adjusted several placements based on consumer understanding and practical implications.

  • 1) Primary Layer: 12 factors were recommended for the primary layer, including independent privacy and security ratings, update coverage, collected-data type, sensors, signed updates, actuations, and default-password status.
  • 1) Primary Layer: Experts also prioritized data-sharing frequency, data granularity, and access control because these factors convey critical information and inform purchase decisions.
  • 1) Primary Layer: The authors excluded the device warranty period because it has few, if any, privacy, security, or safety implications.
  • 1) Primary Layer: All consumers understood the primary-layer information and connected the device expiration date with its security-update lifetime, although one questioned companies’ ability to sustain service.
  • 1) Primary Layer: The authors moved physical actuation and data-sharing frequency to the secondary layer because their privacy or security implications were less direct or less understood.

2) Secondary Layer:

The secondary layer accommodates detailed or less critical information, while the authors moved selected factors to the primary layer when consumers needed them for privacy and security decisions.

  • 2) Secondary Layer: 13 factors were recommended for the secondary layer, including data-collection purpose, inferred information, storage location, controls, collection frequency, and handling of children’s data.
  • 2) Secondary Layer: Experts mainly preferred the secondary layer when a factor required detailed information to convey risk or was not critical to device privacy and security.
  • 2) Secondary Layer: The authors moved firmware-update date, data-collection purpose, and storage location to the primary layer instead.
  • 2) Secondary Layer: Firmware version and date were recommended for both layers because updates occur frequently while consumers need to know which version the label covers.
  • 2) Secondary Layer: The authors placed data-collection purpose on the primary layer because consumers consider it important for purchase decisions, despite experts’ concerns about fitting all purposes there.
  • 2) Secondary Layer: The authors placed storage location on the primary layer because local versus cloud storage can have different privacy and security implications, which consumers could discuss as a trade-off.

3) Factors with no Specific Layer:

Experts identified several factors as important but disagreed about whether they belonged on the primary or secondary layer. The authors placed some factors on the secondary layer based on relevance, updateability, and consumer feedback.

  • Four factors divided expert opinion between the primary and secondary layers: data recipients, data purchasers, smart-feature functionality, and parental controls.At least four of seven experts supported including each factor, but opinions split over its placement.
  • Parental controls and functionality when smart features are disabled were assigned to the secondary layer because they were not directly privacy or security factors.
  • Data-sharing and data-sale recipients were placed on the secondary layer because experts noted that these relationships may change over time and require easier updating.
  • Six factors that most experts opposed including were device compatibility, bills of materials, accompanying apps, bug-bounty programs, incorporation details, and Consumer Reports ratings.
  • Experts commonly rejected factors because they lacked privacy-security relevance or could not adequately convey risk, including bug-bounty programs and Consumer Reports ratings.

C. Attitudes toward Labels and Layered Design

Consumers valued point-of-sale privacy and security information and generally supported layered labels, although some secondary-layer factors were difficult to understand or access. Participants still wanted most detailed information available.

  • All participants found privacy and security information difficult to locate before purchase and wanted a label available at the point of sale.Participants primarily wanted the label to be as informed as possible when purchasing smart devices.
  • Most participants supported layered labels because two layers could present more information than a single label while enabling further insights into manufacturer practices.11 of 15 participants expressed positive attitudes toward the layered design.
  • Four participants considered layered labels inconvenient in stores, particularly for older shoppers who might struggle to scan QR codes.
  • All participants understood the primary-layer factors, but identifiable data was not associated with privacy risk in the security-camera context.Further testing was suggested for interactions between data-collection purpose and data granularity.
  • Participants wanted most secondary-layer factors even when they did not understand them, because they wanted to be informed and could search unfamiliar terms online.One participant specifically described looking up TCP and UDP to understand their implications.
  • Consumers recognized that detailed secondary-layer information could also support experts who investigate questionable practices and raise issues publicly or with regulators.

D. Prototype Privacy and Security Label

The prototype uses a two-layer label for a hypothetical smart security camera, combining glanceable information with expandable online details. Experts favored ratings for comparison, but the paper identifies unresolved design and incentive problems.

  • The prototype was informed by expert elicitation, consumer studies, and IoT standardization efforts, with primary and secondary layers and expandable secondary details.
  • Experts recommended privacy and security star ratings mainly to help consumers compare devices more easily.
  • The authors excluded third-party assessments because no organization was evaluating a broad range of IoT devices at scale.Consumers nevertheless liked assessments from trustworthy organizations.
  • Experts identified a rating-scale trade-off: granular scales could differentiate manufacturers, while consumers might struggle to distinguish many ratings.
  • Experts warned that companies might game star ratings to obtain full scores, potentially making ratings less discriminative.
  • Multiple certification levels with a secure baseline were discussed as an alternative to star ratings, but minimum-level certification could reduce incentives to pursue higher levels.

1) Digital Standard:

The Digital Standard organizes IoT evaluation across security, privacy, ownership, and governance, while the proposed label covers many of these factors. The paper notes that detailed information and third-party assessment remain important for interpretation.

  • Consumer Reports’ Digital Standard evaluates IoT products across security, privacy, ownership, and governance and compliance.
  • Its security category includes build quality, data security, and personal safety, with the proposed label covering these areas through related factors.
  • Its privacy category includes user controls, data use and sharing, data retention, and overreach, with overreach omitted from the proposed label.The paper suggests overreach may warrant third-party assessment rather than self-reporting.
  • The proposed label includes factors spanning the YourThings rubric, but some areas require third-party evaluation for complete scoring.The paper specifically notes incomplete assessment of TLS configuration and network-communication security.
  • Concise YourThings scores support comparison, but users may need detailed information tailored to their specific needs.Automatic-update penalties may not reflect cases where poorly timed updates interfere with critical device functions.
  • UL’s seven-category certification could provide a concise security assessment alongside the label’s more detailed privacy information, although compliance requires third-party evaluation.
  • The conclusion calls for additional contextual user testing, consumer glossaries, and manufacturer implementation guidance to improve label use and interpretation.

APPENDIX A EXPERT ELICITATION STUDY

The study introduced a layered IoT privacy and security label to experts, elicited their preferences and rationales through surveys, and examined how consumers interpreted and used the resulting information.

  • Experts were introduced to an IoT privacy and security label modeled on a food nutrition label.
  • The label used primary and secondary layers to separate the most important or understandable information from additional information.
  • The first survey asked experts to review interview-derived factors, indicate whether each belonged on the label, and explain their preferences.
  • Experts evaluated factors and arguments concerning consumer education, informed purchasing, and accountability for IoT companies.
  • The second survey presented common arguments from the first survey and asked experts to reconsider each decision and provide additional arguments.
  • Consumer interviews examined what participants could learn from labeled smart-camera packaging, how they compared devices, and whether secondary-layer information was understandable and useful.
Loading 2002.04631v1…