PaperlayerView exact revision on arXiv

Source-linked AI summary

Trustworthy AI in the Age of Pervasive Computing and Big Data

Abhishek Kumar, Tristan Braud, Sasu Tarkoma, Pan Hui

arXiv:2002.05657v1cs.CY

TL;DR

Pervasive data collection and AI use in sensitive domains raise ethical concerns about surveillance, bias, and trustworthiness. The paper frames trustworthy AI through ethics of algorithms, data, and practice, reviews design-stage approaches, and discusses their potential in a smart-city use case.

  • Problem

    Pervasive behavioural-data collection and AI use in sensitive fields raise concerns about surveillance, biased decisions, misuse, and opaque systems.

    Source: §I · p. 1, §III · p. 4

  • Method

    The paper frames trustworthy AI through ethics of algorithms, data, and practice, reviews requirements and research challenges, and examines design-stage techniques and a smart-city use case.

    Source: §I · pp. 1–2, §V · p. 5

  • Results

    The paper proposes a data-centric framework for trustworthy AI and discusses how existing techniques could support more trustworthy AI use in smart cities.

    Source: §V · p. 5

  • Takeaways & Limitations

    The smart-city discussion identifies federated, on-device learning as technically feasible for predicting charger locations while keeping users’ raw data from leaving vehicles.

    Source: §IV · p. 5

Abstract

from arXiv · show

The era of pervasive computing has resulted in countless devices that continuously monitor users and their environment, generating an abundance of user behavioural data. Such data may support improving the quality of service, but may also lead to adverse usages such as surveillance and advertisement. In parallel, Artificial Intelligence (AI) systems are being applied to sensitive fields such as healthcare, justice, or human resources, raising multiple concerns on the trustworthiness of such systems. Trust in AI systems is thus intrinsically linked to ethics, including the ethics of algorithms, the ethics of data, or the ethics of practice. In this paper, we formalise the requirements of trustworthy AI systems through an ethics perspective. We specifically focus on the aspects that can be integrated into the design and development of AI systems. After discussing the state of research and the remaining challenges, we show how a concrete use-case in smart cities can benefit from these methods.

I. INTRODUCTION

Pervasive sensing supplies behavioural data that can improve services but also raises risks of surveillance, bias, misuse, and opaque decisions in sensitive domains. The paper frames trustworthy AI through ethics of algorithms, data, and practice, focusing on design-stage issues and reviewing techniques and a smart-city use case.

I. INTRODUCTION

  • Pervasive sensing creates abundant behavioural data that can support service quality but also enable surveillance, while AI in sensitive domains faces risks of erroneous and unfair decisions.The introduction also describes biased training data, maliciously altered continuous-learning systems, questionable data use, and opaque models.Source: §I · p. 1
  • Trustworthy AI comprises ethics of algorithms, data, and practice, while this paper focuses on algorithm and data issues that can be addressed during system development.The authors present these as data-centric abstractions for ethical questions.Source: §I · p. 1
  • The paper reviews trustworthy-AI requirements, current research, and remaining challenges, then considers how a smart-city use case can benefit from these techniques.It notes that federated learning and pervasive computing resources support distributed systems addressing AI-ethics challenges.Source: §I · p. 2

II. ETHICS OF ALGORITHMS

The European Commission’s ethical guidelines organize trustworthy AI around four principles and seven core requirements. Human autonomy is supported through user agency, fundamental rights, and human oversight.

II. ETHICS OF ALGORITHMS

  • The European Commission guidelines build seven core requirements on four principles: human autonomy, prevention of harm, fairness, and explicability.The passage identifies the guidelines as released in 2018 by the High-Level Expert Group on AI.Source: §II · p. 2

1) Human agency and oversight:

  • Respect for human autonomy calls for AI to support user agency and fundamental rights while allowing human oversight.The passage also describes AI as an enabler of a democratic and equitable society.Source: 1) Human agency and oversight · p. 2

2) Technical Robustness and Safety:

Trustworthy AI requirements connect robustness and privacy to prevention of harm, and transparency to explicability. They address risk, data governance, and visibility into systems and their context.

2) Technical Robustness and Safety:

  • Technical robustness and safety require preventive risk management so systems behave reliably and minimize harm, including amid environmental changes or adversarial attacks.The passage links this requirement to prevention of harm.Source: 2) Technical Robustness and Safety · p. 2

3) Privacy and Data Governance:

  • Privacy protection requires data governance covering data quality, integrity, relevance, access protocols, and privacy-preserving processing.The passage links privacy to prevention of harm and identifies it as a fundamental right affected by pervasive data collection.Source: 3) Privacy and Data Governance · p. 2

4) Transparency:

  • Transparency, linked to explicability, concerns the data, AI system, and business models, and helps justify pervasive data collection and its benefits to users.Source: 4) Transparency · p. 2

5) Diversity, Non-discrimination and Fairness:

The requirements extend fairness to inclusion and equal treatment, broaden stakeholder concern to society and the environment, and connect accountability to responsibility for AI outcomes.

5) Diversity, Non-discrimination and Fairness:

6) Societal and Environmental Well-being:

  • Societal and environmental well-being calls for ecological responsibility and AI that benefits people, including future generations.The passage encourages research into AI solutions addressing global concerns.Source: 6) Societal and Environmental Well-being · p. 2

7) Accountability:

  • Accountability requires mechanisms for responsibility for AI systems and their outcomes before and after development, deployment, and use.The passage links accountability closely to fairness.Source: 7) Accountability · p. 2

B. Recent Research Towards Trustworthy AI System

Research on trustworthy AI addresses seven requirements, including human agency and oversight, and technical robustness and safety.

1) Human Agency and Oversight:

  • Human-in-the-Loop AI uses operator feedback, including active-learning labels, while Meaningful Human Control preserves intervention during design and operation.A proposed fairer paradigm repays legitimate knowledge owners when AI systems generate revenue; human operators may override unlawful decisions.Source: 1) Human Agency and Oversight · p. 2

2) Technical Robustness and Safety:

  • Defenses against poisoning, evasion, and model-stealing attacks use adversarial training, hiding model information, reducing complexity, or limiting transferability.These approaches aim to increase model robustness.Source: 2) Technical Robustness and Safety · p. 2
  • Reliable AI combines failure prevention, output monitoring, and maintenance, with checks for data, environment, model, and reporting errors.After deployment, unreliable outputs may be rejected using density and local-fit principles; maintenance costs compound development complexity.Source: 2) Technical Robustness and Safety · p. 2

3) Privacy and Data Governance:

Research on trustworthy AI covers privacy and data governance, transparency, and methods for discovering and preventing discrimination.

3) Privacy and Data Governance:

  • Privacy-by-design calls for data-owner consent and honoring the right to be forgotten; federated architectures train without centralizing users’ raw data.Federated learning and peer-to-peer machine learning share training updates instead; differential privacy and secure aggregation add protection, while efficient implementation of privacy clauses remains largely unexplored.Source: 3) Privacy and Data Governance · p. 3

4) Transparency:

  • For black-box models, explainability research uses input attribution, concept extraction, example influence, and distillation; transparency also requires disclosing AI interaction.Efficient ways to distinguish human-initiated from machine-initiated actions remain unexplored.Source: 4) Transparency · p. 3

5) Diversity, Non-discrimination and Fairness:

6) Societal and Environmental Well-being:

The section links societal well-being to social AI’s possible effects on agency and relationships, and environmental well-being to efficient, lower-carbon AI.

6) Societal and Environmental Well-being:

7) Accountability:

Accountability is addressed through data provenance and auditing, alongside principles for responsible data use and approaches to limiting or governing data collection.

7) Accountability:

  • Data provenance tracks data flows across technical and administrative boundaries, while auditing checks development compliance and supports deployment record integrity.Development verification can check regulations and company policies; blockchain or distributed ledgers can establish records without trusted third parties.Source: 7) Accountability · p. 3

III. ETHICS OF DATA AND ETHICS OF PRACTICE: TOWARDS TRUSTWORTHY AI SYSTEMS

  • Data ethics is framed as responsible, sustainable data use to sustain trust, with principles of human centrality, individual control, transparency, accountability, and equality.The passage cites Target’s pregnancy inference and Cambridge Analytica’s voter influence as abuses threatening the implicit user-service understanding.Source: §III · p. 4

A. Ethics of Data

  • Linking large datasets with auxiliary data can re-identify people; proposed directions minimize stored raw data or let users choose sharing and receive economic benefits.The latter approach aims to return control of personal data to users and challenge surveillance capitalism.Source: A. Ethics of Data · p. 4

1) Design Principles for Ethics of Data: •

The section describes data-ethics principles and personal data infrastructures that give users more control over data collection, use, and sharing.

1) Design Principles for Ethics of Data: •

  • Federated learning and P2P systems can avoid retaining raw user data by keeping it on the user’s device, going beyond GDPR’s minimum retention requirement.Organisations can also delete data early through auto-deletion.Source: 1) Design Principles for Ethics of… · p. 4
  • Contextual Integrity evaluates whether personal-information flows between agents are acceptable in a given context, even when the information was already public.The OkCupid dataset release illustrates that public availability does not mean users intended their data to be used for research.Source: 1) Design Principles for Ethics of… · p. 4

2) Personal Data Infrastructure for Ethics of Data: •

  • A personal databox acts as a physical gateway for data leaving the user’s control and lets users capture, index, store, and manage their data.Source: 2) Personal Data Infrastructure for… · p. 4
  • The Hub-of-All-Things infrastructure advocates that users receive explicit economic benefits for their data, including data that may lose value after an expiry window.It captures perishable data and brokers benefits from parties in its ecosystem.Source: 2) Personal Data Infrastructure for… · p. 4

B. Ethics of Practice

Ethics of practice concerns responsibility for the ethical use of data and models. The smart-city use-case considers privacy-preserving vehicle data analysis to plan electric-vehicle chargers.

B. Ethics of Practice

  • Ethics of practice calls for technical and ethical frameworks to guide data and model disclosure across stages and parties, including protection against privacy risks such as membership inference.Source: B. Ethics of Practice · p. 4

IV. USE-CASE: TRUSTWORTHY AI IN SMART CITIES

  • Smart-city sensing can expose private information, including users’ locations, to misuse by authorities or companies.Source: §IV · p. 4
  • The charger-planning scenario contrasts collecting and anonymising vehicle data with using an AI system to recommend charger numbers and locations.Source: §IV · p. 4, Figure 3 · p. 5
  • Federated learning can keep raw vehicle data on-device while predicting charger locations, but blockchain consensus may add energy costs and prevent real-time use.Differential privacy or homomorphic encryption can add privacy; algorithm design is still needed for fairness, explainability, and resilience to evasion attacks.Source: §IV · p. 5

V. CONCLUSION

The conclusion presents a data-centric framework for ethical questions in AI and data science, focused on aspects that can be integrated into system design and development.

V. CONCLUSION

  • The paper proposes a framework for trustworthy AI systems in an era of pervasive data collection and computing.Source: §V · p. 5
  • The framework organizes ethical questions around data, algorithms, and practice, while focusing on data and algorithm ethics that can be integrated into system design and development.Source: §V · p. 5
  • The paper reviews requirements, challenges, and research, then discusses how existing techniques could support more trustworthy AI use in a smart-city case.Source: §V · p. 5

Related papers

Loading 2002.05657v1…