Source-linked AI summary
Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
Kashif Ahmad, Majdi Maabreh, Mohamed Ghaly, Khalil Khan, Junaid Qadir, Ala Al-Fuqaha
TL;DR
AI deployment in human-centric smart cities raises security, robustness, interpretability, and ethical challenges with implications for safety, privacy, bias, and governance. The paper surveys these challenges and their interconnections, including how one may help address or worsen another. It identifies limitations, pitfalls, and open research directions as a baseline for future work.
Problem
Smart-city AI deployment faces security, robustness, interpretability, privacy, bias, and ethical challenges that matter in sensitive human-centric applications.
Method
The paper provides a detailed survey and analysis of literature on these challenges, emphasizing their connections and dependencies.
Results
The paper identifies relationships among the challenges, including explainability’s potential to mitigate bias and interpretability problems while also aiding adversarial attacks.
Takeaways & Limitations
Understanding these interconnections can inform safer, more interpretable, and more ethical AI deployment in smart-city applications.
Abstract
from arXiv · showhide
As the globally increasing population drives rapid urbanisation in various parts of the world, there is a great need to deliberate on the future of the cities worth living. In particular, as modern smart cities embrace more and more data-driven artificial intelligence services, it is worth remembering that technology can facilitate prosperity, wellbeing, urban livability, or social justice, but only when it has the right analog complements (such as well-thought out policies, mature institutions, responsible governance); and the ultimate objective of these smart cities is to facilitate and enhance human welfare and social flourishing. Researchers have shown that various technological business models and features can in fact contribute to social problems such as extremism, polarization, misinformation, and Internet addiction. In the light of these observations, addressing the philosophical and ethical questions involved in ensuring the security, safety, and interpretability of such AI algorithms that will form the technological bedrock of future cities assumes paramount importance. Globally there are calls for technology to be made more humane and human-centered. In this paper, we analyze and explore key challenges including security, robustness, interpretability, and ethical (data and algorithmic) challenges to a successful deployment of AI in human-centric applications, with a particular emphasis on the convergence of these concepts/challenges. We provide a detailed review of existing literature on these key challenges and analyze how one of these challenges may lead to others or help in solving other challenges. The paper also advises on the current limitations, pitfalls, and future directions of research in these domains, and how it can fill the current gaps and lead to better solutions. We believe such rigorous analysis will provide a baseline for future research in the domain.
1. Introduction
The paper situates AI-enabled smart cities within rapid urbanisation and examines the security, robustness, interpretability, and ethical challenges affecting human-centric deployment. It surveys these challenges together, emphasizing their interdependence and identifying limitations, pitfalls, and research directions.
- AI in smart cities: AI uses data from IoT sensors to support pattern recognition, resource management, and service optimization in smart cities.The paper broadly includes data-driven learning techniques and intelligent systems under AI.
- Challenges: Smart-city AI faces risks involving data availability, bias, privacy, adversarial attacks, security threats, and limited interpretability.These risks can affect sensitive domains such as healthcare, law enforcement, transportation, and autonomous vehicles.
- Challenge interdependence: Explainability can help address biased decisions and interpretability problems, but its explanations may also help attackers generate more effective attacks.The paper presents this as a knock-on relationship among the challenges.
- Applications: The survey reviews AI applications including healthcare, transportation, autonomous cars, tourism, culture, services, and entertainment.Figure 1 illustrates selected smart-city AI applications discussed in the paper.
- Future directions: It identifies current limitations, pitfalls, and open research challenges to inform future research and better solutions.The paper presents this analysis as a way to address gaps in the literature.
- Survey scope: The paper analyzes existing literature on security, safety, robustness, interpretability, and data and algorithmic ethics in human-centric applications.Its distinctive emphasis is how solutions to one challenge may help or worsen others.
2. Smart City AI Security and Robustness
Machine learning has potential to improve the productivity and effectiveness of different city systems, despite positive outcomes and promise.
- Smart-city AI: Machine learning has tremendous potential in smart cities.The supplied passage introduces this potential without specifying particular applications or evaluations.
- Smart-city AI: Machine learning can improve the productivity of different city systems.
- Smart-city AI: Machine learning can improve the effectiveness of different city systems.
II. Smart City AI Security and Robustness
The supplied passage identifies smart-city data management and ethics as a section topic.
- Smart City Data Management/Ethics: Smart-city data management and ethics are identified as a major topic.
- Smart City Data Management/Ethics: The section is framed around ethical considerations associated with smart-city data.
- Smart City Data Management/Ethics: The supplied heading does not specify particular data-management methods or ethical findings.
III. Smart City AI Interpretability
The supplied passages identify challenges in collection and the scope of the survey as section topics.
- Challenges in Collection: Challenges in collection are identified as a topic in the paper’s organization.
- Scope and organization: The supplied passages provide section headings but no substantive findings about collection or survey scope.
- Scope of the Survey: The scope of the survey is identified as a separate topic in the paper’s organization.
A. Explainable AI for Smart City Applications
The supplied passages identify explainable AI and datasets as subsection topics within the survey.
- The section includes a subsection on Explainable AI and adversarial issues.
- The passages provide topic headings but no substantive findings or analysis.
- The section also includes a subsection connecting explainability and datasets.
VII. Open Issues and Future Research Directions
The supplied passages identify academic publications and policies and guidelines as topics associated with future research directions.
- The material lists academic publications as one topic.
- The supplied passages do not state specific open issues or future research recommendations.
- The material lists policies and guidelines as another topic.
C. Analytical Review of the Key Issues
The review presents smart-city AI security as a broad challenge involving adversarial manipulation, safety incidents, and the need for robust solutions. It surveys attack mechanisms and illustrates consequences across autonomous vehicles, chatbots, healthcare, and other applications.
- Analytical Review of the Key Issues: AI security requires investigation because small input or data modifications can change model decisions and cause serious consequences.The paper frames these as challenges distinct from conventional software-security problems.
- Analytical Review of the Key Issues: Safety failures have affected autonomous vehicles and public-facing AI, including crashes, lane errors, offensive chatbot behavior, and data leakage.Examples span Tesla, Google AV, Uber, Microsoft’s chatbot, and a South Korean chatbot.
- Analytical Review of the Key Issues: The paper argues that good predictive performance is insufficient and calls for safe, robust AI solutions at both technical and policy levels.
- Analytical Review of the Key Issues: Adversarial attacks can exploit crafted inputs that appear normal to humans, causing models to misclassify images, sounds, or other data.The review describes untargeted and targeted attacks and white-box, gray-box, and black-box threat models.
- Analytical Review of the Key Issues: Healthcare models can be manipulated or misled, potentially reversing disease-related decisions while manipulated low-resolution images may evade radiologists.The review also notes a DNN-based detector with high accuracy for manipulated medical data.
2.2. Security Attacks on AI
The section surveys common attacks against AI in cloud and edge smart-city deployments, including poisoning, evasion, trojans, model extraction, and membership inference. It emphasizes that safety depends on uncertainty, representative data, and application context.
- Security Attacks on AI: The surveyed attack strategies include data poisoning, evasion, exploratory attacks, model extraction, backdoors, trojans, membership inference, and model inversion.
- Security Attacks on AI: Data poisoning injects manipulated training data so retraining degrades model performance, creating risks for crowdsensing in transportation, pollution, and energy services.
- Security Attacks on AI: Evasion attacks occur after training by perturbing inputs, and common classifiers can be evaded with limited system knowledge.
- Security Attacks on AI: Trojan attacks preserve normal behavior until a trigger activates a target prediction, while model extraction reconstructs a target model through queries and returned labels.
- Security Attacks on AI: Membership-inference severity depends on application and training-data type, with potentially serious implications for education, finance, and healthcare tabular data.
- Security Attacks on AI: AI safety concerns minimizing damage risk and uncertainty, but evaluation may require effort beyond testing datasets because real environments are more uncertain.The paper links representative datasets with safer and more robust solutions.
- Security Attacks on AI: Unsafe AI can directly affect lives or indirectly produce racist outcomes, as illustrated by autonomous-vehicle deaths and racist image-app results.
3. Smart City AI Interpretability
Smart city AI interpretability concerns whether users can understand model decisions, especially in critical applications where predictions alone are insufficient. Explainable AI offers transparency and potential bias detection, but its explanations may also affect adversarial security.
- Black-box AI models produce predictions without justifying the reasons behind their outcomes.
- Interpretability is especially important in critical applications because users need to understand the causes behind AI decisions, not merely receive predictions.
- Explainability can build trust, support bias detection, and is particularly needed in sensitive domains such as healthcare, banking, transportation, and education.
- Explainable AI includes transparent models that restrict complexity and post-hoc methods that analyze model behavior after training.
- Explainable AI may help identify adversarial inputs through anomalous explanations, while generated explanations may also help attackers create more adverse attacks.
4. Smart City and Data-related Challenges
Smart city data management faces challenges in collecting, storing, sharing, and maintaining heterogeneous data under resource and quality constraints. Ethical concerns include consent, ownership, privacy, bias, transparency, interpretation, and re-identification.
- Smart city data infrastructures must manage storage, processing, energy consumption, carbon emissions, heterogeneous sources, and continuously collected data.
- Sensor accuracy and external conditions such as temperature and weather can affect data quality and downstream AI performance.
- Constrained sensors and networks have limited storage, bandwidth, and processing power, requiring reliable collection and transmission infrastructure.
- Data sharing raises concerns about transparency, interpretation, trust, re-identification, privacy, and discrimination based on linked personal or group information.
- Informed consent requires participants to understand data collection, its goals, and how and why their data will be used.
- Hidden collector or respondent biases can directly affect analysis and are especially risky in human-centric applications.
5. AI Ethics and Smart Cities
AI ethics in smart cities spans human-centered and machine-centered perspectives and draws on interdisciplinary scholarship, policies, and guidelines. The reviewed discourse addresses practical concerns including privacy, transparency, bias, accountability, and contested long-term risks.
- AI ethics has expanded from limited attention in earlier AI scholarship into an interdisciplinary discourse covering data, information, robot, internet, machine, and military ethics.
- Academic publications, handbooks, policies, guidelines, and corporate principles provide the main forms through which AI ethics has developed.
- Human-centered AI ethics examines the morality of people who develop, manufacture, operate, or consume AI systems, whereas machine ethics examines how systems can behave ethically.
- Smart-city moral discourse engages issues such as privacy and information transparency, while stakeholders seek applicable policies and practical governance guidance.
- The singularity hypothesis is contested, with some treating it as an existential moral concern and others viewing it as dubious or irrelevant to current policy.
- Explainability is linked to fairness, bias, accountability, and trust, while data-driven algorithms can reproduce sexism, racism, and other negative stereotypes.
- Predictions about AI’s labor-market and economic effects are difficult for policymakers to use because economics research is limited and forecasts rely on past technologies.
6. Insights and Lessons Learned
The reviewed literature identifies interconnected lessons about AI safety, explainability, adversarial robustness, and ethics in smart-city applications. It emphasizes stakeholder responsibility, trade-offs between competing objectives, and the need to integrate these concerns throughout AI development and deployment.
- Security and Robustness: Safety strategies include studying potential bias before development, constraining protected-group risk, enabling human intervention when confidence is low, and supporting testing through open algorithms and datasets.
- Security and Robustness: Adversarial attacks can seriously affect people’s lives, privacy, opportunities, assets, economies, and environments in smart-city applications.
- Security and Robustness: Anti-adversarial defenses are not sufficient alone, because risk prevention and mitigation remain shared responsibilities among authorities and organizations.
- Security and Robustness: Adversarial examples can transfer across models, enabling attacks on black-box systems, while no effective defense mechanism currently exists against this threat.
- Interpretability and Explainability: Explainability supports stakeholder trust and fair decisions by helping identify decisions based on protected attributes, but transparent models may perform worse than black-box deep-learning models.
- Interconnections and Ethics: Explainability is connected with ethics and adversarial security: it may distinguish genuine samples from adversaries, while explanations can also help attackers generate more effective attacks.
- AI Ethics: AI ethics is developing as an interdisciplinary field involving philosophers, engineers, researchers, practitioners, industry, and governments, with policies and guidelines addressing ethical governance.
- AI Ethics: Ethical literature identifies serious smart-city challenges involving big-data management, including privacy, explainability, and transparency, although disagreement remains about some moral issues.
7. Open Issues and Future Research Directions
Future research should address security, robustness, interpretability, and ethical challenges while examining their interactions and practical deployment constraints. Key priorities include evaluation under attack, explainability trade-offs, conceptual clarity, and stronger links between ethics, policy, and AI practice.
- Smart City AI Security and Robustness: Smart-city AI security research should evaluate response-time and safety trade-offs, especially where decisions must be made quickly.Adversarial-attack detection and safety procedures may affect timely decisions in applications such as autonomous vehicles.
- Smart City AI Security and Robustness: Researchers should model ripple effects because attacking one smart-city model can produce consequences across interconnected services and other models.Loss estimation, functional-dependency analysis, and simulation are proposed as parts of future AI-system development lifecycles.
- Smart City AI Security and Robustness: Future evaluations should include robustness and safety metrics because high test accuracy may fail under production-time noise or attacks.The paper calls for revisiting evaluation policies before deployment and incorporating robustness against different attack types.
- Interpretation vs. Performance: Explainability research must address its trade-off with performance, since simpler models are usually more explainable while complex models often achieve greater accuracy.The paper identifies improving explainability methods as a way to optimize this trade-off.
- Concepts and Evaluation Metrics: The literature lacks a unified concept and evaluation framework for explainability, requiring clearer definitions and metrics for comparing interpretation capabilities.Deep-learning interpretability and visualization-based explanations remain active research challenges.
- AI Ethics: AI ethics needs greater conceptual clarity, practical bridges between research and policymaking, and broader cultural representation in moral discourse.The paper notes divergent meanings across disciplines, limited impact of guidelines on industry practice, and the predominantly Western character of existing discourse.
8. Conclusions
The paper reviews four challenges in deploying AI for smart-city applications: security and robustness, interpretability, and ethical data and algorithmic concerns. It emphasizes their connections, identifies limitations and open research issues, and presents the analysis as a baseline for future research.
- Conclusions: The paper reviews security, robustness, interpretability, and ethical data and algorithmic challenges in deploying AI for human-centric smart-city applications.It examines these challenges together rather than as isolated concerns.
- Conclusions: The analysis focuses on how the challenges are linked and how addressing one may help solve or create problems involving others.This connection-oriented perspective is a central emphasis of the paper.
- Conclusions: The paper identifies limitations, pitfalls, and open research issues in existing approaches to these deployment challenges.These findings are intended to support further work in the domain.