Source-linked AI summary
Ontology Bulding vs Data Harvesting and Cleaning for Smart-city Services
Pierfrancesco Bellini, Paolo Nesi, Nadia Rauch
TL;DR
Public and private city datasets are often not semantically interoperable, complicating the creation of integrated smart-city knowledge bases. The paper describes a system for ingesting and reconciling these data into an ontology-backed knowledge base; it reports a base exceeding 81 million triples at launch and provides a tool for exploring its data.
Problem
Public and private city datasets often lack semantic interoperability, while their varied formats and meanings make integration a substantial challenge.
Source: §I · p. 1, A. Data Ingestion · p. 7Method
The system ingests city data, maps it to the smart-city ontology, and applies reconciliation, verification, and validation to produce a knowledge base.
Source: §I · p. 1, §VI · p. 8Results
More than 81 million triples were present in the knowledge base on its first day, and it grew by 4 million per month.
Source: A. Data Ingestion · p. 7Takeaways & Limitations
The RDF store includes a visualization and exploration tool for inspecting the available data.
Source: §VI · p. 8The Sensors macroclass is not yet complete and currently consists of four parts.
Source: §III · p. 4
Abstract
from arXiv · showhide
Presently, a very large number of public and private data sets are available around the local governments. In most cases, they are not semantically interoperable and a huge human effort is needed to create integrated ontologies and knowledge base for smart city. Smart City ontology is not yet standardized, and a lot of research work is needed to identify models that can easily support the data reconciliation, the management of the complexity and reasoning. In this paper, a system for data ingestion and reconciliation smart cities related aspects as road graph, services available on the roads, traffic sensors etc., is proposed. The system allows managing a big volume of data coming from a variety of sources considering both static and dynamic data. These data are mapped to smart-city ontology and stored into an RDF-Store where they are available for applications via SPARQL queries to provide new services to the users. The paper presents the process adopted to produce the ontology and the knowledge base and the mechanisms adopted for the verification, reconciliation and validation. Some examples about the possible usage of the coherent knowledge base produced are also offered and are accessible from the RDF-Store.
I. INTRODUCTION
Public and private city datasets are extensive but often lack semantic interoperability, limiting their integration into a coherent smart-city knowledge base. The paper outlines the construction, reconciliation, verification, and validation of such a base.
I. INTRODUCTION
- Public and private datasets are often semantically incompatible, and integrating them requires substantial human effort.Sources include public statistics and points of interest as well as private mobility, transport, and sensor data.Source: §I · p. 1
- Differences in formats, field semantics, and update frequency complicate integration of static and dynamic city data.Updates range from monthly or yearly for static data to daily or more frequent for dynamic data.Source: §I · p. 1
- The paper describes ontology construction, data ingestion, reconciliation, and knowledge-base verification and validation.The resulting knowledge base is intended to support queries for suggestions, predictions, and deductions.Source: §I · p. 1
II. ONTOLOGY MAIN ELEMENTS
The ontology was developed by analyzing regional public and private datasets and organizing their elements and relationships into six macroclass areas. It reuses established vocabularies alongside city-specific classes.
II. ONTOLOGY MAIN ELEMENTS
- Analysis of more than 800 Florence and Tuscany datasets produced an integrated ontology organized into six macroclass areas.The analyzed material includes street and civic-number data, points of interest, transport, and other regional and municipal data.Source: §II · p. 2
- The six areas cover administration, the street guide, points of interest, local public transport, sensors, and temporal concepts.They represent public administration, road systems, citizen services, transport networks, sensor observations, and time.Source: §II · p. 2
- The ontology reuses Dublin Core, FOAF, vCard, and WGS84 position vocabularies for metadata, people and organizations, and geographic coordinates.The source also notes reuse of the OTN vocabulary for traffic modeling in the street-guide macroclass.Source: §II · p. 2
III. SMART-CITY ONTOLOGY DETAILS
The ontology details encode relationships and constraints across administration, streets, services, transport, sensors, and time. Several models remain incomplete or approximate, as specified for the sensor and temporal macroclasses.
III. SMART-CITY ONTOLOGY DETAILS
- The Administration macroclass models the Italian hierarchy of regions, provinces, and municipalities, linking public administrations to resolutions and shared statistical data.Region, Province, and Municipality are classified through restrictions on object properties; statistical data can link both administrations and roads.Source: §III · p. 2
- The Street-guide macroclass represents roads through road elements, nodes, sections, access rules, maneuvers, and street addresses.Nodes require one latitude/longitude pair; Road and AdministrativeRoad have a many-to-many relationship, and road links and junctions add 20 million triples.Source: §III · p. 3
- The Point of Interest macroclass classifies citizen services and connects them to the street network through an access or road relationship.A service can connect to one external access through hasAccess, or to a road through isIn when access details are unavailable.Source: §III · pp. 3–4
- The transport model represents bus routes as road segments delimited by stops and models rail lines through railway elements and junctions.Bus lines, rides, routes, stops, and mapped route links are represented; railway elements connect to railway junctions and may form lines or directions.Source: §III · p. 4
- The Sensors macroclass covers parking, weather, traffic, and bus AVM data, but the authors state that it is not yet complete.Traffic sensors may be located only within a toponym rather than at a precise coordinate; observations include speed, flow, concentration, and headway.Source: §III · pp. 4–5
- The Temporal macroclass links time instants to records and uses OWL-Time concepts for instants, intervals, and durations, but remains sketchy.Resource-specific time links are used to avoid ambiguity when different resources share an identical time instant.Source: §III · p. 5
IV. DATA ENGINEERING ARCHITECTURE
The architecture ingests data, maps it to the ontology, validates the interoperable knowledge, and exposes it for application use. A scheduler supports distributed processing and recurring updates.
IV. DATA ENGINEERING ARCHITECTURE
- The data-engineering process comprises ingestion, knowledge mapping, validation, and access or exploitation by services.Ingested data are transcoded and mapped to the DISIT Smart City Ontology.Source: §IV · p. 5
- A process scheduler allocates ingestion across a parallel, distributed server architecture and regularly checks for updates.The scheduler retrieves data on a regular basis to support updates.Source: §IV · p. 5
- Mapped data are stored in OWLIM-SE and made available through a SPARQL endpoint for applications and city services.Examples include Linked Open Data browsing and the Service Map.Source: §IV · p. 5
A. Data Ingestion
The ingestion workflow harvests heterogeneous static and real-time data, maps them into RDF using the smart-city ontology, and reconciles services with the street guide. The resulting knowledge base contains more than 81 million triples and reports both street-number- and street-level matches.
A. Data Ingestion
- In Florence, ingestion addresses about 150 of 564 sources, harvesting static and semi-static files alongside real-time sensor feeds through dataset-specific ETL processes.The acquired data are logged and versioned in HBase; real-time data are typically streamed through SQL before conversion to RDF triples.Source: A. Data Ingestion · p. 6
- Mapping moves data from HBase through temporary MySQL into RDF triples, with Karma generating R2RML mappings against the DISIT ontology and reused vocabularies.The mapping specifies dataset semantic types, links SQL columns to ontology properties, and defines relationships between ontology classes.Source: A. Data Ingestion · p. 6
- Reconciliation progresses from exact address matching through notation variants and geocoding, while typos, 0/SNC street numbers, and unusual street characters remain partly unresolved.Matching first targets street-number level and then street level; support tables address alternate street-name forms, and later steps include geocoding and character cleanup.Source: A. Data Ingestion · p. 7
- More than 81 million triples were created on day one, growing by 4 million monthly; 13,185 of 30,182 services/POIs reconciled at street-number level and 21,207 at street level.Another 149 services with coordinates had no reconciliation result because some streets or civic numbers were missing or incomplete.Source: A. Data Ingestion · p. 7
V. VERIFICATION AND VALIDATION
The authors validate ingestion and mapping with recurring SPARQL checks, then use visualization and geographic queries to explore the resulting knowledge base. Weather-forecast ingestion illustrates the volume processed, while ServiceMap exposes geographic and transit information.
V. VERIFICATION AND VALIDATION
- Recurring SPARQL queries check for inconsistencies and incompleteness after ingestion updates, with validation findings feeding an iterative process of data and ontology changes.For weather data, municipality ISTAT codes support reconciliation; some road matches still required manual selection and owl:sameAs links.Source: §V · p. 7
- 286 weather files updated twice daily add approximately 270,000 HBase lines and about 2.5 million triples per month.Each file contains 16 lines of weekly weather prediction.Source: §V · p. 8
- The Linked Open Graph supports exploration of entity relationships, while ServiceMap handles geographic queries and exposes bus waiting times and real-time parking availability.Examples include finding points of interest near a bus station and street numbers near a map location.Source: §V · p. 8, Figure 5 · p. 8
VI. CONCLUSIONS
The paper presents a system that ingests public and private smart-city data, maps it to an ontology, and stores it in an RDF store for SPARQL access. It reports an incrementally developed ontology and verification and validation intended to improve the model and support its deductive capabilities.
VI. CONCLUSIONS
- The system integrates public and private data, including static and dynamic sources, maps them to a smart-city ontology, and stores them in an RDF store accessible through SPARQL.The ontology was built incrementally by analyzing, integrating, and validating each added dataset.Source: §VI · p. 8
- Verification and validation identify triples for improvement and enrichment, with corrections intended to enable deductive use of the final model.The paper also provides a tool to visualize and explore RDF-store data.Source: §VI · p. 8
- Planned work includes linking names and points of interest to DBpedia or GeoNames, integrating GoodRelations, and automating reconciliation with link discovery and machine learning.Source: §VI · p. 8