PaperlayerView exact revision on arXiv

Source-linked AI summary

Scikit-mobility: a Python library for the analysis, generation and risk assessment of mobility data

Luca Pappalardo, Filippo Simini, Gianni Barlacchi, Roberto Pellungrini

arXiv:1907.07062v6physics.soc-ph

TL;DR

Mobility research spans data preparation, pattern analysis, generation, and privacy assessment, but lacks software integrating these aspects. scikit-mobility provides a Python library for these tasks, while its current scope excludes next-location prediction and faces scalability limits for terabyte-scale data.

  • Problem

    No statistical software supports all the major aspects of mobility analysis, including preprocessing, pattern analysis, generation, and privacy assessment, in one environment.

  • Method

    scikit-mobility combines pandas-based mobility-data management with analysis, visualization, preprocessing, generative models, and privacy-risk assessment modules.

  • Results

    The paper presents scikit-mobility as a single environment for managing trajectories and flows and supporting mobility analysis, synthetic-data generation, and privacy-risk assessment.

  • Takeaways & Limitations

    The library brings multiple mobility-analysis functions into one Python environment for reproducing research and working with mobility data.

  • Takeaways & Limitations

    The current library is not scalable to terabyte-scale mobility data because pandas DataFrames must be fully loaded in memory.

Abstract

from arXiv · show

The last decade has witnessed the emergence of massive mobility data sets, such as tracks generated by GPS devices, call detail records, and geo-tagged posts from social media platforms. These data sets have fostered a vast scientific production on various applications of mobility analysis, ranging from computational epidemiology to urban planning and transportation engineering. A strand of literature addresses data cleaning issues related to raw spatiotemporal trajectories, while the second line of research focuses on discovering the statistical "laws" that govern human movements. A significant effort has also been put on designing algorithms to generate synthetic trajectories able to reproduce, realistically, the laws of human mobility. Last but not least, a line of research addresses the crucial problem of privacy, proposing techniques to perform the re-identification of individuals in a database. A view on state of the art cannot avoid noticing that there is no statistical software that can support scientists and practitioners with all the aspects mentioned above of mobility data analysis. In this paper, we propose scikit-mobility, a Python library that has the ambition of providing an environment to reproduce existing research, analyze mobility data, and simulate human mobility habits. scikit-mobility is efficient and easy to use as it extends pandas, a popular Python library for data analysis. Moreover, scikit-mobility provides the user with many functionalities, from visualizing trajectories to generating synthetic data, from analyzing statistical patterns to assessing the privacy risk related to the analysis of mobility data sets.

1 Introduction

scikit-mobility addresses the lack of a unified statistical software environment for mobility data by combining analysis, generation, and privacy-risk assessment capabilities in a Python library.

  • Research context: Massive mobility datasets have supported research on preprocessing, statistical laws of human movement, synthetic trajectory generation, and privacy risks.
  • Research gap: No statistical software previously supported all these aspects of mobility analysis in one environment.
  • Contribution: scikit-mobility provides a Python environment for reproducing existing research and analyzing mobility data.
  • Capabilities: The library loads, represents, visualizes, cleans, and preprocesses individual and collective mobility data.
  • Capabilities: It computes mobility measures, runs mechanistic models for individual and collective movement, and estimates re-identification risk.
  • Scope and limitations: Next-location prediction is not covered, most features can extend beyond human mobility, and the implemented methods are not exhaustive.

2 Data Structures

scikit-mobility represents trajectories and flows through pandas-based data structures with defined fields, metadata, loading functions, and tessellation support.

  • Core structures: TrajDataFrame and FlowDataFrame extend pandas DataFrames, preserving DataFrame functionality, efficient tabular I/O, and compatibility with Python tools.
  • Trajectories: A trajectory is a temporally ordered sequence of timestamped locations, represented in TrajDataFrame rows with latitude, longitude, and datetime fields.
  • Trajectories: TrajDataFrame optionally uses uid for the associated object and tid for the trajectory identifier, while absent identifiers trigger single-object or single-trajectory assumptions.
  • Metadata: TrajDataFrame objects include coordinate-reference-system and operation-history attributes, with epsg:4326 as the default reference system.
  • Loading and validation: The library loads trajectory and flow data from files, validates or converts mandatory trajectory types, and records file provenance in parameters.
  • Flows: FlowDataFrame represents origin-destination flows with origin, destination, and integer flow columns, and is associated with a spatial tessellation.

3 Trajectory preprocessing

scikit-mobility’s preprocessing module supports noise filtering, stop detection, and trajectory compression for mobility trajectories. These operations remove implausible points, identify visited stops, and reduce trajectory size while preserving its structure.

  • Preprocessing pipeline: The preprocessing module provides noise filtering, stop detection, and trajectory compression, applying methods separately to individual trajectories when needed.These are the three main preprocessing steps implemented for mobility data.
  • Noise filtering: Points are filtered when their speed from the previous point exceeds max_speed, whose default value is 500 km/h.The filtering threshold controls how aggressively recording-error points are removed.
  • Noise filtering: 108,874 of 217,653 trajectory points are filtered out in the example, and lowering max_speed makes filtering more intense.The example applies a max_speed of 10 km/h and retains 108,779 points.
  • Stop detection: The stops function detects locations where an object remains for at least minutes_for_a_stop minutes within a specified spatial radius.Stop detection adds a leaving_datetime column indicating when the user left the stop.
  • Trajectory compression: Trajectory compression merges nearby points to reduce their number while preserving trajectory structure, typically after stop detection.The example merges points closer than 0.2 km, facilitating subsequent visualization.

4 Plotting

scikit-mobility provides interactive map-based visualization for trajectories, stops, diaries, tessellations, and flows. Its plotting methods encode temporal visits, stop clusters, geographic flows, and customizable map details for exploratory mobility analysis.

  • Interactive visualization: Interactive folium visualizations support exploratory analysis of trajectories and flows, including zooming and interaction with map components.Plots can be saved as HTML or screenshots as PNG files.
  • Trajectory visualization: TrajDataFrame plotting includes trajectory lines, stop markers, and diaries showing visited locations over time.Trajectory plots connect time-ordered points, while stop plots require leaving_datetime and diaries require clustered stops.
  • Interactive visualization: Plotting methods return folium.Map objects that can be reused to add further scikit-mobility or folium visualizations to the same map.Map size and displayed elements can be controlled through arguments such as max_users, maxitems, and min_flow.
  • Visualization caveat: When trajectories represent abstract mobility, straight-line connections may ignore walls, buildings, and other road-network structures.This limitation applies, for example, to trajectories inferred from social-media posts or mobile-phone calls.
  • Trajectory visualization: Diary plots place time on the x axis, use rectangle length for visit duration, color rectangles by stop cluster, and show movement as white rectangles.The method can compare multiple moving objects by plotting their diaries next to each other.
  • Flow visualization: FlowDataFrame plotting displays tessellation tiles and draws flow lines between the centroids of tiles connected by flows.Flow-line thickness can represent flow magnitude, and popup windows can expose flow or origin information.

5 Mobility measures

scikit-mobility provides individual and collective mobility measures for characterizing movement patterns, with functions that process trajectory data and return structured outputs. The section illustrates measures such as travel distances, radius of gyration, and visits per location.

  • Individual measures: Individual measures characterize movement using quantities such as traveled distances, radius of gyration, entropies, frequencies, and waiting times.The radius of gyration quantifies an individual’s characteristic travel distance, while entropy-based measures quantify movement predictability.
  • scikit-mobility implements individual and collective mobility measures as functions operating on TrajDataFrame inputs.The library separates individual and collective measures into corresponding modules and generally returns pandas DataFrames.
  • Individual measures: The example computes jump lengths and radius of gyration for each object by applying the corresponding functions to a TrajDataFrame.The resulting DataFrame contains an object identifier and a column named after the invoked measure; jump_lengths stores each object’s traveled distances.
  • Collective measures: Collective measures summarize mobility across objects, including the number of visits to each location.The visits_per_location function produces a DataFrame whose lat, lng, and n_visits columns identify locations and their visit counts.
  • Tables 3 and 4 list the individual and collective measures available in the library.

6 Individual Generative Algorithms

scikit-mobility implements individual generative algorithms to simulate synthetic trajectories for populations of moving agents. The example uses DensityEPR to generate trajectories for 1000 agents over a specified tessellation and time period.

  • Generative algorithms aim to create agent populations whose mobility patterns are statistically indistinguishable from those of real individuals.The library implements the Exploration and Preferential Return model and its variants for individual mobility generation.
  • The DensityEPR example generates synthetic trajectories for 1000 agents moving across locations in a Tessellation.The simulation is configured with a tessellation, start and end times, and model-specific parameters.
  • DensityEPR takes the simulation interval, Tessellation, number of agents, and other model-specific parameters as inputs to its generate method.Its output is a TrajDataFrame containing the trajectories of the simulated agents.

7 Collective Generative Algorithms

scikit-mobility implements collective generative algorithms for estimating flows between discrete locations. Its Gravity and Radiation models use tessellations and can fit model parameters from observed flows before generating new flows.

  • Collective generative algorithms estimate spatial flows between discrete locations, including commuting, migration, freight, and phone-call flows.
  • A valid collective-algorithm Tessellation contains geometry and relevance columns used to compute inter-tile distance and tile attractiveness.The algorithm returns a FlowDataFrame containing generated flows and the input Tessellation.
  • The library implements the Gravity and Radiation models for collective flow generation.The Gravity class provides fit and generate methods for calibrating parameters and producing flows on a tessellation.
  • The Gravity example fits a singly constrained power-law model to observed New York county commuting flows.The workflow loads a tessellation and FlowDataFrame, instantiates the model, fits its parameters, and generates flows on the same tessellation.
  • Table 5 lists the generative models implemented in scikit-mobility.

8 Privacy Risk Assessment

scikit-mobility assesses mobility-data privacy risk by simulating re-identification attacks against trajectories. It supports multiple attack models, reports risks on a 0–1 scale, and offers options to restrict or accelerate computation.

  • Mobility data can reveal confidential personal information or enable re-identification, motivating privacy-risk assessment in scikit-mobility.The paper also connects this need to GDPR requirements for assessing data-protection impact in riskiest analyses.
  • The library provides several attack models implemented as Python classes, including LocationAttack, which uses known locations without their temporal order.The knowledge_length parameter specifies how many locations the adversary knows, and risk uses the worst combination of that many locations.
  • The assess_risk method returns each object’s re-identification risk on a scale from 0 to 1, where 0 is minimum risk and 1 is maximum risk.
  • Risk assessment can target only selected objects when computation on massive datasets would be time-consuming.The targets argument restricts assessment to specified object identifiers.
  • When the maximum risk is found for an object, remaining location combinations are skipped unless force_instances=True.The result records each evaluated combination, its risk, and the locations comprising that combination.
  • Table 6 lists the privacy attacks available in the library.

9 Conclusion and Future Developments

scikit-mobility provides a unified Python environment for mobility-data analysis, generation, and privacy-risk assessment, while future work targets broader functionality and improved scalability.

  • scikit-mobility manages trajectories and fluxes through modules dedicated to specific aspects of mobility-data analysis.Its functions cover preprocessing and cleaning, mobility metrics, synthetic trajectories and flows, and privacy-risk assessment.
  • A single environment combines preprocessing, metric computation, synthetic-data generation, and privacy-risk assessment.
  • Future developments: Future modules are planned for next-location prediction, map matching, and trajectory-similarity computation.
  • Computational improvements: The current implementation handles gigabyte-scale mobility datasets but is not scalable to datasets in the order of terabytes.Planned re-implementation using more computationally efficient Python libraries aims to improve scalability.

10 Existing tools

Existing movement-data tools provide trajectory management, animal-movement analysis, mobile-phone features, or geospatial operations, but differ from scikit-mobility in domain coverage and mobility-specific functionality.

  • Existing libraries include more than 50 R packages for trajectory data, alongside Python libraries focused on mobility or trajectory management.
  • R libraries: spacetime supports spatiotemporal-data handling, while trajectories adds non-domain-specific track management, plotting, simulation, and model fitting.
  • R libraries: The adehabitat family targets animal movement and habitat selection, with adehabitatLT mainly handling regularly sampled trajectories rather than human-mobility measures and models.
  • R libraries: TrajDataMining provides trajectory preparation, clustering, and movement-pattern recognition but lacks generative models and advanced plotting.
  • Python libraries: bandicoot analyzes mobile-phone metadata, whereas scikit-mobility supports diverse sources including GPS, social-media, and mobile-phone data.
  • Python libraries: movingpandas enables geospatial trajectory operations but omits mobility-specific statistical laws, generative models, standard preprocessing, and privacy-risk assessment.
Loading 1907.07062v6…