Source-linked AI summary
Data for Development: the D4D Challenge on Mobile Phone Data
Vincent D. Blondel, Markus Esch, Connie Chan, Fabrice Clerot, Pierre Deville, Etienne Huens, Frédéric Morlot, Zbigniew Smoreda, Cezary Ziemlicki
TL;DR
The paper addresses limited access to scientifically valuable mobile-phone data while protecting privacy and supporting development research. It describes how anonymized Ivory Coast call-detail records were processed into datasets for traffic, trajectory, and communication-graph analysis, with privacy and technical constraints.
Problem
Sensitive mobile-phone traces require restricted access, but limited access can create a new digital divide in scientific research.
Method
The paper describes anonymized Call Detail Records collected over 150 days, preprocessing them for homogeneous sampling and double-count removal, and organizing them into analytical datasets.
Results
The resulting datasets support hourly communication-traffic visualization, individual trajectory analysis, and communication-graph analysis.
Takeaways & Limitations
The challenge makes large mobile-phone datasets more available for scientific research and encourages combining them with other information sources.
Takeaways & Limitations
About one in four calls have missing antenna identifiers, represented by the code −1 rather than removed.
Abstract
from arXiv · showhide
The Orange "Data for Development" (D4D) challenge is an open data challenge on anonymous call patterns of Orange's mobile phone users in Ivory Coast. The goal of the challenge is to help address society development questions in novel ways by contributing to the socio-economic development and well-being of the Ivory Coast population. Participants to the challenge are given access to four mobile phone datasets and the purpose of this paper is to describe the four datasets. The website http://www.d4d.orange.com contains more information about the participation rules. The datasets are based on anonymized Call Detail Records (CDR) of phone calls and SMS exchanges between five million of Orange's customers in Ivory Coast between December 1, 2011 and April 28, 2012. The datasets are: (a) antenna-to-antenna traffic on an hourly basis, (b) individual trajectories for 50,000 customers for two week time windows with antenna location information, (3) individual trajectories for 500,000 customers over the entire observation period with sub-prefecture location information, and (4) a sample of communication graphs for 5,000 customers
1. Introduction.
The D4D challenge makes anonymized Ivory Coast mobile-phone data available for scientific research and development-oriented analysis. Participants are encouraged to combine these datasets with additional information sources.
- Orange provides anonymized Ivory Coast mobile-phone datasets to improve data availability for scientific research and foster development collaborations.The initiative responds to privacy concerns and the potential for restricted data access to create a new digital divide.
- Participants are encouraged to combine the D4D mobile-phone datasets with other datasets and sources of information.
2. Other datasets on Ivory Coast.
The paper identifies numerous Ivory Coast and Africa-focused sources that can complement the D4D data, spanning development, demographic, economic, geographic, health, ICT, and food-security information. It also documents the underlying 150-day anonymized CDR collection.
- The listed complementary sources cover development, economic, demographic, geographic, health, agricultural, food-security, and ICT information.Examples include the African Development Bank, United Nations, World Bank, FAO, FEWS NET, Measure DHS, and ICT-focused sources.
- Mobile and Development Intelligence aggregates publicly available data and analysis to address information gaps about mobile industries and their socio-economic impact in developing countries.
- 150 days of data, from December 1, 2011 to April 28, 2012, contain 2.5 billion calls and SMS exchanges among around five million users.The original CDR format includes timestamp, caller ID, callee ID, call duration, and antenna code.
3. Data Preprocessing.
Preprocessing standardizes the customer and communication records while preserving antenna information with privacy and completeness constraints. Antenna locations are blurred, missing identifiers are coded as −1, and about 100 of 3600 hours are missing.
- Customers who joined or left Orange during the observation period were removed, and incoming and outgoing calls were paired to eliminate double counts.
- Exact antenna locations were slightly blurred because Orange considered them sensitive information.
- About one in four calls lack antenna identifiers, which are represented by the code −1 rather than being removed.
- Approximately 100 hours are missing from the datasets’ 3600-hour coverage because of technical reasons.
- All datasets are distributed as tabulation-separated values plain-text files.
4. Published Datasets.
The paper publishes four anonymized mobile-phone datasets spanning aggregate traffic, individual mobility at two spatial-temporal resolutions, and ego-centered communication graphs. Each dataset uses a distinct aggregation or privacy design to support analysis while limiting identification risk.
- Antenna-to-antenna traffic: Hourly antenna-to-antenna data reports call counts and total call duration between antenna pairs across the full observation period.Calls spanning multiple time slots are assigned to the hour in which they started, and communications with non-Orange customers are removed.
- High spatial resolution trajectories: SET2 provides high-resolution trajectories for 50,000 randomly sampled individuals over consecutive two-week periods.Antenna locations are retained, timestamps are rounded to the minute, and new anonymized identifiers are assigned in each period.
- Communication subgraphs: SET4 contains 5,000 ego-centered communication graphs per two-week period, covering each ego’s first- and second-order neighborhood.Edges indicate whether a pair communicated, while communications between second-order neighbors and communication counts or durations are excluded.
- Communication subgraphs: SET4 preserves identifiers across time within each ego graph but assigns different identifiers when the same individual appears in another ego-centered graph.Customers identified as public phone providers are excluded from the selected egos.