Source-linked AI summary
D4D-Senegal: The Second Mobile Phone Data for Development Challenge
Yves-Alexandre de Montjoye, Zbigniew Smoreda, Romain Trinquart, Cezary Ziemlicki, Vincent D. Blondel
TL;DR
Large-scale behavioral data must be made broadly available and interpreted in context to support practical development questions. This paper presents the D4D-Senegal challenge, describing three sampled and aggregated datasets built from anonymized mobile-phone metadata and documenting research uses of such data.
Problem
Big Data’s promises depend on broader availability and contextual interpretation, requiring development economists, urban planners, sociologists, and NGOs to engage with these data.
Method
The paper describes D4D-Senegal, its data preprocessing, and three sampled and aggregated datasets made available through the challenge.
Results
More than 80 research papers were produced within three months of the preceding challenge, covering topics including bus-route optimization, social divisions, and disease-containment policies.
Takeaways & Limitations
The challenge makes mobile-phone metadata available for research on practical questions while emphasizing that such data require conceptual frameworks for interpretation.
Takeaways & Limitations
Geographical coordinates are not released for commercial and privacy reasons, and the data include only calls or texts between Sonatel customers.
Abstract
from arXiv · showhide
The D4D-Senegal challenge is an open innovation data challenge on anonymous call patterns of Orange's mobile phone users in Senegal. The goal of the challenge is to help address society development questions in novel ways by contributing to the socio-economic development and well-being of the Senegalese population. Participants to the challenge are given access to three mobile phone datasets. This paper describes the three datasets. The datasets are based on Call Detail Records (CDR) of phone calls and text exchanges between more than 9 million of Orange's customers in Senegal between January 1, 2013 to December 31, 2013. The datasets are: (1) antenna-to-antenna traffic for 1666 antennas on an hourly basis, (2) fine-grained mobility data on a rolling 2-week basis for a year with bandicoot behavioral indicators at individual level for about 300,000 randomly sampled users, (3) one year of coarse-grained mobility data at arrondissement level with bandicoot behavioral indicators at individual level for about 150,000 randomly sampled users
Introduction
The paper motivates broader access to mobile-phone metadata while emphasizing that interpretation requires context and conceptual framing. It introduces D4D-Senegal, a challenge providing one year of metadata for up to 300,000 people across Senegal.
- Large-scale behavioral datasets have supported research on malaria, poverty, human mobility, social communities, transport, social divisions, and disease containment.
- Broader access to Big Data is needed, but sound interpretation depends on understanding context and applying an explicit conceptual framework.
- The earlier D4D-Cote d’Ivoire challenge received 260 applications worldwide and produced more than 80 research papers after three months.
- D4D-Senegal gives selected teams access to one year of metadata for up to 300,000 people across Senegal.
- The paper describes the challenge’s data preprocessing, three datasets, and research questions suggested by local partner organizations.
Data preprocessing
The preprocessing pipeline anonymizes and filters one year of Call Detail Records before release. It removes or transforms information that could increase re-identification risk, including geographic coordinates.
- Call Detail Records were collected from January 1 through December 31, 2013, with customer identifiers anonymized before preprocessing.
- The original dataset contained more than 9 million unique aliased mobile phone numbers.
- Retained users had interactions on more than 75% of days in the relevant period and averaged fewer than 1000 interactions per week.
- Users averaging more than 1000 weekly interactions were presumed to be machines or shared phones and excluded.
- Real BTS coordinates were withheld, and each site was assigned a new position uniformly within its Voronoi cell to make re-identification harder.
Datasets
The challenge provides three sampled and aggregated mobile-phone datasets that trade spatial, temporal, and user-level detail against re-identification risk. They cover site traffic, fine-grained site mobility, and coarse-grained arrondissement mobility, with behavioral indicators for the mobility datasets.
- Dataset design: Three sampled and aggregated datasets balance broad research use with re-identification risks.The design varies spatial and temporal precision or aggregates across users to manage this trade-off.
- Dataset 1: Site traffic: Dataset 1 records one year of hourly site-to-site traffic across 1666 sites.It includes monthly text and voice traffic files, with calls represented by counts and total duration.
- Dataset 2: Fine-grained mobility: Dataset 2 provides site-level mobility over rolling two-week periods for about 300,000 randomly sampled users.The data include timestamps and site identifiers, with a new sample selected for each period and behavioral indicators computed per user.
- Dataset 3: Coarse-grained mobility: Dataset 3 provides one year of arrondissement-level mobility for 146,352 randomly selected users.User trajectories and behavioral indicators are organized month by month, with arrondissement identifiers and geographic reference files.
- Behavioral indicators: Mobility datasets include Bandicoot indicators covering activity, call duration, contact entropy, places, interactions, and inter-event patterns.The indicators are computed from metadata, while selected columns are binned and 3-anonymized to remove outliers and reduce re-identification risk.
- Data assumptions: Traffic data count only calls or texts between Sonatel customers, and calls spanning time slots are assigned to their starting slot.Site traffic files use hourly timestamps and distinguish outgoing from incoming sites.
Research collaboration
The challenge encourages collaboration between participants and local teams because sound research questions and valid interpretations require Senegalese political, cultural, and socio-economic context. A Sparkboard space is provided to facilitate these collaborations.
- Collaboration: The D4D team encourages scientific collaboration between challenge participants and local teams.The paper presents collaboration as relevant both to data production and to interpretation.
- Context: Understanding Senegal's political, cultural, and socio-economic context is described as essential for sound questions and valid interpretations.The paper emphasizes that data result from contingent and contested social practices.
- Collaboration: A collaborative Sparkboard space lets participants announce projects and specify the competencies or collaboration they seek.The platform is provided to help organize connections between challenge participants and local teams.