PaperlayerView exact revision on arXiv

Source-linked AI summary

D2D Big Data: Content Deliveries over Wireless Device-to-Device Sharing in Large Scale Mobile Networks

Xiaofei Wang, Yuhua Zhang, Victor C. M. Leung, Nadra Guizani, Tianpeng Jiang

arXiv:1802.00893v1cs.NI

TL;DR

Prior D2D studies were largely confined to small datasets and limited user features, despite interest in using nearby sharing to offload cellular traffic. The paper proposes and implements a big-data platform for D2D analysis using a large real-world trace. Its measurements report power-law group sizes and D2D social-group diameters around 6 to 10, larger than those of online social-network services.

  • Problem

    Prior studies focused on small groups and small-scale data, leaving large-scale D2D delivery across millions of users and wide areas insufficiently studied.

    Source: §I · p. 2

  • Method

    The paper proposes a cloud-based D2D Big Data framework and analyzes user, content, and sharing features using a real-world D2D trace.

    Source: §I · pp. 3–4, §II · pp. 4–5

  • Results

    D2D social-group diameters are around 6 to 10, larger than online social-network services, while group sizes follow a power law.

    Source: §IV · pp. 8–9

  • Takeaways & Limitations

    The measured group-size patterns provide scope for system optimization and group-recommendation policies.

    Source: §IV · p. 9

    The authors identify unresolved security and privacy challenges that may grow as the platform scales to nation-wide or world-wide users and organizations.

    Source: A. Privacy, Security and Energy Issues… · p. 11

Abstract

from arXiv · show

Recently the topic of how to effectively offload cellular traffic onto device-to-device (D2D) sharing among users in proximity has been gaining more and more attention of global researchers and engineers. Users utilize wireless short-range D2D communications for sharing contents locally, due to not only the rapid sharing experience and free cost, but also high accuracy on deliveries of interesting and popular contents, as well as strong social impacts among friends. Nevertheless, the existing related studies are mostly confined to small-scale datasets, limited dimensions of user features, or unrealistic assumptions and hypotheses on user behaviors. In this article, driven by emerging Big Data techniques, we propose to design a big data platform, named D2D Big Data, in order to encourage the wireless D2D communications among users effectively, to promote contents for providers accurately, and to carry out offloading intelligence for operators efficiently. We deploy a big data platform and further utilize a large-scale dataset (3.56 TBytes) from a popular D2D sharing application (APP), which contains 866 million D2D sharing activities on 4.5 million files disseminated via nearly 850 million users in 13 weeks. By abstracting and analyzing multidimensional features, including online behaviors, content properties, location relations, structural characteristics, meeting dynamics, social arborescence, privacy preservation policies and so on, we verify and evaluate the D2D Big Data platform regarding predictive content propagating coverage. Finally, we discuss challenges and opportunities regarding D2D Big Data and propose to unveil a promising upcoming future of wireless D2D communications.

I. INTRODUCTION

The paper motivates D2D sharing as a way to offload cellular traffic, but notes that prior studies largely use small user groups. It proposes combining big-data techniques with large-scale D2D analysis to support sharing, content promotion, and operator offloading.

I. INTRODUCTION

  • D2D lets users obtain interesting files from nearby devices, potentially offloading duplicate cellular downloads and relieving infrastructure bottlenecks.The paper includes multiple wireless D2D techniques, including 3GPP D2D underlaying LTE networks.Source: §I · p. 2
  • Earlier studies largely analyze groups of hundreds or thousands, leaving large-scale D2D delivery across millions of users and wide areas unaddressed.The paper identifies this gap as motivation for integrating big-data techniques and using activity logs, traffic logs, and user and content profiles.Source: §I · p. 2
  • The framework is motivated by the complex, heterogeneous data and multidimensional user and content features available for large-scale D2D analysis.The paper points to geographical homophily, social relationships, interest similarity, mobility, content properties, and sharing graphs as relevant features and structures.Source: §I · pp. 2–3
  • The proposed D2D Big Data framework aims to encourage sharing, promote content for providers, and support operator offloading intelligence.The authors describe a cloud-based platform and identify measurement, analytics, learning, prediction, and recommendation among its contributions.Source: §I · pp. 3–4

II. D2D BIG DATA FRAMEWORK

D2D Big Data extends sharing tools toward large-scale offline content dissemination and mobile-interest services. Its framework comprises promotion, interest-exploitation, and analysis platforms, aimed respectively at provider strategies, tailored recommendations, and feature extraction from D2D traces.

II. D2D BIG DATA FRAMEWORK

  • The framework is intended to extend D2D sharing beyond small-user studies toward offline content dissemination and mobile-interest services for users across large areas.The paper describes its industrial vision as moving valuable content into offline, face-to-face D2D delivery and its academic objective as processing data from millions of users.Source: §II · pp. 4–5
  • Content Promotion Big Data supports provider promotion strategies through content uploading, budgeting, supervision, visualization, and analysis of user and social features.The platform targets rapid content promotion through D2D deliveries.Source: §II · pp. 4–5
  • Interest Exploitation Big Data mines user preferences and sharing traces to recommend interests, discover nearby users’ interests, establish groups, and find content.Its recommendations use preferences, sharing habits, and content popularity for customization.Source: §II · p. 5
  • D2D Analysis Big Data processes datasets and extracts multidimensional features from user traces, including online behavior, location, content, meetings, and friendship trees.The analysis platform uses Spark-based big-data tools for processing and prediction.Source: §II · p. 5

III. MEASUREMENT AND ANALYTICS OVER REALISTIC DATA AND SYSTEM

The system combines large-scale data inputs with streaming and batch processing and learning-based prediction and recommendation. The authors implemented it on a cloud-based cluster and evaluated it using a 13-week, 3.56-TB Xender dataset, while noting that filtered records limit its representativeness of business scale.

III. MEASUREMENT AND ANALYTICS OVER REALISTIC DATA AND SYSTEM

  • The framework processes user profiles, content statistics, activity logs, and traffic records through streaming and batch workflows, then uses learning tools for predictions and recommendations.The authors say resulting analytics can inform mobile-app marketing and development strategies.Source: §III · p. 5, Figure 3 · p. 6
  • The implemented platform uses HDFS for scalable storage, Spark for in-memory processing, and OpenStack virtual machines to host big-data tools.The implementation uses a 24-server cluster with Python, Hadoop, Spark, MLlib, GraphX, and Kafka-supported processing.Source: §III · p. 6
  • The Xender dataset spans 13 weeks and approximately 3.56 TBytes, and the authors caution that it does not exactly reflect realistic business scale.They report filtering out many incomplete or invalid records and note that their experimental system is smaller than industrial big-data systems.Source: §III · pp. 6–7
  • The platform extracts online behavior, location relations, meeting dynamics, content properties, social characteristics, and popularity trends for analysis across user and content features.The authors describe these features as supporting further exploitation of offline D2D deliveries, including incentive-based promotion and pushing content to influential users.Source: §III · p. 7

IV. RESULTS OF MEASUREMENT, ANALYSIS, AND PREDICTION BY D2D BIG DATA PLATFORM

The platform reports measurements of D2D traffic, analyses of offline social groups, and experiments predicting content propagation. Its findings include group structure, feature-selection tradeoffs, duplicate traffic, and seed-based coverage.

Social graph analysis

  • D2D groups follow a power-law size distribution, with few large groups and many small ones, offering scope for group-level optimization and recommendations.Groups are formed from users who have shared at least one file via D2D.Source: §IV · pp. 8–9, Figure 4 · p. 8
  • Group diameters of around 6 to 10 are larger than those of online social networks, consistent with D2D sharing’s spatial and temporal constraints and strong user demands.Source: §IV · pp. 8–9

Multi-feature prediction

  • Prediction accuracy varies across feature combinations, and the authors recommend selectively analyzing two or three features rather than all features because of the efficiency tradeoff.The described workflow trains on empirical data, such as six weeks, and tests on later data, such as seven weeks.Source: §IV · pp. 9–10, Figure 5 · p. 9

Traffic measurement

  • APPs and videos have a high percentage of duplicate traffic, indicating skewed popularity and motivating prioritization for resource optimization.Source: §IV · p. 10, Figure 6 · p. 10

Predictive propagation

  • Nearly half of users in 100 randomly selected groups received content through offline D2D sharing when each group’s most influential seed was selected.The evaluation assumes users share content when they encounter one another in the second half of the trace.Source: §IV · p. 10, Figure 6 · p. 10

V. OPPORTUNITIES AND CHALLENGES

The section introduces research directions and pending problems focused on widening the use of D2D techniques.

  • The authors identify promising research directions and pending problems for elaborating and widening D2D usage.Source: §V · p. 11

A. Privacy, Security and Energy Issues for D2D Big Data

Large-scale D2D Big Data leaves privacy, security, fairness, and device-energy challenges unresolved.

Privacy and security

Energy

B. Big Data-assisted Base Station Caching Integrated with D2D

The section identifies untested integration of base-station caching with D2D and outlines resource-allocation and coverage questions for future work.

Integration gap

Resource allocation

Nationwide coverage

C. Business Models for D2D Big Data

The section proposes exploring business models for offline D2D sharing beyond users’ common interests. It highlights incentive-based forwarding and asks whether apps, music, and video could reach suitable audiences through user-to-user delivery.

C. Business Models for D2D Big Data

VI. CONCLUSION

The conclusion presents D2D Big Data as an integration of wireless D2D and big data techniques for traffic offloading and content dissemination. The authors report implementing a realistic platform and analyzing real data, and identify future work in recommendation, prediction, and cross-sector collaboration.

VI. CONCLUSION

  • The authors implemented a realistic D2D Big Data platform and used a significant volume of real data to conduct measurement, analysis, and evaluation.Source: §VI · p. 12
  • The framework aims to encourage D2D deliveries, promote contents for providers, and provide offloading intelligence for operators.Source: §VI · p. 12
  • The authors discuss implications, techniques, opportunities, and challenges for improving D2D sharing services and D2D Big Data capabilities.Source: §VI · p. 12
  • Future work includes improved group recommendation and more accurate prediction of content popularity and user behaviors.Source: §VI · p. 12
  • The authors also propose multidisciplinary collaboration involving MNOs, industrial vendors, content service providers, and government copyright policies.Source: §VI · p. 12
Loading 1802.00893v1…