PaperlayerView exact revision on arXiv

Source-linked AI summary

A First Look at Commercial 5G Performance on Smartphones

Arvind Narayanan, Eman Ramadan, Jason Carpenter, Qingxu Liu, Yu Liu, Feng Qian, Zhi-Li Zhang

arXiv:1909.07532v2cs.NIcs.PF

TL;DR

This paper addresses limited evidence about commercial 5G performance on smartphones by measuring three carriers across diverse urban settings and examining network, mobility, prediction, and application behavior. It finds that 5G performance varies across conditions and layers: mmWave can deliver much higher throughput, but handoffs, environmental sensitivity, and application overhead limit consistent user benefits.

  • Problem

    Commercial 5G smartphone performance across network conditions, handoffs, environmental prediction, and applications had not been quantitatively characterized in a unified field study.

  • Method

    The study conducts field measurements across three U.S. carriers and cities, analyzing throughput, latency, handoffs, location-based prediction, and web and HTTP-download performance on COTS smartphones.

  • Results

    The study finds that stationary mmWave 5G significantly outperforms mid-band 5G and improves throughput over 4G by ∼10x, while offering little latency improvement and inconsistent application-level benefits.

  • Takeaways & Limitations

    The findings provide a baseline for tracking commercial 5G evolution and identify cross-layer research directions for improving 5G user experience.

  • Takeaways & Limitations

    The sampled locations are too limited to compare or rank carriers, particularly while commercial 5G remains at an early stage.

Abstract

from arXiv · show

We conduct to our knowledge a first measurement study of commercial 5G performance on smartphones by closely examining 5G networks of three carriers (two mmWave carriers, one mid-band carrier) in three U.S. cities. We conduct extensive field tests on 5G performance in diverse urban environments. We systematically analyze the handoff mechanisms in 5G and their impact on network performance. We explore the feasibility of using location and possibly other environmental information to predict the network performance. We also study the app performance (web browsing and HTTP download) over 5G. Our study consumes more than 15 TB of cellular data. Conducted when 5G just made its debut, it provides a "baseline" for studying how 5G performance evolves, and identifies key research directions on improving 5G users' experience in a cross-layer manner. We have released the data collected from our study (referred to as 5Gophers) at https://fivegophers.umn.edu/www20.

1 INTRODUCTION

This paper presents an early, user-centric measurement study of commercial 5G on smartphones, spanning carriers, cities, mobility, environment, handoffs, prediction, and applications. It finds substantial throughput gains but highlights variability, limited latency improvement, mobility challenges, and cross-layer barriers to application QoE.

  • Study scope: The study measures three U.S. carriers across Minneapolis, Chicago, and Atlanta using commercial smartphones, including two mmWave networks and one mid-band network.The experiments span diverse urban environments and consume more than 15 TB of cellular data.
  • Overview of today’s 5G performance: Average mmWave 5G throughput significantly outperforms mid-band 5G in typical stationary urban environments, while commercial 5G offers little latency improvement.The limited latency gain is attributed to the Non-Standalone deployment model retaining much of the existing 4G infrastructure.
  • Stationary performance: ∼10x improvement in throughput over 4G is observed for commercial mmWave 5G with stationary smartphones.The study also finds substantially higher throughput variation than 4G, even under clear line-of-sight.
  • Mobility performance: 30+ 4G/5G handoffs in less than 8 minutes can occur even while walking, producing inconsistent experiences and confusing application adaptation.Mid-band 5G and 4G provide better mobility stability than mmWave 5G because their radio signals are omni-directional.
  • Location-based performance estimation: Location-based throughput prediction is difficult because small environmental perturbations, including orientation, moving objects, humidity, and phone cases, affect mmWave performance.A 30-day field study found higher throughput variation at a given location for mmWave 5G than for 4G.
  • Application performance: 23.5% higher average median download time occurs with HTTPS than HTTP because of TLS overhead, and high mmWave bandwidth does not always improve application QoE.Web browsing benefits mainly for large pages, while HTTP/2 and HTTP/3 (QUIC) optimizations remain effective over 5G.

2 BACKGROUND AND RELATED WORK

The paper frames commercial 5G around mmWave’s abundant bandwidth and environmental fragility, contrasts it with mid-band deployment, and introduces NSA infrastructure and the smartphone measurement gap.

  • Background: mmWave 5G uses 24–53 GHz spectrum and directional phased-array beams, but attenuation and blockage can sharply reduce or interrupt data rates.Signals may switch from line-of-sight to non-line-of-sight paths when blocked.
  • Related Work: Prior work studied mmWave across testbeds and environments, but not in commercial 5G contexts on commodity smartphones.This gap motivates the paper’s first measurement study of commercial mmWave 5G on smartphones.
  • Background: Mid-band 5G operates at 1–6 GHz with largely omni-directional propagation, offering decent data rates but potentially less long-term spectrum availability.The paper contrasts mid-band’s propagation advantages with mmWave’s speed and spectrum advantages.
  • Background: All carriers’ initial commercial deployments use Non-Standalone architecture, combining 5G-NR data-plane operations with existing 4G control-plane infrastructure.Standalone deployment is defined as independent of legacy cellular infrastructure.

3 MEASUREMENT METHODOLOGY

The study measures commercial 5G using commodity smartphones, multiple carriers and urban sites, while controlling server selection and collecting transport, application, location, and network-state data.

  • Measurement Scope: The experiments cover VZ, T-Mobile, and Sprint across Minneapolis, Chicago, and Atlanta, including mmWave and 2.5 GHz mid-band networks.VZ and T-Mobile use mmWave, while Sprint uses mid-band 5G.
  • Measurement Scope: The researchers use Samsung Galaxy S10 5G smartphones after finding that the Motorola Moto Z3 significantly underperforms in 5G throughput.The device choice is intended to reduce hardware-related measurement artifacts.
  • Experiment Sites: Experiments span representative urban settings including crowded and open spaces, indoor and outdoor locations, and different building densities.Four primary Verizon sites include downtown, coverage-boundary, hotel-window, and stadium-area environments.
  • Server Selection: Server selection is tested across cloud providers and locations, with most experiments using a Microsoft Azure server on the U.S. east coast.The selected server provided statistically highest 5G throughput and a wired path exceeding measured 5G speeds.
  • Test Workload: The workload measures TCP/UDP throughput, RTT, packet loss, HTTP(S) downloads, and web-page loading using bulk transfers and iPerf.Bulk downloads use one or more TCP connections or a UDP session to an Internet server.
  • Monitoring: A custom UE-side tool records fine-grained location, network interfaces, active interface, and 5G connection status because available Android tooling lacked 5G-NR support.The tool distinguishes no coverage, 4G connection within 5G coverage, and active 5G connection.

4 OVERVIEW OF TODAY’S 5G PERFORMANCE

Across diverse Atlanta urban sites, commercial 5G delivers very high but volatile throughput, with mmWave outperforming mid-band, while latency remains close to 4G and uploads lag downloads. The authors present these findings as an early baseline rather than a carrier ranking.

  • Performance Overview: Up to 2 Gbps TCP downlink throughput was measured on Verizon’s network in real-world urban environments.This exceeds the paper’s comparison with top residential broadband performance.
  • Performance Overview: mmWave carriers VZ and T-Mobile provide much higher median throughput than Sprint’s 2.5 GHz mid-band network.The result reflects the higher-speed advantage associated with mmWave in these measurements.
  • Performance Overview: Commercial 5G throughput varies substantially, sometimes dropping close to zero for VZ and T-Mobile.The paper reports this variation alongside otherwise high observed performance.
  • Performance Overview: Today’s commercial 5G offers little latency improvement over 4G, which the authors attribute to NSA infrastructure and limited edge support.The paper links both factors to the continued high end-to-end latency.
  • Performance Overview: Upload speeds peaked at ∼60 Mbps for VZ and T-Mobile and were around 30 Mbps for Sprint, so the paper primarily studies downloads.The reported upload speeds were far below download speeds during the study.
  • Scope: The study does not rank carriers because its sampled locations are limited and commercial 5G was still at an early stage.The results are intended as a first impression of commercial 5G services.

5 5G PERFORMANCE OF STATIONARY UE

Stationary mmWave 5G delivers much higher throughput than 4G, but performance varies substantially with transport concurrency and environmental conditions. Latency remains similar to 4G, while obstructions, orientation, distance, and client sharing constrain reliability.

  • TCP/UDP Performance Under LoS: 1467 Mbps versus 167 Mbps: median 5G throughput exceeds 4G with eight parallel TCP connections, despite much higher 5G variation.The comparison is measured under clear line of sight; the study attributes variation to PHY/MAC and other-layer inefficiencies.
  • TCP/UDP Performance Under LoS: More than eight concurrent TCP connections fully utilize 5G bandwidth, unlike wired, WiFi, and 4G networks.The authors suggest per-connection rate limiting or limited single-connection support as possible explanations.
  • TCP/UDP Performance Under LoS: Around 56 ms: 5G and 4G have similar base RTT, with the first hop contributing about 28 ms to end-to-end RTT.The authors relate the limited latency improvement to today’s NSA deployment.
  • TCP/UDP Performance Under LoS: Two devices sharing one panel divide its available bandwidth approximately evenly, suggesting concurrent clients can reduce perceived throughput.In one test, a roughly 1.8 Gbps maximum was divided into averages of about 882 Mbps and 931 Mbps.
  • Impact of the Environment: A 40% median throughput drop occurs at 90° UE-panel orientation, while 0° and 45° show little difference.The measurements use 8 parallel TCP connections at a fixed 25 m distance; reflections and beamforming are cited as explanations for the smaller difference.
  • Impact of the Environment: Obstruction generally has the largest environmental impact: bodies, trains, pillars, and tinted glass are difficult for mmWave signals to penetrate.Nearby building reflections can sometimes preserve connectivity under NLoS conditions.

6 MOBILITY AND LOCATION-AWARENESS

Mobility produces frequent 4G/5G transitions and highly variable mmWave throughput, while fine-grained location alone is insufficient for stable performance prediction. The study therefore motivates context-aware prediction using environmental and behavioral information.

  • Handoff and Mobility Performance: About 10 seconds of user-traffic inactivity can trigger a 5G-to-4G handoff, while renewed traffic can restore 5G when signal quality permits.The stated rationale is reducing energy consumed by 5G standby operation.
  • Handoff and Mobility Performance: Primitive handoffs combine into higher-level sequences; for example, a 5G-to-5G handoff can require downgrade, panel change, and upgrade steps over several seconds.The study identifies these sequences by clustering primitive handoffs within a 10-second interval.
  • Handoff and Mobility Performance: 31 primitive handoffs and 13 4G/5G bounces during an eight-minute walk produce throughput ranging from 0 to 954 Mbps.The authors connect these fluctuations to inconsistent application experiences and propose cross-layer adaptations.
  • Handoff and Mobility Performance: T-Mobile handles handoffs more poorly than Verizon in the driving test, while Sprint shows higher throughput and lower variation during most of the trace.At t=110 s, one T-Mobile handoff completely disconnects the TCP connections.
  • Location-based Performance Estimation: At a given location, normalized throughput variation is 0.70 for 4G versus 1.07 for 5G across 2-meter segments.The result indicates that location-based estimation is more difficult for mmWave 5G than for 4G.
  • Location-based Performance Estimation: Small changes in orientation, moving objects, humidity, or phone cases can alter mmWave performance, motivating context-aware prediction beyond location alone.Suggested contextual inputs include time, mobility, weather, traffic, and train schedules.

7 APPLICATION PERFORMANCE

Over 5G, application performance depends on page size, protocol behavior, and Internet-side bottlenecks rather than wireless throughput alone. 5G benefits large sites and downloads, while HTTP/2 and HTTP/3 provide effective protocol optimizations.

  • Web Page Loading: For most pages of 3 MB or less, 4G and 5G achieve similar page-loading times.Client-side processing, unchanged RTT, and Internet-side bottlenecks can limit the benefit of faster 5G access.
  • Web Page Loading: Compared with 4G, today’s 5G benefits only large, content-rich sites, while HTTP/2 and HTTP/3 optimizations remain effective over 5G.The authors identify standalone 5G, faster client computation, and edge caching as possible page-load improvements.
  • HTTP(S) Download: The download experiment uses a 1 GiB file and eight parallel byte-range requests across geographically distributed cloud and CDN servers.Tests are repeated at two clear-line-of-sight locations using a custom HTTP(S) client.
  • HTTP(S) Download: 119 to 730 Mbps: HTTP(S) downloads across cloud and CDN servers have a median throughput of 222 Mbps, well below iPerf measurements.The study attributes the gap to factors including DNS, request latency, server processing, and TLS encryption.

8 CONCLUDING REMARKS

The study quantitatively reveals 5G performance on commodity smartphones and identifies cross-layer directions for improving users’ experience.

  • The study quantitatively reveals 5G performance on COTS smartphones.
  • It identifies research directions spanning transport protocols, interface selection, and mobile operating-system support for 5G QoE.
  • The publicly available 5Gophers dataset is intended to foster further 5G research.
Loading 1909.07532v2…