PaperlayerView exact revision on arXiv

Source-linked AI summary

Interactive Analytical Processing in Big Data Systems: A Cross-Industry Study of MapReduce Workloads

Yanpei Chen, Sara Alspaugh, Randy Katz

arXiv:1208.4174v1cs.DB

TL;DR

MapReduce workloads increasingly include interactive, small, short jobs, but these production workloads were not well characterized across industries. The paper analyzes industrial traces from Facebook and Cloudera customers, finding diverse workload behaviors and many small jobs that challenge batch-oriented assumptions. It also identifies practical difficulties in building a representative TPC-like benchmark.

  • Problem

    MapReduce lacked empirical cross-industry workload characterization despite the emergence of interactive jobs and query-like processing needs.

  • Method

    The paper empirically analyzes industrial MapReduce traces using data, temporal, and compute dimensions, supplemented by job-level Hadoop logs.

  • Results

    Small jobs form over 90% of all jobs for all workloads, while the workloads exhibit diverse behaviors that invalidate assumptions about uniform access, regular diurnal patterns, and predominantly large jobs.

  • Takeaways & Limitations

    Performance claims should be qualified by workload assumptions, and system engineers should reassess design priorities as use cases evolve.

  • Takeaways & Limitations

    Constructing a representative processing stream is difficult because it must capture job size, shape, sequence, dependencies, and aggregate load variation.

Abstract

from arXiv · show

Within the past few years, organizations in diverse industries have adopted MapReduce-based systems for large-scale data processing. Along with these new users, important new workloads have emerged which feature many small, short, and increasingly interactive jobs in addition to the large, long-running batch jobs for which MapReduce was originally designed. As interactive, large-scale query processing is a strength of the RDBMS community, it is important that lessons from that field be carried over and applied where possible in this new domain. However, these new workloads have not yet been described in the literature. We fill this gap with an empirical analysis of MapReduce traces from six separate business-critical deployments inside Facebook and at Cloudera customers in e-commerce, telecommunications, media, and retail. Our key contribution is a characterization of new MapReduce workloads which are driven in part by interactive analysis, and which make heavy use of query-like programming frameworks on top of MapReduce. These workloads display diverse behaviors which invalidate prior assumptions about MapReduce such as uniform data access, regular diurnal patterns, and prevalence of large jobs. A secondary contribution is a first step towards creating a TPC-like data processing benchmark for MapReduce.

1. INTRODUCTION

MapReduce workloads have expanded beyond large batch jobs to include small, short, interactive analysis across diverse industries. This paper characterizes those workloads empirically and identifies behaviors that challenge prior assumptions.

  • Organizations now use MapReduce for many small, short, increasingly interactive jobs alongside traditional large batch computations.
  • The literature lacked cross-workload evidence spanning multiple companies and industries, limiting production benchmarking and configuration guidance.
  • The study analyzes seven long-duration industrial traces from Facebook and Cloudera customers in e-commerce, telecommunications, media, and retail.The traces cover over a year, more than two million jobs, approximately 1.6 exabytes, and 5000 machines.
  • The methodology separates workload behavior into data, temporal, and compute patterns, finding diverse interactive and semi-streaming workloads.Query-like frameworks such as Hive and Pig account for a considerable fraction of activity.
  • The workloads challenge assumptions of uniform data access, regular diurnal patterns, and predominantly large jobs.
  • The study identifies challenges in constructing a representative TPC-style MapReduce benchmark, including scaling production behavior and modeling diverse workload characteristics.

2. PRIOR WORK

Prior MapReduce workload research was fragmented across individual technology-industry use cases and stand-alone microbenchmarks. This paper motivates cross-workload analysis to identify common behavior and test established workload assumptions.

  • Workload characterization has historically supported system design, with database research producing TPC benchmarks and established access and query-size patterns.
  • MapReduce lacked comparable cross-workload insights, hindering development of a representative TPC-like benchmark suite.
  • Existing MapReduce microbenchmarks focus on a small number of large, stand-alone jobs rather than workloads composed of many heterogeneous jobs.
  • Earlier workload studies generally examined one technology-industry use case at a time, creating a need to generalize across deployments and industries.
  • Hypotheses on Workload Behavior: The analysis asks about data-access skew and locality, load regularity and bursts, job types and durations, and query-like framework use.
  • Hypotheses on Workload Behavior: Prior mental models expected predictable diurnal load and workloads dominated by large fixed-computation jobs, but the study finds tremendous diversity and opposing behaviors.

3. WORKLOAD TRACES OVERVIEW

The study uses production Hadoop traces from Facebook and Cloudera customers across several industries, collected from standard logs and summarized at the job level. The traces span substantial operational history despite practical collection constraints.

  • The dataset contains seven business-critical Hadoop workloads: five from Cloudera customers and two Facebook periods on one cluster.
  • The traces cover over a year of activity and a significant volume of jobs and processed bytes across the clusters.
  • Standard Hadoop logging supplied per-job identifiers, names, input, shuffle, and output sizes, durations, submission times, task times, task counts, and file paths.Some traces lack certain dimensions.
  • Cloudera traces came from time-range selections of Hadoop history logs, while Facebook traces came from queries over an internal log database.
  • Table 1 summarizes the traces, distinguishing Cloudera Customer and Facebook workloads and defining bytes moved as total input, shuffle, and output bytes.
  • Trace boundaries introduce partial information for jobs spanning the start or end, but the long trace durations make resulting errors negligible.Nine days were queried for two workloads to capture weekly behavior.

4. DATA ACCESS PATTERNS

MapReduce data access patterns vary widely in per-job size, skew, and temporal locality, challenging assumptions that workloads are dominated by uniformly accessed large datasets. Despite this diversity, access frequencies are consistently Zipf-like, and many workloads repeatedly access data within hours.

  • Data access analysis examines per-job data sizes, access-frequency skew, and temporal locality as MapReduce counterparts to query size and access behavior in RDBMSs.
  • 4.1 Per-job Data Sizes: Median per-job input, shuffle, and output sizes differ across workloads by 6, 8, and 4 orders of magnitude, respectively.Most jobs have input, shuffle, and output sizes in the MB-to-GB range, so TB-scale benchmarks cover only a narrow set of patterns.
  • 4.1 Per-job Data Sizes: Facebook workloads shifted toward larger per-job input and shuffle data but smaller output data from 2009 to 2010.The paper attributes this pattern possibly to growing raw datasets while final business metrics remained similar in size.
  • 4.2 Skews in Access Frequency: File access frequencies are Zipf-like, with a few files receiving very high access counts, challenging equal treatment of datasets in HDFS.The observed skew suggests tiered storage architectures rather than identical media and replication policies for all data.
  • 4.2 Skews in Access Frequency: 90% of jobs access files smaller than a few GB, yet those files account for up to only 16% of stored bytes across input and output data.The distributions of job fractions vary widely by workload, while converging in the upper-right region of the plots.
  • 4.3 Access Temporal Locality: 75% of data re-accesses occur within 6 hours, while up to 78% of jobs involve re-accesses in some workloads.These patterns support workload-specific cache eviction thresholds, but potential cache benefits differ across workloads.
  • Preserving data size, access-frequency skew, and temporal locality in performance comparisons is non-trivial because workload behavior is highly diverse.The Zipf-like distribution shape is the only numerical feature reported as relatively fixed across workloads, so no behavior should be treated as universally typical.

5. WORKLOAD VARIATION OVER TIME

MapReduce workloads vary substantially over time, with noisy, bursty, and sometimes diurnal behavior across jobs, data, computation, and utilization. These dimensions are only weakly aligned except for a stronger relationship between data size and task time, complicating single-metric evaluation and workload management.

  • Weekly Time Series: Workload intensity is analyzed across job submissions, aggregate I/O, aggregate task time, and cluster utilization.Cluster utilization reflects how the execution environment services the submitted workload.
  • Weekly Time Series: The time series are noisy, and job counts do not reliably predict the resulting I/O or computation.The traces lack known signal and noise models, making standard signal-to-noise quantification difficult.
  • Weekly Time Series: Some workloads exhibit daily diurnal patterns, while others show substantial variation across workloads and dimensions.Figure 7 displays diversity both among dimensions within a workload and across workloads for the same dimension.
  • Burstiness: All workloads are highly bursty, with peak and minimum submission rates orders of magnitude from their median rates.The burstiness metric normalizes task-time distributions by each workload’s median; more horizontal curves indicate greater burstiness.
  • Burstiness: Facebook’s peak-to-median ratio fell from 31:1 to 9:1 over a year but remained bursty.The decrease accompanied broader adoption of MapReduce by internal organizations.
  • Temporal Correlations: Average temporal correlation is 0.62 between data size and task time, versus 0.21 for job count and data size and 0.14 for job count and task time.The strongest relationship indicates that these workloads remain data-centric rather than compute-centric.
  • Temporal Correlations: Maximum jobs per second is an inadequate performance metric because burst impact depends on aggregate data, compute, and workload-management decisions.A MapReduce benchmark should therefore use multiple performance metrics.

6. COMPUTATION PATTERNS

The computation analysis finds that industrial MapReduce workloads are dominated by a small number of frameworks and computation types, while job sizes and execution behavior remain diverse. Most jobs are small enough for interactive latency, but some low-parallelism jobs run for days, creating scheduling and straggler challenges.

  • Scope and Questions: The analysis examines framework usage, common computation types, and job size, shape, duration, and frequency.It treats map and reduce steps as analogs of database operators and data volume as an analog of cardinality.
  • By Job Names: A small number of common computation types dominate each workload, creating targets for static or manual optimization.These regularities can simplify duration and resource prediction, scheduling, placement, and task-granularity optimization.
  • By Job Names: Two frameworks account for a dominant majority of jobs in every workload.The workloads use small sets of frameworks such as Hive, Pig, and Oozie, suggesting that multiplexing two or three frameworks covers most jobs.
  • By Job Names: Hive select and insert queries form a large fraction of activity in several workloads, alongside evidence of migrated ETL and EDW use cases.The available job-name information identifies likely targets for query optimization but does not provide direct query text.
  • By Multi-Dimensional Job Behavior: More than 92% of jobs touch less than 10GB of total data and can achieve analyst-facing durations under a minute.This distribution counters the assumption that MapReduce workloads consist primarily of jobs at terabyte scale or larger.
  • By Multi-Dimensional Job Behavior: Map-only jobs account for 7% to 77% of bytes and 4% to 42% of task time across workloads.They include launcher and maintenance jobs and benefit less from networks optimized for shuffle patterns.
  • By Multi-Dimensional Job Behavior: Some jobs process comparable data volumes but run much longer because they have inherently low parallelism; one FB-2010 map-only transform takes 3 days.These jobs cannot exploit available cluster parallelism even when spare capacity exists.
  • By Multi-Dimensional Job Behavior: Small jobs dominate all workloads but differ in shape, size, and duration, so representative benchmarks require many job types rather than a few microbenchmarks.Facebook’s job profiles also changed significantly over one year, requiring policy parameters to be revisited periodically.

7. TOWARDS A BIG DATA BENCHMARK

A representative MapReduce benchmark must model diverse, realistic processing streams rather than isolated jobs or a single workload profile. The paper identifies major challenges in representing data access, job generation, workload scaling, statistical behavior, and mixed processing frameworks.

  • Data generation: Realistic benchmarks must reproduce data-size ranges, access-frequency skew, and temporal locality because each affects system performance.This requires pre-generating data that reflects complex real-world access patterns.
  • Processing generation: Processing-stream generation must capture job size, shape, sequence, and aggregate cluster-load variation while accounting for dependencies among features.The paper notes that deciding which features can be omitted across performance comparisons is difficult.
  • Mixed processing: Because workloads use both native MapReduce and query-like frameworks, benchmarks should mix and multiplex both processing types realistically.The paper specifically identifies Hive, Pig, and HBase as query-like frameworks requiring representation.
  • Scaled-down workloads: Production-scale reproduction is economically challenging, and the best way to normalize workload or cluster size for scale-down remains unclear.Possible normalization dimensions include data size, job count, processing per data, node count, CPU capacity, and available memory.
  • Empirical models: Observed workload behavior generally fits no well-known statistical distribution, except Zipf-like data-access frequency, so benchmark traces may need to serve as empirical models.This departs from benchmark approaches that generate data and processing streams using simple statistical models.
  • Benchmark perspective: A big data benchmark should treat workloads as steady processing streams that superpose many processing types, rather than microbenchmarks running isolated jobs.Microbenchmarks remain useful for diagnosing specific system subcomponents, but they do not represent production workload behavior.

8. SUMMARY AND CONCLUSIONS

The study finds broad variation across industrial MapReduce workloads, including skewed and temporally local data access, bursty load, and a dominance of small jobs. It therefore recommends workload-aware evaluation, richer replay and monitoring tools, and caution about claims of typical behavior.

  • Job scheduling: Small jobs are the most common job type across workloads and typically involve tens of kilobytes to gigabytes of data with durations from tens of seconds to a few minutes.These jobs are small across all dimensions relative to other jobs in the same workload.
  • Query-like frameworks: Query-like frameworks are generally used for interactive data exploration and semi-streaming analysis, with framework usage ranging from at least 20% to up to 80%.Hive commonly uses insert and select operators, while additional Hive/Pig/HBase tracing is needed for deeper analysis.
  • Performance comparison: The diversity between workloads makes claims about “typical” behavior risky and implies that performance claims should specify underlying workload assumptions.The paper recommends replaying a suite of workloads and reassessing system-design priorities as use cases evolve.
Loading 1208.4174v1…