Characterizing Data Analysis Workloads in Data Centers

• The study compares eleven workloads selected from search, social-network, and e-commerce domains, using larger datasets and whole-run measurements after warm-up than the cited CloudSuite approach.The data range from 147 to 187 GB, compared with 4.5 GB for Naive Bayes in.Source: §I · p. 1
• Data analysis workloads differ from desktop, HPC, traditional-server, and scale-out-service workloads, while scale-out services resemble SPECweb2005 in micro-architectural characteristics.The authors released DCBench for data-center workloads.Source: §I · pp. 1–2
• Data analysis workloads have higher IPC than service workloads but lower IPC than computation-intensive HPCC workloads such as HPC-HPL and HPC-DGEMM.Source: §I · p. 2
• Data analysis workloads average about 57% of stalls in the out-of-order pipeline, whereas service workloads average about 73% before instructions enter it.Source: §I · p. 2
• Data analysis workloads average about 11 L2 cache misses per thousand instructions versus about 60 for service workloads, and the authors suggest optimizing LLC capacity for energy efficiency and die area.The data-analysis average is higher than HPCC workloads; 85.5% of data-analysis L2 misses and 94.9% of service L2 misses hit in L3.Source: §I · p. 2
• The authors report lower branch-misprediction ratios than in most service workloads and suggest a simpler branch predictor to save power and die area.Source: §I · p. 2

A. Workloads Requirements

• The workload-selection requirements are representation of important application domains, distribution across nodes, and recent use with emerging techniques.Source: A. Workloads Requirements · p. 2

B. Can one data analysis workload represent all?

• CloudSuite includes four scale-out service benchmarks but only one data-analysis workload, Naive Bayes; the authors conclude that one application cannot represent varied data-analysis categories.Source: B. Can one data analysis workload… · pp. 2–3
• On an eight-node system, workload speedups range from 3.3 to 8.2, while Naive Bayes reaches 6.6, demonstrating variation across the eleven workloads.Speedup uses the one-slave runtime as the baseline; the experiments also find differing micro-architectural characteristics.Source: B. Can one data analysis workload… · p. 3, Figure 2 · p. 3

C. Workloads Choosing

• The authors focus on search, social-network, and e-commerce domains after classifying the top 20 sites by application category and share.Sort, WordCount, and Grep are included as frequently used data-analysis operations.Source: C. Workloads Choosing · p. 3, 1) Basic operations · p. 3

2) Classification and Clustering:

• The selected classification workloads are Naive Bayes and SVM, while K-means and Fuzzy K-means represent clustering algorithms used across the chosen domains.Source: 2) Classification and Clustering · p. 3
• Fuzzy K-means extends K-means by allowing an observation to belong to multiple clusters with given probabilities.Source: 2) Classification and Clustering · p. 3

3) Recommendation:

• Item Based Collaborative Filtering is selected as a recommendation workload and estimates a user's preference for an item from preferences toward related items.Source: 3) Recommendation · p. 3

4) Segmentation:

• Web-search segmentation is implemented with a Hidden Markov Model, which models a Markov process with unobserved hidden states.Segmentation is described as important for web search, especially for Chinese.Source: 4) Segmentation · p. 3

6) Data warehouse operations:

• Hive-bench supplies representative SQL-like statements for the data-warehouse workloads and is based on Hive.The paper uses a series of these statements as part of its workload set.Source: 6) Data warehouse operations · p. 3

III. EXPERIMENTAL SETUP

• The experimental setup section covers both the study environment and its methodology.Source: §III · p. 4

A. Hardware Configurations

• The data-analysis workloads run on a five-node Hadoop cluster with Xeon E5645 processors, private L1 and L2 caches, and shared L3 cache.Each node has two processors, each with six physical out-of-order cores.Source: A. Hardware Configurations · p. 4, Table III · p. 4

B. Hadoop Cluster Environments

• All workloads use Hadoop 1.0.2 and JDK 1.6.0 on CentOS 5.5, with 24 map slots, 12 reduce slots, and a 1 GB heap per task.Data-warehouse workloads use Hive 0.6; the Linux kernel is 2.6.34.Source: B. Hadoop Cluster Environments · p. 4

C. Compared Benchmarks setups

• In addition to data-analysis workloads, the study deployed several benchmark suites, including SPEC CPU2006, HPCC, SPECweb 2005, and CloudSuite, a scale-out benchmark suite for cloud computing.Source: C. Compared Benchmarks setups · p. 4

1) Traditional benchmarks setups:

• Traditional benchmarks use SPEC CPU2006 reference inputs, seven HPCC benchmarks, and SPECweb2005’s bank application with 3,000 simultaneous sessions.SPEC CPU results are averaged into integer and floating-point groups; SPECweb clients are distributed, with a 24 GB dataset.Source: 1) Traditional benchmarks setups · pp. 4–5

2) CloudSuite Setups:

• CloudSuite’s Naive Bayes workload uses a 147 GB input, while the five other benchmarks follow the suite’s published setup guidance.The remaining benchmarks cover data serving, media streaming, software testing, web search, and web serving.Source: 2) CloudSuite Setups · p. 5

3) Data Analysis Workload Setups:

• Data-analysis inputs range from 147 to 187 GB, with data stored in memory and on disk; workloads retire thousands to tens of thousands of billions of instructions.The authors contrast this with CloudSuite’s 4.5 GB Naive Bayes input, which was entirely memory-resident.Source: 3) Data Analysis Workload Setups · p. 5

D. Experimental Methodology

• Because out-of-order execution overlaps pipeline work, the study focuses on cycles stalled by resource conflicts, such as a full reorder buffer.The retirement-centric view also cannot fully account for CPU-cycle use when instructions continue executing during blocked retirement.Source: D. Experimental Methodology · p. 5
• Figure 3 reports instructions per cycle for each workload.Source: Figure 3 · p. 5
• Hardware performance counters measure about 20 architectural events, while proc-file-system data provide operating-system measures such as disk writes.Perf manipulates the processor’s event-select and monitoring-counter registers.Source: D. Experimental Methodology · p. 5
• Measurements begin after ramp-up and cover each application’s full lifetime, including map, shuffle, and reduce stages; results average the four worker nodes.This differs from CloudSuite’s 180-second measurement period.Source: D. Experimental Methodology · p. 5

A. Instructions Execution

• Data-analysis workloads have higher IPC than services but lower IPC than computation-intensive HPCC workloads; their IPC ranges from 0.52 to 0.95, averaging 0.78.HPC-HPL and HPC-DGEMM are close to 1.2, while service workloads are below 0.6.Source: A. Instructions Execution · p. 6
• Most data-analysis workloads use fewer kernel-mode instructions than services, which exceed 40%; data-analysis workloads average about 4%.Sort is the exception, with about 24% kernel-mode instructions; equal input and output data sizes lead to frequent I/O, and its compare-only computation processes data quickly.Source: A. Instructions Execution · p. 6, Figure 4 · p. 6
• Sort has the highest disk-write frequency among the data-analysis workloads, alongside more frequent network communication than the others.Its MapReduce stages write substantial output to local disk or transfer data over the network.Source: A. Instructions Execution · p. 6, Figure 5 · p. 6
• HPCC RandomAccess has about 31% kernel-mode instructions and measures the rate of integer random updates to remote memory.The passage says other factors contributing to its kernel-mode share need further investigation.Source: A. Instructions Execution · p. 6

B. Pipeline Behaviors

• Figure 6 normalizes each stall type’s blocked cycles by the sum of blocked cycles across types, which can overlap.For example, RS-full stalls and L1 instruction-cache-miss stalls can occur simultaneously.Source: B. Pipeline Behaviors · p. 7, Figure 6 · p. 7
• Data-analysis workloads average about 37% RS-full stalls and 20% ROB-full stalls, whereas services average 60% RAT stalls and 13% instruction-fetch stalls.The study reports more stalls in the out-of-order part for data-analysis workloads and more in the in-order part for services; it says the underlying causes need further investigation.Source: B. Pipeline Behaviors · p. 7
• HPCC workloads show highly variable stall data because their microbenchmarks and kernel programs target different system aspects.Source: B. Pipeline Behaviors · p. 7
• Both data-analysis and service workloads have notable front-end stalls, potentially associated with long memory latency or large binaries from high-level languages and third-party libraries.The paper also notes that this observation corroborates previous work.Source: B. Pipeline Behaviors · p. 7

C. Front-end Behaviors

• Data-analysis workloads average about 23 L1 instruction-cache misses per thousand instructions, above SPECINT, SPECFP, and HPCC workloads but below most service workloads.Media Streaming has about three times the data-analysis average; Naive Bayes has the fewest instruction-cache misses and instruction-TLB page walks among the data-analysis workloads.Source: Figure 7 · p. 8, C. Front-end Behaviors · p. 8
• Instruction-TLB page walks are more frequent in data-analysis workloads than in SPECINT, SPECFP, and HPCC workloads; Media Streaming and Data Serving exceed them.Page walks add long-latency instruction-fetch stalls; the study recommends attention to instruction-cache and instruction-TLB hit ratios and application code size.Source: Figure 8 · p. 8, C. Front-end Behaviors · p. 8

D. Unified Cache and Data TLB Behaviors

• Figure 9 shows L2 cache misses per thousand instructions: data-analysis workloads average about 11, compared with about 60 for services, and exceed HPCC workloads.The authors interpret the lower miss count relative to services as better locality.Source: D. Unified Cache and Data TLB Behaviors · p. 8, Figure 9 · p. 9
• L3 satisfies 85.5% of data-analysis L2 misses and 94.9% of service L2 misses; these averages exceed HPCC’s.The authors conclude that the LLC caches most data missed from L2 for most data-analysis and service workloads, and suggest properly optimizing LLC capacity for energy efficiency and die area.Source: D. Unified Cache and Data TLB Behaviors · pp. 8–9, Figure 10 · p. 9
• For most data-analysis workloads, completed page walks from data-TLB misses are below most service and SPEC CPU2006 workloads but above most HPCC workloads.Naive Bayes, HPCC-RandomAccess, and HPCC-PTRANS are exceptions to the stated comparisons.Source: Figure 11 · p. 9, D. Unified Cache and Data TLB Behaviors · p. 9

E. Branch Prediction

• Most data-analysis workloads have lower branch-misprediction ratios than service and SPEC CPU workloads; HPCC workloads have very low ratios with simple, regular branch logic.The paper suggests that simple branch patterns in data-analysis workloads may help explain their low misprediction ratios.Source: E. Branch Prediction · p. 9
• The authors say modern processors’ branch predictors are good enough for data-analysis workloads and suggest a simpler predictor may save power and die area.Source: E. Branch Prediction · pp. 9–10

V. THE SUMMARY OF DCBENCH

• DCBench is an openly released datacenter benchmark suite containing both data analysis and service workloads.The authors also provide implementations using different programming models and VM images; programming-model effects are beyond this paper's scope.Source: §V · p. 10
• The authors also released BigDataBench for large-scale system and architecture research and CloudRank for cloud capacity planning and evaluation.Source: §V · p. 10

VI. CONCLUSION AND FUTURE WORK

• The study characterizes eleven representative data analysis workloads on modern superscalar out-of-order systems using hardware performance counters.Source: §VI · p. 10
• Data analysis workloads differ substantially in speedup performance and micro-architectural characteristics; one application cannot represent their variety.Source: §VI · p. 10
• The authors distinguish data analysis workloads from desktop, HPC, traditional server, and scale-out service workloads, and offer architecture and system optimization recommendations.They also report micro-architectural similarities between scale-out service workloads and SPECweb2005, an observation they plan to investigate with more workloads.Source: §VI · p. 10