PaperlayerView exact revision on arXiv

Source-linked AI summary

GreenPipe: Power Modeling for Containerized DNN Inference on Kubernetes Edge Nodes

Mengxue Wang, Peini Liu, Amir Taherkordi, Jordi Guitart

arXiv:2609.04952v1cs.DCcs.PF

TL;DR

GreenPipe addresses the challenge of estimating power for containerized DNN inference on ARM edge nodes without hardware power counters, where CPU-only models miss multi-resource behavior. It builds and deploys multi-resource regression models from external-meter measurements, then evaluates them across inference models, precisions, thread counts, and deployment modes. On a Raspberry Pi 4, system-level validation MAPE is 6.3%–9.4%, while the study also reports latency–energy trade-offs and uses heuristic container attribution.

  • Problem

    ARM edge nodes often lack hardware power interfaces, while CPU-centric models can miss cache, memory, disk, and network behavior in containerized DNN inference.

  • Method

    GreenPipe collects multi-resource metrics with external meter labels, trains and validates regression models, deploys the selected model online, and heuristically attributes predicted node power to containers.

  • Results

    6.3%–9.4% system-level MAPE is achieved on Raspberry Pi 4 validation inference workloads, with 26.9% average MAPE improvement over CPU-stress and utilization-only baselines.

  • Takeaways & Limitations

    GreenPipe provides per-node power estimation across local and network-served inference while exposing latency–energy trade-offs across workload configurations.

  • Takeaways & Limitations

    Container-level power attribution is heuristic because the external meter provides only node-level ground truth, so per-container power accuracy is not validated.

Abstract

from arXiv · show

Distributed DNN inference is increasingly deployed in containerized edge-cloud environments, where workloads run on-device or are exposed to remote clients over the network. Accurate online power estimation on resource-constrained ARM nodes without hardware power counters such as RAPL remains a challenge, and CPU-only models fail to capture multi-resource behavior. We present GreenPipe, an automated profiling-training-validation pipeline that builds multi-resource regression models from external power meter measurements and attributes power to containers proportionally. GreenPipe is evaluated on a Raspberry Pi 4 edge node in a K3s edge-cloud testbed, covering DNN inference with three vision models, multiple precisions, thread counts, and both local and serving scenarios. System-level MAPE is 6.3-9.4%, improving over CPU-stress and utilization-only baselines by 26.9% MAPE on average. We jointly report inference latency and energy per inference, exposing performance-energy trade-offs across workload configurations.

1 Introduction

GreenPipe addresses the need for multi-resource, online power estimation for containerized DNN inference on ARM edge nodes, including workload-level attribution. It evaluates this pipeline across deployment modes and workload configurations while reporting both prediction accuracy and latency–energy behavior.

  • GreenPipe builds and deploys per-node power models from multi-resource metrics and external meter measurements for Kubernetes ARM edge nodes.
  • 26.9% average MAPE reduction over CPU-stress and utilization-only baselines demonstrates improved prediction accuracy.Validation node-level MAPE is 6.3%–9.4%.
  • The evaluation covers three image-classification models, multiple precisions and thread counts, and local versus network-served inference.
  • GreenPipe supports runtime node-level estimation with heuristic attribution of predicted power to containers.
  • Latency–energy characterization exposes trade-offs across workload configurations and deployment modes.

2 Related Work

Prior software power models and container-aware tools provide useful foundations but leave gaps for ARM edge nodes running containerized DNN inference. GreenPipe combines external-meter labeling, DNN-oriented multi-resource training, Kubernetes deployment, and heuristic container attribution to address these gaps.

  • CPU utilization, performance counters, frequency features, and regression methods have been used to estimate full-system power without direct hardware measurements.Raspberry Pi studies include utilization-based and utilization-plus-frequency models.
  • Container-aware tools can expose workload-level energy, but Kepler depends on RAPL, ACPI, or hwmon interfaces often unavailable on Raspberry Pis.
  • GreenPipe differs by combining external-meter labeling, DNN-oriented multi-resource training, Kubernetes online deployment, and heuristic container attribution for ARM edge nodes.
  • GreenPipe validates deployable per-node models on realistic DNN inference workloads while reporting latency–energy behavior across deployments.

3 System Overview

GreenPipe separates model generation and validation from runtime estimation: benchmarks and measurements are collected on the edge, while preprocessing, training, and validation occur on the server. The selected model is then deployed for per-second node prediction and heuristic container attribution.

  • GreenPipe operates in offline and online phases for model generation, validation, and runtime power estimation.
  • Offline Workflow: Offline profiling launches benchmark Jobs on the edge, exports metrics through a monitor, and stores time-series data in server-side Prometheus.
  • Offline Workflow: The offline workflow preprocesses edge traces, trains and cross-validates regression models on the server, then evaluates candidates on local and serving inference benchmarks.
  • Online operation: After validation, an estimator sidecar loads the selected model and predicts node-level power every second from live monitor metrics.
  • Online operation: Predicted node power is attributed to containers on the edge and exported to Prometheus for analysis and future energy-aware scheduling.
  • GreenPipe targets reproducible per-node power modeling for a given edge hardware configuration, with the server also orchestrating workloads and generating served-inference requests.

4 GreenPipe Methodology

GreenPipe builds deployable per-node power models for Kubernetes ARM edge nodes by combining multi-resource monitoring, external-meter labeling, regression training, and heuristic container attribution. Its workflow uses containerized benchmarks spanning isolated resource loads, combined inference-like behavior, and DNN kernels, then validates selected models for online estimation.

  • Resource and Power Monitoring: GreenPipe customizes ARM-compatible monitoring to collect process-, container-, and node-level CPU, cache, memory, disk, and network metrics alongside 1 Hz external-meter power.The monitor adapts Kepler for Raspberry Pi hardware without RAPL and exports per-second estimates online.
  • Training Benchmarks: Three benchmark groups—micro-benchmarks, combined benchmarks, and DNN inference-targeted benchmarks—generate containerized training data on the edge node.Micro-benchmarks isolate components, combined workloads approximate local or serving inference, and DeepBench supplies matrix-multiply and convolution kernels.
  • Model Training: 13,938 samples from 12 benchmarks are aligned at 1 Hz, normalized, and reduced to seven features: cpu_cycles, cpu_time, cpu_instructions, cache_miss, page_cache_hit, net_bandwidth, and disk_io.Samples with missing metric or power values are removed before model fitting.
  • Container-level Power Model: GreenPipe decomposes node power into idle and dynamic components, then attributes dynamic power across containers in proportion to weighted resource usage.The trained linear-regression weights define the resource weighting, while idle power is distributed according to each container’s process count.
  • Container-level Power Model: Container-level attribution is heuristic because the external meter provides only node-level ground truth, so per-container power accuracy is not validated.The attribution is intended for operational visibility rather than validated container-level measurement.

5 Experimental setup

The experiments use a Raspberry Pi 4 ARM edge node in a two-node K3s environment, with image-classification inference evaluated across model formats, precision settings, and local versus serving scenarios. Validation traces are held out from training, and errors are computed over aligned validation samples.

  • Hardware: The edge node is a Raspberry Pi 4 with a quad-core Cortex-A72 ARM processor, dual-band Wi-Fi, and a Ruideng UM25C USB power meter.The paired server uses an Intel Core i7-8650U system.
  • Platform Settings: The testbed runs Debian 12 on the edge and Ubuntu 22.04 on the server, using K3s, containerd, and cgroup v2.The Raspberry Pi kernel is customized to enable eBPF attachment to specific events.
  • Workloads: Validation covers MobileNetV2, EfficientNetB0, and ResNetV2 image-classification models using float32 saved models and lighter TFLite variants.The inference dataset is sampled from the ILSVRC2012 ImageNet validation set, with MLPerf data used for quantization calibration.
  • Inference Scenarios: The study evaluates local inference on on-device data and serving inference through gRPC requests sent from other nodes.Local inference uses LiteRT for TFLite models.
  • Validation: No validation inference trace is used for training, and reported errors cover all aligned samples in the validation traces.This separates model fitting from validation evaluation.

6 Evaluation

GreenPipe is evaluated across inference architectures, precisions, thread counts, engines, deployment scenarios, and online attribution. It achieves low system-level prediction error while exposing workload-dependent accuracy and latency–energy trade-offs.

  • System-level accuracy: MAPE ranges from 6.3% to 9.4%, while GreenPipe reduces MAPE by 26.9% on average versus CPU-stress training.All GreenPipe models also improve over CPU-utilization-only regression, and multi-resource coverage matters more than regressor choice.
  • Model architecture and precision: MobileNet workloads reach 6.6%–9.2% MAPE, while floating-point inference is predicted more accurately than integer inference.Non-linear models produce smaller errors than linear models, indicating non-linear relationships between selected features and power.
  • Thread counts: Power prediction accuracy decreases as inference thread count increases, although both linear and non-linear models remain reasonable.The trend is reported for workloads evaluated with different numbers of threads.
  • Inference engines and scenarios: Power models predict pb workloads more accurately than tflite workloads, whose actual power is consistently slightly higher than predicted.The paper attributes this difference possibly to LiteRT being more optimized for edge devices than TensorFlow.
  • Workload factors: Thread count and deployment mode dominate prediction difficulty, while precision and model architecture have limited impact on prediction error.The evaluation covers local and network-served inference across multiple workload factors.
  • Latency–energy trade-offs: Smaller models benefit from parallelism in latency and energy, whereas larger models may consume more energy because higher power offsets shorter runtime.Int8 is fastest and least energy-consuming among precisions, while ResNet has the highest latency and energy.
  • Online attribution: Container-level attribution is demonstrated at runtime rather than validated against container-level ground truth because the external meter measures only node-level power.The model accounts for idle, dynamic, and multi-resource power, while exporter and estimator overhead remains below 2% of node power.

7 Conclusion and Future Work

GreenPipe provides per-node power models for containerized DNN inference on ARM edge devices and achieves 6.3%–9.4% system-level MAPE on Raspberry Pi 4 validation workloads. The study also characterizes latency–energy trade-offs and identifies extensions to broader hardware and operating conditions.

  • Conclusion: GreenPipe integrates multi-resource profiling, benchmark-driven training, online estimation, and heuristic container attribution in a Kubernetes cluster.The pipeline is evaluated on a Raspberry Pi 4 edge node.
  • Conclusion: 6.3%–9.4% MAPE is achieved for system-level power prediction on validation inference workloads.The reported evaluation uses a Raspberry Pi 4 edge node.
  • Conclusion: Thread-level parallelism can reduce latency without always reducing energy, motivating joint performance–energy characterization.The conclusion reports this trade-off across workload configurations.
  • Future work: Future work will examine additional edge platforms, GPU-equipped devices, broader DVFS settings, and periodic meter-assisted recalibration.The recalibration direction addresses the absence of synchronous ground-truth power labels during online deployment.
Loading 2609.04952v1…