PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantization-Aware Collaborative Inference for Large Embodied AI Models

Zhonghao Lyu, Ming Xiao, Mikael Skoglund, Merouane Debbah, H. Vincent Poor

arXiv:2602.13052v1cs.LGeess.SP

TL;DR

Resource-limited embodied agents struggle to run LAIMs because of their massive computational and parameter demands, while quantization introduces distortion whose system-level trade-offs remain insufficiently characterized. The paper develops a quantization-aware co-inference framework with distortion bounds and jointly optimizes quantization bit-width and computation frequency under delay and energy constraints. Evaluations validate the analysis and report improved balance among inference quality, latency, and energy consumption.

  • Problem

    LAIM scale and computation challenge resource-limited embodied agents, while the quantization-distortion trade-off and its coupling with computation frequency lack unified tractable characterization under QoS constraints.

  • Method

    The paper approximates quantization-induced inference distortion, derives lower and upper rate-distortion bounds, and jointly optimizes bit-width and computation frequency under delay and energy constraints.

  • Results

    Evaluations validate the distortion approximation and rate-distortion bounds, while simulations and real-world testbed experiments demonstrate balanced inference quality, latency, and energy consumption.

  • Takeaways & Limitations

    Quantization and computation should be designed jointly for practical LAIM co-inference in edge embodied AI systems.

Abstract

from arXiv · show

Large artificial intelligence models (LAIMs) are increasingly regarded as a core intelligence engine for embodied AI applications. However, the massive parameter scale and computational demands of LAIMs pose significant challenges for resource-limited embodied agents. To address this issue, we investigate quantization-aware collaborative inference (co-inference) for embodied AI systems. First, we develop a tractable approximation for quantization-induced inference distortion. Based on this approximation, we derive lower and upper bounds on the quantization rate-inference distortion function, characterizing its dependence on LAIM statistics, including the quantization bit-width. Next, we formulate a joint quantization bit-width and computation frequency design problem under delay and energy constraints, aiming to minimize the distortion upper bound while ensuring tightness through the corresponding lower bound. Extensive evaluations validate the proposed distortion approximation, the derived rate-distortion bounds, and the effectiveness of the proposed joint design. Particularly, simulations and real-world testbed experiments demonstrate the effectiveness of the proposed joint design in balancing inference quality, latency, and energy consumption in edge embodied AI systems.

I. INTRODUCTION

Embodied AI needs LAIM capabilities, but resource limits and network sensitivity make conventional deployment inadequate. The paper therefore develops quantization-aware co-inference and jointly designs quantization and computation under QoS constraints.

  • Research Background and Related Work: Embodied AI requires real-time responsiveness, energy efficiency, and reliable inference, while on-device and on-cloud LAIM deployment each face fundamental limitations.On-device deployment is constrained by computation, memory, and energy; on-cloud deployment depends strongly on network conditions and concurrent inference demands.
  • Research Background and Related Work: Co-inference distributes LAIM processing across agent, edge, and cloud resources, allowing resource-limited agents to access LAIM services.The agent performs lightweight front-end processing and transmits intermediate features for subsequent inference.
  • Research Background and Related Work: Existing co-inference studies largely target regular-scale models, leaving LAIM-oriented resource management and model compression insufficiently addressed.Prior LAIM work includes cloud-edge co-inference and dynamic task offloading, but the on-agent component remains constrained by limited resources.
  • Motivations and Contributions: Quantization introduces inference distortion whose dependence on bit-width lacks a unified tractable characterization, while bit-width and computation frequency are coupled under delay and energy constraints.These gaps motivate a system-level treatment of quantization and computation frequency as joint design variables.
  • Motivations and Contributions: The paper develops a quantization-induced distortion approximation and derives lower and upper rate-distortion bounds that characterize dependence on LAIM statistics and quantization bit-width.The approximation links parameter-level perturbations to output-level distortion.
  • Motivations and Contributions: The proposed framework jointly optimizes quantization bit-width and computation frequency under delay and energy constraints, minimizing the distortion upper bound while maintaining a tight approximation gap.Evaluations across multiple LAIMs, datasets, and quantization schemes report gains in balancing inference quality, latency, and energy consumption.

B. On-server Inference and Result Feedback

The server completes inference from the agent's transmitted embedding and returns the result, while the analysis models parameter statistics and computation-related delay and energy across both stages.

  • On-server Inference and Result Feedback: The server completes the remaining inference from the received embedding and feeds the final output back to the agent.This forms the second stage of the collaborative inference process.
  • LAIM Parameter Distribution Modeling: The parameter model preserves sign bits and assumes exponentially distributed parameter magnitudes with distribution parameter λ.The assumption is empirically fitted across ResNet-152, VideoMAE, BERT, BLIP2, GIT, and GPT-3.
  • Inference Delay and Energy Consumption: The system analysis focuses on computation-related delay and energy because LAIM inference is typically computation-dominated.The modeled stages include on-agent and on-server inference.
  • Delay Analysis:: After quantization, the assumed on-agent workload scales linearly with bit-width as ˆbNFLOP/b, linking quantization to inference delay.The delay also depends on processor clock frequency and FLOPs per CPU cycle.
  • Delay Analysis:: On-server delay is determined by the server inference workload, clock frequency, and FLOPs per cycle.The server-stage workload is denoted ˜NFLOP.
  • Energy Consumption Analysis:: On-agent energy consumption is modeled using the agent's power usage effectiveness and chip-dependent power coefficient.The corresponding server energy model uses analogous PUE and power-coefficient parameters.

III. MODEL OUTPUT DISTORTION APPROXIMATION

The paper approximates quantization-induced output distortion by relating parameter perturbations to inference-output changes. For FC DNNs and general AI models, this supports a tractable norm-based surrogate for subsequent analysis.

  • The analysis begins with FC DNNs because exact quantization-to-output distortion relationships are generally intractable for deep nonlinear LAIMs.The FC analysis provides explicit insight into how parameter perturbations propagate through layers.
  • Under Assumptions 1–3, Proposition 3.1 upper-bounds inference-output distortion by a weighted sum of layer-wise parameter distortions.The assumptions cover normalized inputs, smooth activations, and bounded quantization errors.
  • The multiplicative coefficients depend on unquantized parameters and quantization errors, not on the quantized parameters.
  • The paper omits these coefficients and adopts norm-based parameter distortion as a surrogate for inference-output distortion.This surrogate is extended to general AI models, including transformer-based LAIMs.
  • A first-order Taylor approximation with bounded gradient gives ||f(x, W-hat) - f(x, W)||_1 <= H||W - W-hat||_1.This establishes a tractable relationship between parameter perturbation and output distortion for nonlinear, large-scale models.

IV. RATE-DISTORTION ANALYSIS FOR QUANTIZATION

The paper characterizes quantization bit-width versus parameter distortion through rate-distortion theory under an exponential model of LAIM parameter magnitudes. It derives a lower bound and connects bit-width, model statistics, and resource trade-offs.

  • The rate-distortion analysis studies parameter distortion because exact LAIM inference-output distortion is intractable.The analysis uses the surrogate distortion and an assumed exponential weight distribution.
  • R(D) denotes the minimum bits per parameter needed to achieve average quantization distortion no larger than D.
  • Proposition 4.1 derives a Shannon-type lower bound on R(D) for exponentially distributed LAIM parameters under absolute-error distortion.
  • Increasing bit-width generally reduces distortion but increases computation workload, delay, and energy consumption.
  • The distribution parameter λ captures weight concentration near zero: larger λ implies lower bit-width requirements for the same distortion, whereas smaller λ indicates greater distortion sensitivity.

C. An Upper Bound on the Rate-distortion Function for Quantization

The paper derives an upper rate-distortion bound using a valid test channel and Laplacian quantization noise. Together with the lower bound, it forms a computable interval for quantization design.

  • Proposition 4.2 provides an upper bound on R(D) for exponentially distributed LAIM parameters under absolute-error distortion.
  • The upper-bound proof evaluates mutual information using a valid test channel with independent zero-mean Laplacian quantization noise.The noise is chosen to satisfy E[|Z|] = D.
  • The lower and upper bounds define a computable interval DL(R) <= D(R) <= DU(R).DU(R) provides a conservative distortion estimate, while DL(R) provides an optimistic distortion floor.
  • The resulting interval quantifies the quantization-induced distortion trade-off for LAIM co-inference system design.

V. JOINT QUANTIZATION AND COMPUTATION DESIGN FOR LAIM CO-INFERENCE

The paper jointly optimizes quantization bit-width and device/server computation frequencies under delay and energy constraints. Because the resulting problem is non-convex and mixed-integer, it uses relaxation and successive convex approximation.

  • Problem formulation: The design minimizes the distortion upper bound while maintaining tightness through the corresponding lower bound under delay and energy constraints.Decision variables include on-agent bit-width and device/server computation frequencies.
  • Problem formulation: The formulation is a highly non-convex MINLP because bit-width is integer and is coupled with device and server computation frequencies.
  • Proposed solution: The solution relaxes discrete bit-width to a continuous variable and introduces an auxiliary variable to reformulate coupled constraints.
  • Proposed solution: Successive convex approximation replaces non-convex objective and constraint terms with first-order approximations around local points.
  • Proposed solution: Each convex subproblem can be efficiently solved with standard tools, and iterative objective updates are monotonically non-increasing.The relaxed bit-width is rounded to the nearest achievable value after convergence.

VI. EVALUATION

The evaluation validates the proposed joint quantization and computation design for LAIM co-inference.

  • The study evaluates the proposed joint quantization and computation design for LAIM co-inference.

A. Verification of Model Output Distortion Approximation

The evaluation tests the distortion approximation, rate-distortion bounds, and joint design across models, datasets, quantization schemes, and resource constraints. Results show reliable approximation, increasingly tight bounds, and superior joint-design performance.

  • Verification of Model Output Distortion Approximation: The approximation is validated on FCDNN-16, BLIP-2, and GIT under uniform and nonuniform quantization.FCDNN-16 uses MNIST, while BLIP-2 and GIT support co-inference through decoupled architectures.
  • Verification of Model Output Distortion Approximation: Across evaluated models and schemes, the theoretical parameter distortion consistently upper bounds output distortion.The gap narrows with bit-width and becomes notably tight above 3 bits for nonuniform and 4 bits for uniform quantization.
  • Verification of Model Output Distortion Approximation: The simulated distortion-rate function decreases approximately exponentially with rate, matching the scaling predicted by the analytical bounds.The upper bound becomes increasingly tight above 2 bits, while the lower bound captures the distortion-reduction scaling and both converge toward a small distortion floor.
  • Performance of the Proposed Joint Quantization and Computation Design: The proposed design outperforms PPO-based, fixed-frequency, and feasible-random baselines by jointly exploiting quantization and computation resources.The comparison uses the three benchmark schemes defined for the evaluation.
  • Performance of the Proposed Joint Quantization and Computation Design: The joint design consistently achieves the highest CIDEr score for BLIP-2 and GIT across delay, energy, and quantization settings.Figures 5–8 cover MS-COCO and VaTeX under uniform and nonuniform quantization.
  • Performance of the Proposed Joint Quantization and Computation Design: Higher frequency improves quality in delay-limited regimes, whereas lower frequency improves quality when energy is tight.Frequency changes alter feasible bit-widths: higher frequency can permit larger bit-width under delay constraints but can force more aggressive quantization under energy constraints.

VII. CONCLUSIONS

The paper develops quantization-aware LAIM co-inference for embodied AI through distortion analysis and constrained joint design. Simulations and testbed experiments show consistent gains in LAIM co-inference performance.

  • The paper approximates quantization-induced inference distortion through parameter-level perturbations and derives rate-distortion bounds.
  • It formulates joint quantization bit-width and computation-frequency optimization under QoS constraints.The objective minimizes the distortion upper bound while maintaining a tight approximation gap through the lower bound.
  • Extensive simulations and real-world testbed experiments consistently outperform benchmark schemes in LAIM co-inference performance.

APPENDIX

The appendix proves an upper bound on inference output distortion for a quantized FC DNN. The proof repeatedly applies stated assumptions, norm inequalities, substitution, and scaling operations.

  • The proof begins by upper bounding FC DNN inference output distortion under model quantization.
  • The derivation applies the triangle inequality and Assumptions 2 and 3 to obtain successive bounds.
  • The proof recursively bounds the network term using Assumption 2 and the Cauchy-Schwarz inequality.
  • The resulting inequalities are obtained through substitution, division by Q_L, repeated application to the second term, and multiplication by Q_L.
  • The appendix concludes the derivation after these transformations.

B. Proof of Lemma 4.2

The proof solves the entropy-maximization problem by showing the moment constraint is active and applying KKT conditions. Stationarity yields an exponential-form optimizer, whose constants are fixed by the constraints.

  • The entropy-maximization problem is posed over all PDFs PZ(z) supported on R.
  • The moment constraint must be active because scaling any feasible variable with strict slack increases differential entropy while preserving feasibility.
  • Convexity and Slater’s condition justify solving the problem with KKT conditions and Lagrange multipliers for the normalization and moment constraints.
  • The stationarity condition is obtained by perturbing the optimal PDF and requiring the Lagrangian derivative at zero perturbation to vanish.
  • Solving the stationarity equation produces an exponential-form solution, with normalization and moment constraints determining its constants.
  • The derivation concludes after enforcing the constraints to obtain the optimal PDF.
Loading 2602.13052v1…