PaperlayerView exact revision on arXiv

Source-linked AI summary

Input Resolution Matters: Real-Time Object Detection Latency

Qingyang Zhang, Fumio Machida, Laura Carnevali

arXiv:2609.12920v1cs.CVcs.PF

TL;DR

Scalar latency summaries provide limited insight into detection-time variability, while resolution can affect pipeline latency differently across stages. The paper models end-to-end latency by combining stage distributions with resolution-dependent parameters; measurements show improvements for selected Normal and Gamma fits, within a study limited to one detector, one platform, and square inputs.

  • Problem

    Scalar measures such as mean inference time or average FPS provide limited insight into real-time detection latency dynamics, whose variability and predictability remain underexplored.

    Source: §I · p. 1

  • Method

    The model combines preprocessing, inference, and postprocessing distributions under an independence approximation and links selected distribution parameters to input resolution.

    Source: §I · p. 1

  • Results

    In measured YOLOv11 experiments, resolution-aware parameters improve Normal preprocessing fits and Normal/Gamma end-to-end fits versus endpoint baselines, with all three CDF-based metrics reduced.

    Source: §VIII · p. 8

  • Takeaways & Limitations

    The analytically tractable, inspectable model connects empirical latency measurements with pipeline structure and exposes how resolution sensitivity enters the end-to-end distribution.

    Source: §VI · p. 7

    The study covers one detector, one embedded platform, and square input resolutions; extensions to non-square inputs and other architectures remain future work.

    Source: §VI · p. 7

Abstract

from arXiv · show

We model total latency as the convolution of preprocessing, inference, and postprocessing distributions under a simplifying independence approximation, with selected stage parameters expressed as functions of source-image resolution. Under this assumption, the probability density of the total latency is the convolution of the stage-wise densities, and its cumulative distribution function (CDF) provides the distribution of end-to-end detection time. Each stage is modeled by a parametric distribution (e.g., Exponential, Erlang, Normal, Gamma), with parameters expressed as functions of the source-image resolution. Experiments with YOLOv11n on NVIDIA Jetson Orin NX using COCO2017 images across multiple resolutions assess the proposed models against fixed-parameter baselines using Kolmogorov Smirnov, Anderson Darling, and Cram\'er von Mises statistics. The results indicate that resolution-aware parameterization can improve distributional approximation in the measured setting, particularly for the more flexible Normal and Gamma models, while the quality of fit remains distribution dependent. Our contribution is a theoretically grounded and lightweight formulation for studying resolution-dependent latency distributions in a measured object detection pipeline.

I. INTRODUCTION

The paper addresses limited attention to latency variability and predictability by modeling end-to-end detection latency as a resolution-aware combination of stage distributions. Experiments assess this formulation against fixed-parameter baselines on measured YOLOv11n latency.

I. INTRODUCTION

  • Scalar metrics such as mean runtime and average FPS provide limited insight into latency dynamics, while latency variability and predictability remain underexplored.The paper situates this gap in real-time detection applications with strict timing constraints.Source: §I · p. 1
  • Preprocessing latency varies with source-image size, whereas inference and postprocessing are largely independent of that size in the studied YOLOv11n pipeline.This stage difference motivates modeling resolution effects within the pipeline rather than treating total latency as a black box.Source: §I · p. 1
  • The model combines preprocessing, inference, and postprocessing latency distributions under an independence approximation, with selected parameters linked to input resolution.The proposed formulation uses stage-wise distributions to represent total latency.Source: §I · p. 1
  • Experiments use YOLOv11n on an NVIDIA Jetson Orin NX with COCO2017 images at multiple resolutions to compare the formulation with fixed-parameter baselines.The assessment uses CDF-based goodness-of-fit statistics and considers improvements as well as remaining limitations.Source: §I · p. 1

II. BACKGROUND

The background motivates distribution-level latency analysis and shows that total-latency distributions vary with input resolution and distribution choice. The next modeling section decomposes latency into stages and introduces resolution-dependent parameters.

II. BACKGROUND

  • At 320 × 320 and 1024 × 1024, empirical total-latency CDFs and PDFs differ in completion-time quantiles and overall distribution shape.Total-latency observations alone do not identify which pipeline stage accounts for the differences.Source: Figure 1 · p. 2, §II · p. 2
  • At 512×512 and 960 × 960, the Normal CDF follows empirical total-latency CDFs more closely than the Exponential CDF, which is a simpler but less accurate reference.The comparisons use directly fitted distributions at intermediate resolutions.Source: Figure 2 · p. 2, §II · p. 2

III. RESOLUTION-DEPENDENT LATENCY DISTRIBUTION

  • The proposed approach decomposes total latency into stage-wise components and introduces resolution-dependent parameters where appropriate.The formulation is presented as a parametric latency distribution whose input-resolution performance characteristics can be represented in the model.Source: §II · p. 2, §III · p. 2

A. Object detection latency

The section defines end-to-end detection latency through preprocessing, inference, and postprocessing stage times. Under a simplifying independence assumption, their distributions are combined to describe total-latency density and completion-time probability.

A. Object detection latency

  • The total-latency distribution function gives the probability that detection finishes within t milliseconds for input resolution r.T denotes end-to-end latency for an object-detection pipeline at resolution r.Source: A. Object detection latency · p. 2
  • Total latency is decomposed into preprocessing, inference, and postprocessing random variables, whose stage-wise latencies are assumed mutually independent for the convolution model.The paper states that this simplifying assumption is evaluated using pairwise correlations and distribution-level impact metrics.Source: A. Object detection latency · p. 2
  • Under the independence assumption, the three stage densities are convolved to obtain the total-latency density, and integration gives its CDF.This formulation combines stage effects while retaining stage-wise interpretability.Source: A. Object detection latency · pp. 2–3

B. Resolution-Dependent Parameters

The model represents stage-wise latency distributions with parameters that may vary with input resolution, then combines the stages by convolution under an independence assumption. It considers four distribution families, with parameters defined as functions of resolution where appropriate.

B. Resolution-Dependent Parameters

  • The model uses stage-specific parameter sets θ(r), which reduce to constants when empirical evidence shows no resolution dependence.For example, λ(r) or β(r) may be modeled as linear functions of resolution; for Normal stages, the summed mean and variance are the sums of the stage parameters.Source: B. Resolution-Dependent Parameters · p. 3, C. Candidate Distribution Families · p. 3
  • The total-latency model combines stage-wise distributions by convolution, assuming independent stage latencies and allowing distribution parameters to depend on resolution or remain constant.The framework is presented as a tractable approximation for measured pipeline behavior.Source: B. Resolution-Dependent Parameters · p. 3

C. Candidate Distribution Families

  • The candidate families are Exponential, Erlang, Normal, and Gamma; Exponential is a unit-shape Gamma special case, while Erlang restricts Gamma shape to positive integers.Exponential, Erlang, and Gamma have positive support; the untruncated Normal is defined on R although measured latency samples are non-negative.Source: C. Candidate Distribution Families · p. 3

IV. EXPERIMENT DESIGN

The experiment evaluates whether stage latencies vary with resolution, whether selected distribution families fit those latencies, and whether linear parameter mappings generalize to unseen configurations. Measurements use YOLOv11n on resized COCO2017 images and the framework’s internal profiler.

IV. EXPERIMENT DESIGN

  • The experiment tests stage-wise resolution dependence, the adequacy of candidate distribution families, and whether linear resolution-to-parameter mappings generalize to unseen configurations.The paper states that the formulation requires empirical evaluation beyond theoretical analysis.Source: §IV · p. 3

A. Hardware and Software Configuration

  • Measurements use 560 COCO2017 validation images at seven square resolutions from 320 × 320 to 1024 × 1024 on Jetson Orin NX with YOLOv11n.The native model tensor remains 640 × 640; 320 and 1024 are parameter-estimation anchors, while 416, 512, 704, and 960 are intermediate evaluation resolutions, and 640 is reported separately.Source: A. Hardware and Software Configuration · pp. 3–4

B. Latency Measurement Methodology

  • Stage-wise times come from YOLO’s internal profiler for preprocessing, inference, and postprocessing; external image loading and resizing are excluded.The profiler records times through the YOLO API’s results.speed attribute, and measurements are collected per image for statistical fitting and distribution analysis.Source: B. Latency Measurement Methodology · p. 4

C. Distribution Fitting Methodology

The fitting procedure models stage-wise latency with standard parametric distributions, estimates parameters at each resolution, and uses a two-anchor linear regression to approximate intermediate resolutions. The proposed resolution-dependent model is compared with three fixed-parameter baselines.

C. Distribution Fitting Methodology

  • Stage-wise latencies are modeled with Exponential, Erlang, Normal, or Gamma distributions, with parameters fitted by maximum likelihood to data collected at each resolution.The fitting process uses scipy.stats.Source: C. Distribution Fitting Methodology · p. 4
  • Parameters fitted at two anchor resolutions are linearly regressed to approximate distribution parameters at intermediate resolutions.The intermediate-resolution approximations are then used for empirical assessment.Source: C. Distribution Fitting Methodology · p. 4

D. Baseline vs. Proposed model

  • Three fixed baselines reuse parameters from resolution 320, resolution 1024, or pooled observations from both anchors while discarding resolution labels.The pooled baseline uses anchor observations only; the four intermediate resolutions are held out from its parameter estimation.Source: D. Baseline vs. Proposed model · p. 4
  • The proposed hybrid assigns resolution-dependent parameters to selected stages and fixed pooled-anchor parameters to the remaining stages, with stage choices varying by distribution family.Exponential and Erlang vary preprocessing only; Normal and Gamma vary preprocessing and inference, while postprocessing stays fixed for all four families.Source: D. Baseline vs. Proposed model · p. 4

E. Evaluation Metrics

The section evaluates fit with complementary CDF-based statistics and documents resolution-dependent stage behavior, while treating independence only as a tractable approximation.

E. Evaluation Metrics

  • KS measures maximum CDF deviation, AD emphasizes tails, and CvM measures integrated squared discrepancy across the full distribution.Lower values indicate better fit; AD and CvM range from 0 to infinity, while KS ranges from 0 to 1.Source: E. Evaluation Metrics · pp. 4–5
  • PDF-based metrics such as KL divergence are excluded from the core evaluation because they are sensitive to histogram binning and unstable with small samples.Source: E. Evaluation Metrics · p. 5

A. Latency Trends Across Processing Stages

  • Inference dominates absolute latency at every resolution, with mean latency nearly constant at about 20.3–20.8 ms because YOLOv11n processes fixed-size internal tensors.Preprocessing generally grows with input resolution, except at native 640 × 640, where the internal spatial transformation is reduced but preprocessing is not skipped.Source: A. Latency Trends Across Processing… · p. 5, Table I · p. 5
  • Preprocessing mean latency correlates strongly with source-image resolution (r = 0.9981, R2 = 0.9963), while inference mean latency shows no significant dependence.Preprocessing STD also correlates with resolution (r = 0.9275); inference STD grows (r = 0.9783), but postprocessing shows no significant trend.Source: A. Latency Trends Across Processing… · p. 5, Table II · p. 5
  • Ignoring cross-stage covariance underestimates total-latency variance by 10.2%–35.2%, with KS distance from independently recombined totals ranging from 0.039 to 0.137.Strict independence is rejected by Spearman and permutation checks, so the model retains it as a tractable approximation with quantified error.Source: A. Latency Trends Across Processing… · p. 5, Table III · p. 5

B. Fitted Parametric Functions

The fitted functions link selected latency-distribution parameters to image width, using linear approximations estimated from anchor-resolution data and evaluated at intermediate resolutions.

B. Fitted Parametric Functions

  • Linear parameter functions approximate resolution trends based on latency data from the lowest and highest measured input sizes, rather than imposing a universal assumption.Stage-level fits across measured resolutions showed trends for several preprocessing parameters and selected inference-variance parameters.Source: B. Fitted Parametric Functions · p. 5
  • The functions cover Exponential, Erlang, Normal, and Gamma distributions, with Erlang shape fixed at k = 10.For Exponential and Erlang preprocessing, λ decreases with resolution; Normal preprocessing µ and σ increase, while Gamma preprocessing varies α and β.Source: B. Fitted Parametric Functions · p. 6
  • Resolution-dependent functions are estimated from stage-level fits at two anchor resolutions, while fixed Model 3 components are bootstrapped from pooled observations at those anchors.The resulting distributions are evaluated against empirical latency distributions at intermediate resolutions.Source: B. Fitted Parametric Functions · p. 6

C. Goodness-of-Fit Evaluation

Resolution-aware fitting benefits preprocessing most clearly, especially with Normal and Gamma distributions; inference-stage gains are smaller and inconsistent.

1) Stage-Level Goodness-of-Fit:

  • The proposed Exponential preprocessing function is comparable to fixed baselines but not uniformly better, while Model 2 has lower KS values across the summarized range.Erlang improves over Model 1, but Model 2 and Model 3 remain competitive under 704 × 704.Source: 1) Stage-Level Goodness-of-Fit · p. 6
  • Normal preprocessing has substantially lower KS values under the proposed model than fixed endpoint baselines at representative intermediate and high resolutions.Gamma preprocessing is also competitive, although two-anchor Model 3 is better at 512×512, showing pooled sampling can sometimes approximate intermediate settings well.Source: 1) Stage-Level Goodness-of-Fit · p. 6
  • Inference-stage gains are smaller and less consistent: Normal at 512×512 favors Model 1, while Gamma at 960 × 960 moderately improves over Model 1 and remains close to Model 2.The parameter trends mainly affect dispersion or distribution shape rather than the nearly resolution-invariant mean latency.Source: 1) Stage-Level Goodness-of-Fit · p. 6

2) End-to-End Total Latency Goodness-of-Fit:

Normal and Gamma provide better end-to-end fit across all three mean statistics than the fixed baselines, while Exponential and Erlang do not show the same advantage. The benefit depends on distribution flexibility and where resolution affects the pipeline.

2) End-to-End Total Latency Goodness-of-Fit:

  • The proposed construction varies preprocessing parameters for Exponential and Erlang, and preprocessing plus inference parameters for Normal and Gamma, while keeping postprocessing fixed.The end-to-end distribution is formed by convolving the stage-wise distributions.Source: 2) End-to-End Total Latency… · p. 6
  • Normal and Gamma improve all three mean statistics over Model 1, Model 2, and two-anchor Model 3.The means are computed over the four intermediate resolutions.Source: Table V · p. 7, 2) End-to-End Total Latency… · p. 7
  • Under Exponential and Erlang, Model 2 and Model 3 have slightly lower mean statistics than the proposed model.These families have limited parameterization and cannot fully represent the measured total-latency shape.Source: 2) End-to-End Total Latency… · p. 7

VI. DISCUSSION

  • Explicit stage-wise parameter functions make the model inspectable, linking measured latency to resolution sensitivity and the end-to-end distribution.The model is described as analytically tractable and connects empirical latency measurements with interpretable system structure.Source: §VI · p. 7
  • The study covers one detector, one embedded platform, and square inputs; other architectures may require different stage-wise dependency assumptions.Non-square inputs may require parameterizing resolution by total pixels or by separate width and height terms.Source: §VI · p. 7

VII. RELATED WORK

Prior benchmarking and latency-modeling work generally does not model full resolution-dependent, stage-wise object-detection latency distributions. The paper reports improvements for selected distribution families and identifies broader extensions for future work.

VII. RELATED WORK

  • Prior detection benchmarks typically report scalar latency summaries, while profiling, queueing, and simulation approaches typically do not model resolution-dependent, stage-wise latency distributions.The paper positions its approach as complementary to those latency-modeling methods.Source: §VII · pp. 7–8

VIII. CONCLUSION AND FUTURE WORK

  • Normal preprocessing and Normal/Gamma end-to-end fits reduce all three CDF-based metrics against endpoint baselines, while Exponential and Erlang show no consistent end-to-end advantage.The end-to-end improvement also holds against the two-anchor Model 3 for Normal and Gamma.Source: §VIII · p. 8
  • Planned extensions include other architectures and tasks, additional system-level variables, and tail-sensitive modeling for latency spikes.The cited variables include batch size and GPU utilization.Source: §VIII · p. 8
Loading 2609.12920v1…