Pricing Time, Not Just Tokens: Latency-Aware Mechanism Design for LLM Inference

1 Introduction

• Token-only models treat inference as a volume-priced commodity, but the timing of delivery affects neither buyer utility nor seller cost in that framing.The paper instead treats delivery time as an economically relevant service dimension.Source: §1 · p. 1

1.1 Primary Contributions

• GPU inference costs reflect compute-bound prefill and bandwidth-bound decode, producing increasingly steep costs as service approaches hardware latency limits.Faster service sacrifices batching efficiency, with the tradeoff becoming more severe near capacity constraints.Source: §1.1 · p. 2
• Discrete hardware tiers can sort buyers by time preference, reducing three-dimensional screening to willingness-to-pay screening within each tier.High-γ buyers prefer premium tiers and low-γ buyers prefer economy tiers when the quality gain from upgrading rises with time preference.Source: §1.1 · pp. 2–3
• The optimal mechanism uses virtual values to set quality within tiers and yields per-task prices independent of buyer volume.The implied absence of quantity discounts provides theoretical grounding for flat per-token API pricing.Source: §1.1 · p. 3
• When hardware costs are significant relative to per-request value, tiered pricing yields 26–66% higher profit than single-tier pricing.At higher valuations, a uniformly faster generational upgrade can make single-tier pricing sufficient.Source: §1.1 · p. 3

1.2 Related Work

• The paper positions its contribution against throughput-focused AI economics, nonlinear-pricing screening, GPU systems, and hard-SLO work by centering latency preferences in inference pricing.It connects inference pricing to service-industry screening and treats latency as part of the objective rather than only a hard constraint.Source: §1.2 · pp. 3–4

2.1 Environment

• A monopolist offers contracts to buyers who privately know their willingness-to-pay, task volume, and time preference, then choose from the posted menu.The model assumes common knowledge of production technology and quasilinear preferences.Source: §2.1 · p. 4

2.2 Buyer Preferences

• Buyer value depends on token inputs and delivery speed, with time preference governing the value of latency slack.The utility specification has diminishing returns to the composite input bundle; high-γ buyers lose utility more rapidly as slack shrinks.Source: §2.2 · p. 4

2.3 Production Technology: Latency

• Latency combines compute-bound prefill and bandwidth-bound decode; batching amortizes setup costs but increases decode congestion, and hardware parameters are estimated from measured GPU latencies.All tasks in a batch share a common delay.Source: §2.3 · pp. 4–5

2.4 Type Space and Distributional Assumptions

• The model assumes independent type components and a regular willingness-to-pay distribution, with binary time preference in the baseline and continuous time preference considered later.Theorem 3.1 establishes self-selection by time preference, reducing screening to willingness-to-pay within each tier.Source: §2.4 · p. 5

2.5 Seller’s Cost Structure

• After time-preference separation, per-task cost is strictly convex in quality, and total cost scales with task volume at a cost determined by per-task quality.Marginal cost depends on per-task quality rather than scale directly, with implications for quantity discounts.Source: §2.5 · p. 6

3.1 Latency Function Estimation

• GPU latency fits separate compute-bound prefill and bandwidth-bound decode phases, with R2 > 0.99 across batch sizes 1–64 and input contexts 1K–8K tokens.The latency function is estimated from measured GPU data using nonlinear regression.Source: §3.1 · p. 6

3.2.2 The Separation Condition.

• High-γ buyers strictly prefer premium hardware and low-γ buyers economy hardware under the separation condition, reducing three-dimensional screening to screening on willingness-to-pay within each tier.The condition requires the quality gain from upgrading to rise with time preference.Source: §3.2.2 · p. 7, §3.2 · p. 7

3.2.1 Hardware Tiers and Latency Slack.

• The separation condition combines high-γ buyers’ greater valuation of slack with premium hardware’s larger advantage at small batch sizes; it holds in 83% of tested configurations and 96% at economically relevant WTP scales.The premium tier costs more but achieves lower latency and greater slack.Source: §3.2.2 · pp. 7–8, §3.2.1 · p. 7

3.2.3 Cost Function Convexity.

• Under the stated latency and production-function conditions, per-task cost is strictly convex in quality, with marginal cost vanishing near zero quality and diverging near maximum feasible quality.The cost-minimizing allocation is unique for each feasible quality level.Source: §3.2.3 · p. 8

3.2.4 Reduction to One-Dimensional Screening.

• After tier choice separates time preferences, each tier becomes a standard one-dimensional screening problem on willingness-to-pay, with task volume entering through the cost structure.The resulting framework follows Mussa and Rosen [1978].Source: §3.2.4 · pp. 8–9

3.2.5 Virtual Values and Optimal Quality.

• Within each tier, the optimal quality schedule equates virtual value with marginal per-task cost for buyers above the exclusion threshold.Under a monotone hazard rate, virtual value increases with willingness-to-pay.Source: §3.2.5 · p. 9

3.2.6 Optimal Transfers.

• Transfers are pinned down by the binding incentive constraint at the margin and the participation constraint at the exclusion threshold.Source: §3.2.6 · p. 9

3.2.8 The Optimal Menu.

• No buyer benefits from misreporting task volume; the complete menu assigns tiers by γ, sets quality where virtual value equals marginal cost, and excludes buyers below the willingness-to-pay threshold.The menu is characterized under the separation condition, monotone hazard rate, and independence of buyer types.Source: §3.2.7 · p. 10, §3.2.8 · p. 10

4.1 Finite Hardware Tiers and Induced Partitioning

• Finite, ordered hardware tiers partition the continuous time-preference range into contiguous segments, with higher-γ types selecting higher-indexed premium tiers.Types at each boundary are indifferent between adjacent tiers.Source: §4 · p. 11, §4.1 · p. 11

4.2 Within-Tier Cost Aggregation

• Because buyers within a tier can have different time preferences and costs, the seller uses expected cost averaged over each segment; this cost remains strictly convex in quality.Uniform quality-transfer schedules within a tier can create cross-subsidization among its time-preference types.Source: §4.2 · p. 12, §4.4 · p. 14

4.3 Optimal Cutoff Characterization

• At an interior optimum, each boundary type yields equal expected profit under either adjacent tier; a solution exists, and a monotonicity condition supports local optimality.The profit advantage of premium hardware must increase with time preference for that local-optimality condition.Source: §4.3 · p. 13

4.4 Within-Segment Screening

• Within each segment, the seller screens on willingness-to-pay using the averaged cost function, and excludes buyers below the willingness-to-pay threshold.The quality and transfer schedules do not vary with time preference within the segment.Source: §4.4 · pp. 13–14

4.5 The Complete Mechanism

• The complete mechanism jointly selects optimal tier cutoffs and solves a separate Mussa-Rosen screening problem within each tier.The cutoff conditions account for the profit effect of shifting types between tiers.Source: §4.5 · pp. 14–15
• With two tiers, the mechanism yields 26–66% profit gains over the best single-tier alternative in favorable parameter regimes.The section notes that these gains occur when hardware costs are a significant fraction of per-request value.Source: §4.5 · p. 15

5.1 Methodology

• The analysis calibrates H100 and B200 8-GPU clusters, sweeps production, latency-budget, and WTP parameters, and solves separation and within-tier profit conditions numerically.The cluster hourly rates are $18.32 for H100 and $30.32 for B200, a 1.66× price ratio.Source: §5 · p. 15, §5.1 · p. 15

5.2 Quality-Difference Separation

• The separation condition holds in 87 of 105 configurations (82.9%) and in 67 of 70 configurations (95.7%) at WTP ranges of $50–100 or higher.Fifteen failures occur at WTP $5–10, where cluster costs leave insufficient surplus for meaningful quality differentiation.Source: §5.2 · p. 15
• B200’s decode advantage is most pronounced at lower batch sizes, supporting an increasing quality gap with time preference and incentive-compatible tier selection.The reported decode ratio is d_B200/d_H100 = 0.68.Source: §5.2 · p. 16

5.3 Mechanism Profit Comparison

• Two-tier pricing raises profit by 26.1–65.8% as the latency budget relaxes beyond 30 seconds, but single-tier H100 pricing is optimal at 10 seconds.At 10 seconds, H100’s lower hourly rate outweighs B200’s decode advantage; the optimal cutoff is approximately γ*=0.25 in the reported comparison.Source: §5.3 · p. 16
• Profit gains depend on the cost-to-WTP regime: higher WTP often makes B200 dominant, while gains reach 29–64% at low WTP when single-GPU costs are reduced eightfold.At higher quality weights, B200 dominates all buyer types and single-tier pricing is optimal; at low WTP, cluster-level costs can also prevent separation.Source: §5.3 · p. 17
• With independently varied tier prices, separation holds in 135 of 185 checks (73.0%), and two-tier profit gains are positive at latency budgets of at least 30 seconds.The maximum gains across the 38 price configurations range from 24% to 67%, depending on the price pair.Source: §5.3 · p. 17

6 Conclusion

• Hardware tiers induce self-selection by time preference, reducing three-dimensional screening to one-dimensional willingness-to-pay screening within each tier.The separation condition holds in 83% of tested configurations and 96% at economically relevant WTP scales; two-tier profit gains are 26–66% when hardware costs are a significant share of per-request value.Source: §6 · pp. 17–18
• The empirical calibration covers one hardware pair, H100 and B200, with a modest decode-latency differential of 0.68.The paper notes that tiering gains are largest when hardware costs are significant relative to per-request value and vanish when the premium tier uniformly dominates.Source: §6 · p. 18

A.1 Computational Phases of LLM Inference

• The latency model separates compute-bound prefill from bandwidth-bound decode, with prefill depending on input length and decode time scaling with output tokens and batch-related terms.Combined latency is modeled as prefill latency plus output-token count multiplied by decode cost.Source: §A.1 · pp. 19–20

A.2 Simulation Methodology

• The simulation methodology runs Llama3-70B on H100 economy and B200 premium tiers and varies batch sizes and input-context lengths.The listed configurations include batch sizes from 1 to 64 and contexts from 1,024 to 8,192 tokens.Source: §A.2 · p. 20

A.3 Regression Specification

• Latency parameters are estimated by nonlinear least squares over a grid of simulated configurations.The passage identifies the simulation grid as D.Source: §A.3 · p. 20

A.4 Estimated Parameters

• Both fitted latency functions achieve R^2 > 0.99 across the operational range.The paper interprets this fit as capturing the essential structure of inference latency.Source: §A.4 · p. 21

Proof

• For a fixed positive willingness-to-pay, separation requires prices that make high-γ types prefer the high tier and low-γ types prefer the low tier.The proof establishes the relevant incentive constraints for each type’s tier choice.Source: B Proof of Theorem 3.1 · p. 21
• Combining the incentive constraints yields an interval of admissible price gaps, and that interval is non-empty precisely when condition (9) holds.The condition does not depend on willingness-to-pay.Source: B Proof of Theorem 3.1 · p. 21
• Any price gap in that interval separates types, while the optimal mechanism’s within-tier Mussa–Rosen schedules supply the willingness-to-pay-dependent transfers.The transfer schedules are identified as those of Theorem 3.7.Source: B Proof of Theorem 3.1 · p. 21

C Scale-Up Domain Architecture as Tier Differentiation

• Scale-up tiers can differentiate identical GPUs by interconnect: high-bandwidth links support tensor parallelism that reduces single-request latency, but require substantial additional capital investment.The passage contrasts PCIe at approximately 32 GB/s per link with NVLink 4.0 at approximately 900 GB/s bidirectional per link and cites an approximately 5× time-to-first-token speedup for an 8-GPU configuration.Source: C Scale-Up Domain Architecture as Tier… · p. 21
• Buyers with higher time preference value scale-up-enabled tiers disproportionately, making infrastructure topology another potential basis for tier separation.The paper frames this as a cost-latency tradeoff complementary to GPU-generation differentiation.Source: C Scale-Up Domain Architecture as Tier… · p. 22

D Derivation of Cost Function Derivatives

• The derivative appendix derives the first and second derivatives of the per-task cost function used in the proof of Theorem 3.3.Its setup defines latency slack as L_max − L and uses the production constraint x^α y^β μ^γ = q.Source: D Derivation of Cost Function… · p. 22, §D.1 · p. 22

D.1 Setup and Notation

• Required slack for quality q determines batch size from the latency constraint, which then enters per-task cost.The setup links latency through L = BD + F and μ = L_max − L.Source: §D.1 · p. 22

D.2 First Derivative (Marginal Cost)

• The appendix applies the envelope theorem to the minimized cost and establishes that its marginal cost with respect to quality is positive on the feasible region.The proof uses positive GPU cost, latency slack, and other stated feasible-region terms.Source: §D.2 · pp. 22–23

D.3 Second Derivative (Convexity)

• The cost function is strictly convex in quality on the feasible region, as the displayed second-derivative proof establishes positivity there.The proof lists positive factors including GPU cost, latency slack, γ, and quality.Source: §D.3 · p. 24

E Proof of Proposition 3.5

• For willingness-to-pay screening, the proof expresses equilibrium utility as wq*(w) − T*(w) and integrates from the zero-utility exclusion threshold.The preceding proof passage invokes the envelope theorem for adjacent-type incentive compatibility.Source: E Proof of Proposition 3.5 · pp. 24–25

F Proof of Proposition 3.6

• For task-volume screening, downward reports yield lower utility, while the stated convexity argument bounds the payoff from upward reports by truthful utility.The upward-deviation argument adjusts the reported willingness-to-pay to w s / s̃.Source: F Proof of Proposition 3.6 · p. 25

• A type who prefers the higher tier implies that all higher-γ types prefer it too, because the quality gain is strictly increasing in γ.Source: G Proof of Proposition 4.2 · p. 25
• For fixed increasing tier prices, the strictly increasing indifference function yields a unique cutoff where a buyer is indifferent between adjacent tiers.Types below the cutoff prefer the lower tier; types above it prefer the higher tier.Source: G Proof of Proposition 4.2 · p. 25
• Applying the adjacent-tier cutoffs iteratively produces an ordered partition across tiers.Source: G Proof of Proposition 4.2 · p. 25

• Averaging the type-specific cost functions over a tier preserves strict convexity.The proof uses that each type-specific cost is strictly convex and that integration preserves the strict inequality.Source: H Proof of Lemma 4.3 · p. 26
• The averaged marginal cost is zero at zero quality and diverges as quality approaches the minimum feasibility bound across types.Near that upper bound, at least some types' marginal costs diverge.Source: H Proof of Lemma 4.3 · p. 26
• The averaged cost is continuous in the tier cutoffs because its numerator and denominator are continuous for valid partitions.Source: H Proof of Lemma 4.3 · p. 26

• Changing a cutoff has a boundary effect and a cost-averaging effect, with the envelope theorem removing the within-tier mechanism's response at the optimum.Source: I Proof of Theorem 4.4: Necessary… · p. 26
• Combining the two effects cancels the averaged-cost terms and yields the cutoff condition; the adjacent segment has an analogous calculation.Source: I Proof of Theorem 4.4: Necessary… · p. 27

J Proof of Theorem 4.4: Existence and Sufficiency

• A continuous map of best-response cutoffs has a fixed point by Brouwer's theorem.The map uses roots of strictly increasing profit-difference functions and continuous within-tier mechanisms.Source: J Proof of Theorem 4.4: Existence and… · p. 27
• Non-degenerate fixed points with all tiers active have ordered cutoffs and satisfy the indifference condition at each boundary.A fixed point may instead have coincident cutoffs, leaving an intermediate tier inactive and reducing the partition to fewer active tiers.Source: J Proof of Theorem 4.4: Existence and… · pp. 27–28

K AI Use Disclosure

• The authors used Gemini 3 Pro and Claude Opus 4.6 as adversarial critics and for proof-verification assistance and stylistic refinement.They state that final text, derivations, and citations were verified and authored by the authors.Source: K AI Use Disclosure · p. 28