PaperlayerView exact revision on arXiv

Source-linked AI summary

The Multi-Lab Enterprise: Governance, FinOps, and Telemetry Challenges of Multi-Model AI Adoption

Fabricio F. Costa

arXiv:2609.31622v1cs.CY

TL;DR

The paper examines why enterprises license multiple frontier AI labs and what operational problems this structure creates. It synthesizes survey, transaction, disclosure, and case evidence, reporting that 81% of surveyed Global 2000 enterprises used three or more model families in January 2026.

  • Problem

    The paper asks why enterprises license multiple frontier labs rather than standardizing on one, and what operational problems this multi-lab structure creates.

    Source: §1 · p. 2

  • Method

    The paper synthesizes survey, transaction, provider-disclosure, and case evidence, treating disagreements among sources as informative rather than averaging them away.

    Source: §3 · p. 5

  • Results

    81% of surveyed Global 2000 enterprises used three or more model families in January 2026, up from 68% a year earlier.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper argues that governance, spend, and telemetry problems are facets of a missing cross-provider enterprise AI control plane.

    Source: §9 · pp. 15–16

    The quantitative base is U.S.-centric, survey sampling frames differ across sources and years, and figures are dated to publication month.

    Source: §3 · p. 5

Abstract

from arXiv · show

Enterprises are not choosing a single frontier AI provider; they are licensing all of them. As of early 2026, 81% of Global 2000 enterprises run three or more model families, and OpenAI, Anthropic, and Google Gemini together account for roughly 88 to 89% of enterprise LLM usage and spend. Drawing on survey data, transaction data, provider disclosures, and case studies across finance, legal, consulting, healthcare, life sciences, retail, and government, this paper shows that multi-lab licensing is a structural feature of the market, driven by durable task-specific model differentiation rather than a transitional phase awaiting commoditization. This structure creates three operational problems. Governance fragmentation (P1): heterogeneous vendor security postures, documentation, and compliance surfaces must be reconciled across overlapping regulatory frameworks while shadow AI proliferates. FinOps breakdown (P2): token-based, behavior-driven consumption defeats budgeting. In the past year, 79% of enterprises overran AI budgets, with FinOps-mature organizations overshooting by a mean of 30.9%, and no standardized cross-provider unit of spend exists. Telemetry fragmentation (P3): each lab exposes adoption and cost data through incompatible consoles, APIs, and metric definitions, forcing bespoke unification layers. We map the emerging responses, including LLM gateways, observability platforms, and the Tokenomics Foundation's FOCUS extension. We then develop a five-metric framework for evaluating API and agent cost burn, with a worked example where the cheapest model per attempt is the most expensive per successful task. We conclude that P1, P2, and P3 reflect one missing abstraction: a cross-provider enterprise AI control plane.

1 Introduction

Enterprises license multiple frontier labs because their task-specific capabilities differ, creating governance, FinOps, and telemetry problems across incompatible operational surfaces. The paper frames these problems as facets of a missing cross-provider control plane.

1 Introduction

  • 81% of surveyed Global 2000 enterprises used three or more model families in January 2026, up from 68% a year earlier.Transaction data and the U.S. federal government’s OneGov agreements also show multi-vendor purchasing.Source: §1 · p. 2
  • Enterprises treat frontier labs as complements rather than substitutes because their capabilities diverge across tasks such as coding, long-context processing, and general knowledge work.The paper argues that enterprise switching behavior confirms durable, task-specific differentiation.Source: §1 · p. 2
  • Governance fragmentation multiplies security reviews, model documentation, data-processing agreements, and certification postures across overlapping regulatory frameworks.FinOps breakdown follows from token use scaling with model behavior rather than seats, while incompatible provider pricing defeats traditional financial planning.Source: §1 · p. 2
  • Telemetry fragmentation leaves adoption and spend data split across vendor consoles and APIs with incompatible definitions, formats, and refresh latencies.No single vendor surface answers how much enterprises spend, on what, and to what effect.Source: §1 · p. 2
  • The paper proposes a cross-provider control plane to normalize policy, spend, and usage, addressing P1–P3 through one abstraction.Its contributions also include a structured comparison of provider telemetry surfaces, a five-metric cost-burn framework, and a research and standardization agenda.Source: §1 · pp. 2–3, Figure 1 · p. 3

2 Background and Related Work

Enterprise LLM spending and provider shares shifted rapidly while enterprises generally added labs rather than replacing existing providers. The related work covers routing, governance frameworks, and token-cost standardization, distinguishing these from the organizational problems addressed here.

2.1 Enterprise LLM Market Structure, 2023–2026

  • Anthropic’s enterprise LLM spend share rose from 12% in 2023 to 40% in 2025 while OpenAI’s fell from 50% to 27%; 11% switched providers and 66% upgraded existing vendors.Provider-level stickiness coexisted with portfolio-level expansion: enterprises rarely substituted labs; they added them.Source: §2.1 · p. 3

2.2 The LLM Routing and Orchestration Literature

  • Routing literature optimizes which model answers a query but does not address enterprise reconciliation of vendor security, billing, or adoption measurement.The paper identifies this organizational layer above routing as unaddressed in peer-reviewed literature.Source: §2.2 · p. 4

2.3 AI Governance Frameworks

  • Enterprises face overlapping governance frameworks, including NIST AI RMF, ISO/IEC 42001, the EU AI Act, and sector-specific regimes such as HIPAA and FINRA expectations.The EU AI Act’s obligations phase in between February 2025 and August 2027.Source: §2.3 · p. 4

2.4 FinOps and the Token as a Cost Unit

  • The Tokenomics Foundation was announced to extend FOCUS to token-based consumption, while 98% of FinOps survey respondents cited AI cost management as a concern in 2026, up from 63% a year earlier.Tokens fell outside FOCUS’s original scope.Source: §2.4 · p. 4

3 Data and Methods

The paper synthesizes evidence from sources with distinct reliability properties and treats disagreements as informative. It triangulates market claims while acknowledging conflicts of interest and limits in the quantitative base.

3 Data and Methods

  • The synthesis combines survey data, transaction records, provider disclosures, and case documentation, each with distinct reliability properties.Transaction data covers more than 70,000 U.S. businesses and captures decentralized purchasing; provider disclosures are flagged as company-claimed.Source: §3 · p. 4
  • The authors report ranges and plausible bias when survey, transaction, and disclosure series diverge, rather than selecting a single point estimate.Figure 2 compares four related but non-identical constructs: spend share, production adoption, paid business adoption, and panel adoption.Source: §3 · p. 5, Figure 2 · p. 5
  • The quantitative base is U.S.-centric, survey sampling frames differ across sources and years, and figures are dated to publication month in a fast-moving market.The framework is intended to outlive the paper’s mid-2026 point estimates.Source: §3 · p. 5

4 The Multi-Lab Licensing Phenomenon

Multi-lab adoption is growing, reflecting task-specific model differentiation; evidence across sectors and enterprise designs supports treating it as a durable market structure rather than a transitional phase.

4.1 Prevalence

  • 81% of surveyed Global 2000 enterprises used three or more model families in January 2026, up from 68%; 37% ran five or more models in production, up from 29%.The three leading labs jointly accounted for approximately 88–89% of enterprise LLM spend and wallet share.Source: §4.1 · p. 5, Figure 4 · p. 7

4.2 Why All of Them: Durable Differentiation by Task

  • Task-specific capabilities make frontier labs complements rather than substitutes, with distinct strengths in coding, reasoning, price-performance, context, multimodality, and general knowledge work.Harvey’s legal-task benchmarking found no single best model: seven models beat its original system, with Gemini strongest at drafting and OpenAI’s o3 strongest at pre-trial analysis.Source: §4.2 · p. 6

4.3 Structural, Not Transitional

  • Only 11% of teams changed providers while three-plus-model adoption rose from 68% to 81%; firms also invested in model-agnostic architectures as capability divergence widened.The paper therefore treats multi-lab operation as the steady state for the foreseeable horizon.Source: §4.3 · p. 7

4.4 The Phenomenon Across Verticals

  • Documented deployments across eight verticals show multi-lab use in finance, consulting, and government, including GSA agreements licensing all three labs for the federal workforce.In finance, JPMorgan and Goldman Sachs institutionalized multi-lab architectures, while Morgan Stanley pursued a deep single-vendor partnership; consulting firms also expanded partnerships with multiple labs.Source: §4.4 · p. 7, Figure 5 · p. 8

5 P1: Governance Fragmentation

Multi-lab licensing multiplies governance work across vendor reviews, documentation, data agreements, and regulatory obligations. Shadow AI further obscures use and spend, while governance tooling inherits underlying data fragmentation.

5 P1: Governance Fragmentation

  • Each additional lab multiplies the work of reconciling distinct security reviews, nonstandard model documentation, data-processing terms, and regulatory obligations on different timelines.Selected deployments are documented across verticals and use direct APIs, cloud platforms, or SaaS seats.Source: §5 · p. 8, Table 1 · p. 9
  • Shadow AI makes sanctioned-vendor governance incomplete: 69% of organizations suspect or have evidence of prohibited public GenAI use, and one case found $120,000 in annual spend absent from provider dashboards.IBM’s 2025 breach study found only 17% had technical controls preventing confidential-data uploads to public AI tools, while 86% lacked visibility into AI data flows.Source: §5 · p. 8
  • Governance platforms help manage inventories, policies, and attestations, but inherit heterogeneous documentation and telemetry and cannot normalize data providers do not expose.The paper frames governance fragmentation as downstream of a data problem.Source: §5 · p. 8

6 P2: FinOps Breakdown — Balancing Tokens and Dollars

Token consumption varies with model behavior rather than stable budgeting units, and provider pricing and billing models are heterogeneous. Surveys document widespread overruns, while the paper identifies a lack of a common spend unit across providers.

6.1 Why Tokens Defeat Budgets

  • AI costs scale with behavior: agentic workflows use roughly 7× the tokens of single-prompt usage, reasoning models multiply output tokens, and template changes can reprice workloads overnight.Average monthly AI token spend grew 13× over eighteen months in one large transaction panel.Source: §6.1 · p. 9
  • Uber exhausted its full-year 2026 AI budget by April after deploying agentic coding tools to roughly 5,000 engineers, then capped spend at $1,500 per employee per month.Power users had consumed $500–$2,000 per engineer per month.Source: §6.1 · pp. 9–10

6.2 Cross-Provider Pricing Heterogeneity

  • Across provider tiers, input prices span more than an order of magnitude and output prices nearly two; output-to-input price ratios vary from 2:1 to 6:1, alongside different commercial models.Pricing modifiers include caching discounts up to 90%, batch processing typically at 50%, and long-context surcharges.Source: §6.2 · p. 10
  • Seat-dollars and token-dollars are incommensurable, preventing direct comparison, attribution, and chargeback; falling inference prices have not matched rising consumption per task.The Tokenomics Foundation’s FOCUS extension is a standardization response, but launched in 2026 and remains unproven.Source: §6.2 · pp. 10–11

6.1 Why Tokens Defeat Budgets

  • 79% of organizations overran AI budgets in the past year; FinOps-mature organizations overran at 89% with a mean margin of 30.9%, versus 69% and 16.1% for novices.A separate survey found 80% missed AI infrastructure forecasts by more than 25%, and half of companies with AI-core products did not track LLM API costs.Source: §6.1 · p. 9, Figure 6 · p. 11

7 Evaluating API and Agent Cost Burn

API and agent workloads need economic evaluation alongside capability, because accuracy rankings can diverge from cost rankings and agent economics worsen with task difficulty. The section proposes five per-task, per-provider metrics and illustrates how cost-per-attempt rankings can reverse when measured per successful outcome.

7.1 From Capability Benchmarks to Economic Evaluation

  • Capability benchmarks omit solution cost, while procurement based on accuracy alone can select expensive configurations; agent components can substantially change cost-of-pass.The literature recommends joint accuracy-and-cost evaluation; one cost-optimized GAIA agent retained 96.7% of a leading open-source agent’s performance while improving cost-of-pass by 28.4%.Source: §7 · p. 11, §7.1 · p. 11

7.2 A Minimal Evaluation Framework for the Multi-Lab Enterprise

  • The five metrics are cost per attempt, cost-of-pass, burn multiplier, difficulty elasticity, and marginal cost of capability, computed per task class and provider.Cost-of-pass is expected successful-outcome cost (attempt cost divided by success rate) and is identified as the business-case metric.Source: §7.2 · p. 12
  • The five metrics must be computed by task class and provider, and measuring success and token use requires per-request, per-outcome telemetry that fragmented provider surfaces do not natively supply.The worked example makes telemetry a prerequisite for evaluating cost-of-pass and consumption by task class.Source: §7.2 · p. 12, §7.3 · p. 13

7.3 A Worked Example

  • Config C cost $0.044 per attempt and $0.367 per resolved ticket, versus Config A’s $0.24 and $0.32, so C was 5.5× cheaper per attempt but costlier per resolved ticket.Source: §7.3 · p. 13
  • Routing tickets to Config B first and escalating failures to Config A yields expected cost $0.264, success 0.86, and cost-of-pass $0.306.The cascade is near Config B’s economics while achieving a success rate above Config A’s.Source: §7.3 · p. 13

7.1 From Capability Benchmarks to Economic Evaluation

  • Claude 3.7 Sonnet led GAIA accuracy at 61.8% versus GPT-4.1’s 53.3%, but its cost-of-pass was 3.6× higher ($3.54 versus $0.98); hardest-tier cost-of-pass rose 534% for Claude 3.7 Sonnet and 646% for OpenAI o1.The cited study’s accuracy leader was not its economic leader, and agent economics deteriorated nonlinearly with task difficulty.Source: §7.1 · p. 11, Figure 9 · p. 14

8 P3: Telemetry Fragmentation - Measuring Adoption and Spend Across Labs

Provider telemetry surfaces differ in their data definitions, refresh timing, and visibility into cloud-routed consumption, so enterprises cannot reliably compare adoption or spend across labs. Third-party unification tools help measure routed API traffic, but leave important usage outside their view.

8 P3: Telemetry Fragmentation - Measuring Adoption and Spend Across Labs

  • Provider surfaces differ in ontology: seat products report users and conversations, APIs report tokens and requests, and providers define “active user” differently, making cross-lab adoption comparisons category errors.Table 4 compares provider administrative telemetry surfaces as of July 2026; its details change frequently.Source: §8 · p. 14, Table 4 · p. 15
  • Telemetry also differs in timing and coverage: refresh cycles range from near-real-time to 4–24 hours with 30-day revisions, while cloud-marketplace use can disappear from lab analytics.Cloud billing data may be the only place that marketplace-routed consumption reappears.Source: §8 · p. 14
  • Gateways, observability platforms, and cost platforms unify parts of measurement, but gateways cover only API traffic routed through them, excluding seat-based products and shadow usage.A single agentic query can fan out into dozens of LLM calls that remain invisible to a feature owner until the invoice arrives.Source: §8 · pp. 14–15

9 Discussion: P1–P3 as One Missing Abstraction

The paper treats governance, spend, and telemetry fragmentation as interdependent facets of a missing cross-provider enterprise AI control plane. It proposes a six-layer Token-Spend Governance framework and a conceptual cockpit, alongside a standards, telemetry, and research agenda.

9 Discussion: P1–P3 as One Missing Abstraction

  • Governance, finance, and platform engineering typically own P1, P2, and P3 separately, but the paper argues they are facets of one missing cross-provider control plane for shared policy, spend, and usage units.The proposed abstraction integrates concerns otherwise assigned to separate tool markets.Source: §9 · p. 15
  • P1 and P2 depend on P3 for consistent usage data, while telemetry alone needs policy and financial semantics; existing tools address slices but inherit lower-layer blind spots.Gateways normalize API traffic, FOCUS/Tokenomics normalizes cost records, and governance platforms normalize attestations, but none integrates all three.Source: §9 · pp. 15–16

9.1 A Token-Spend Governance Framework

  • The six-layer TSG framework runs from data ingestion and normalization through measurement, allocation, control, and assurance, mapping lower layers to P3, middle layers to P2, and upper layers to P1.It composes standards and results established in the paper rather than proposing new ones.Source: §9.1 · p. 16
  • TSG maturity runs from invisible to optimized, progressing through visibility, attribution, and governance; cloud-FinOps maturity does not transfer automatically because each rung must be rebuilt on token-native units.The ladder ranges from no unified view to cost-of-pass-driven routing across the lab portfolio.Source: §9.1 · p. 16

9.2 A Unified FinOps–Governance Cockpit (Concept)

  • The conceptual cockpit would combine comparable headline economics, task-class cost-of-pass by lab, governance alerts, and provenance labels for each number’s source and refresh latency.Its alerts include shadow-AI detections, agentic-cap breaches, cloud-routed consumption missing from lab consoles, and compliance-evidence status; it is a design concept, not a product.Source: §9.2 · pp. 16–17, Figure 10 · p. 17
  • The proposed agenda calls for shared definitions of normalized tokens, active users, and task classes; more complete provider telemetry; and routing research evaluated for governance and budget constraints.It also calls for longitudinal studies of multi-lab portfolios to strengthen the structural-not-transitional claim’s causal footing.Source: §9.2 · pp. 17–18

10 Conclusion

The conclusion presents multi-lab licensing as a settled, for-now enterprise configuration rooted in models’ different strengths, and identifies governance, FinOps, and telemetry problems as unified by missing cross-provider infrastructure. It offers the TSG framework and cockpit as starting designs for that layer.

10 Conclusion

  • The paper concludes that large enterprises license frontier labs because they are good at different things, creating governance, FinOps, and telemetry problems that share a missing control-plane abstraction.The proposed starting design combines standards, telemetry parity, and integrated tooling; the conclusion says enterprises otherwise keep rebuilding the missing layer privately and partially.Source: §10 · p. 18
  • The paper states that all quantitative claims cite public sources and that no proprietary data was used.Source: §10 · p. 18
Loading 2609.31622v1…