PaperlayerView exact revision on arXiv

Source-linked AI summary

Cost-Aware Mixture-of-Experts Coordination for Model Markets

Yizhou Ma, Wenbo Wu, Xikun Jiang, Zhuoqin Yang, Luis-Daniel Ibáñez

arXiv:2610.11908v1cs.DBcs.LG

TL;DR

Existing model marketplaces typically trade individual models, limiting support for complementary experts. This paper uses MoE as a market-level coordination framework with cost-aware gating and revenue allocation. Across 15 tabular and image datasets, it reports higher mean welfare than alternatives and lower mean expected cost than Standard MoE in every case, with competitive predictive performance.

  • Problem

    Existing model marketplaces typically trade individual models, limiting support for services that combine complementary capabilities across providers.

    Source: §1 · p. 1

  • Method

    The paper formalizes an MoE-based model market and develops cost-aware gating, market-aware training, and cost-adjusted revenue allocation.

    Source: §1 · p. 2

  • Results

    MoE Market achieves the highest mean welfare on all 15 datasets and lower mean expected cost than Standard MoE in every case, while retaining competitive predictive performance.

    Source: §6 · p. 12

  • Takeaways & Limitations

    The results support MoE as a market-level coordination principle for collaborative AI service delivery.

    Source: §6 · pp. 11–12

    The analysis assumes expert participation costs are measurable and uniformly bounded.

    Source: §3.3 · p. 5

Abstract

from arXiv · show

Existing model marketplaces typically trade and select individual models as indivisible units, limiting their ability to exploit complementarities among heterogeneous experts. This paper proposes an MoE-based model market framework that lifts Mixture-of-Experts from a model-level learning architecture to a market-level coordination mechanism. In this framework, brokers use gating networks to coordinate multiple heterogeneous experts and deliver a composite model service. We formalize the market participants, service workflow, expert cost structure, and a welfare objective that combines predictive utility with heterogeneous execution costs. We then derive a cost-aware gating mechanism and market-aware training objective, and introduce a cost-adjusted revenue allocation rule that distributes residual revenue according to realized expert participation and execution cost. We also establish basic theoretical properties of the allocation rule, including budget balance, participation monotonicity, and cost sensitivity. Experiments over five random seeds on fifteen tabular and image benchmarks use independently trained and frozen neural and tree-based experts together with latency-derived execution costs. MoE Market achieves the highest mean welfare on all fifteen datasets and a lower mean expected cost than Standard MoE in every case, while maintaining competitive predictive performance. The allocation experiments further demonstrate systematic sensitivity to expert participation and cost, together with substantially lower computational overhead than exact Shapley allocation. These results suggest that MoE can serve as a market-level coordination principle for collaborative, cost-aware, and economically grounded model marketplaces.

1 Introduction

Existing model marketplaces typically sell models from individual providers, limiting support for services built from complementary experts. This paper extends MoE into a market-level coordination framework and develops cost-aware coordination and revenue allocation.

1 Introduction

  • Existing marketplaces typically let buyers select one provider’s model, limiting support for tasks that benefit from complementary expert capabilities.The paper motivates cooperative mechanisms to coordinate providers in delivering composite models.Source: §1 · p. 1
  • The paper formalizes an MoE-based model market and evaluates cost-aware coordination, reporting improved welfare with competitive predictive performance.It also compares cost-adjusted revenue allocation with participation-only and exact Shapley allocation.Source: §1 · p. 2
  • The framework defines market participants, workflow, expert costs, utility, and a welfare objective for collaborative composite services.Its cost-aware gating mechanism and market-aware training objective coordinate experts under model quality–cost trade-offs.Source: §1 · p. 2
  • The allocation comparison reports cost-sensitive payments and substantially lower runtime than exact Shapley allocation.The proposed rule distributes residual revenue according to realized expert contribution and execution cost.Source: §1 · p. 2

2 Related Work

Prior work covers data and model marketplaces, including pricing, privacy, and contribution mechanisms, while MoE research applies gating and specialized experts across learning settings. This paper positions MoE as a market-level coordination mechanism among heterogeneous providers.

Related Work

  • Data marketplaces exchange datasets, while model marketplaces trade models and address pricing, privacy, and contributor compensation.Existing marketplaces largely treat models as indivisible commodities and do not dynamically coordinate heterogeneous models for a composite service.Source: §2 · p. 2
  • Conventional MoE systems use gating networks to select and combine specialized models, but co-train gates and experts within one learning pipeline.MoE has been applied across continual, meta-, multi-task, reinforcement, federated, and language-model learning.Source: §2 · p. 2
  • The paper extends MoE to market-level coordination, where expert participation reflects both model utility and operational cost.The authors state that MoE had not previously been proposed as a coordination mechanism among heterogeneous model providers.Source: §2 · p. 2

3 Problem Formulation

The formulation defines a broker-mediated market in which a gating network coordinates independently trained heterogeneous experts to deliver composite services. It formalizes the service workflow, expert execution costs, a predictive-utility-and-cost welfare objective, and the residual revenue pool for expert compensation.

3.1 Market Participants

  • The broker receives buyer requests, acquires task-relevant data, coordinates experts through a gating network, and delivers the resulting composite model service.The market also includes buyers, data providers, and heterogeneous experts.Source: §3.1 · p. 3
  • Routing weights are generated for each input–request pair over independently instantiated experts, and only activated experts execute and qualify for payment.Expert parameters remain fixed while the broker trains the gate.Source: §3.1 · p. 4

3.2 MoE Market Workflow

  • The workflow moves from request submission and data acquisition through expert routing and weighted prediction to feasible service delivery, payment, and residual-revenue allocation.A transaction must meet the buyer’s quality requirement and budget constraint.Source: §3.2 · p. 4

3.3 Cost Model

  • Expert cost captures operational or economic expense, while instance-level service cost weights each expert’s invocation cost by its workload share and omits fixed invocation fees.Cost factors may include computation, latency, resource use, opportunity cost, or internal service charges.Source: §3.3 · pp. 4–5

3.4 Utility and Welfare Objective

  • The welfare objective evaluates predictive utility and expected service cost on the target deployment distribution, assuming request-specific training and validation data represent that distribution.It is a trainable criterion, not a model of all monetary transfers.Source: §3.4 · p. 5
  • Predictive loss is task-specific—cross entropy for classification and mean squared error for regression—and utility is defined as an exponential transformation of that loss.Source: §3.4 · p. 5
  • The market-aware objective trains the gate to reward composite prediction quality and penalize costly expert usage while leaving independently trained expert models fixed.The loss is the optimization form induced by the market-welfare objective.Source: §3.4 · p. 5
  • After the broker retains its coordination fee, residual revenue is available for allocation among participating experts according to their service contributions.The formulation assumes a non-negative residual pool.Source: §3.4 · pp. 5–6

4 Revenue Allocation

The allocation divides residual revenue among participating experts using contribution scores that account for both realized participation and execution cost. Its stated properties include budget balance, participation monotonicity, cost sensitivity, and limiting behavior controlled by the cost-discount parameter.

Allocation Properties

  • Total expert payments equal the residual revenue pool, making the allocation budget-balanced.The pool remains after broker coordination fees and expected expert execution costs have been accounted for.Source: §4 · p. 6

Cost-Adjusted Revenue Allocation

  • Experts share residual revenue in proportion to scores that combine realized participation with an exponential execution-cost discount; only experts with positive scores receive revenue.The parameter λ controls the discount strength: larger values penalize costly experts more, while smaller values move allocation closer to participation-proportional sharing.Source: §4 · p. 6

Allocation Properties

  • When the residual revenue is positive, an expert with a higher contribution score receives at least as much as one with a lower score, and strictly more if their scores differ.This comparison assumes the total score across experts is positive.Source: §4 · p. 6
  • If one expert’s cost increases pointwise while other quantities stay fixed, that expert’s score and payment weakly decrease.The result assumes a positive cost-discount parameter and a nonnegative residual pool.Source: §4 · p. 6

Limiting Cases of λ

  • As λ approaches zero, allocation approaches participation-proportional sharing; with constant costs and a unique lowest-cost participating expert, it concentrates on that expert as λ grows without bound.The latter limit assumes the unique lowest-cost expert has positive expected participation.Source: §4 · p. 6

5 Experimental Evaluation

The evaluation tests whether MoE Market can coordinate heterogeneous experts for predictive performance and welfare under latency-derived costs. It uses fifteen tabular and image benchmarks, independently fitted experts, and shared experimental conditions for method comparisons.

5 Experimental Evaluation

  • The evaluation asks whether MoE Market matches or exceeds baseline predictive performance, improves welfare under latency-derived costs, and allocates payments meaningfully with lower overhead than exact Shapley allocation.The comparison baselines are Welfare-aware Static Selection, Uniform Average, and Standard MoE.Source: §5 · p. 7

5.1 Experimental Setup

  • Each benchmark dataset represents a buyer request, with four independently trained experts, a broker gate, and a weighted ensemble representing the delivered composite service.Data-provider discovery, dataset integration, and negotiation are not separately simulated.Source: §5.1 · p. 7

5.1.1 Datasets and tasks.

  • The study covers fifteen tabular and image datasets, including regression, binary classification, and multiclass classification tasks, with shared preprocessing and seed-specific splits across methods.The authors state that fixed, method-agnostic preprocessing helps isolate architectural and coordination differences.Source: §5.1.1 · p. 7

5.1.2 Heterogeneous Expert pools.

  • Each task uses four independently trained and then frozen experts, so differences between Standard MoE and MoE Market arise from their gating objectives rather than different expert models.The expert pool combines neural and tree-based architectures, and the gate learns input-dependent suitability without assuming any expert is universally superior.Source: §5.1.2 · p. 7

5.1.3 Gating Network Architecture.

  • Task-specific gates produce four expert-routing weights using an MLP for tabular tasks and a convolutional network for image tasks, while the experts remain frozen.Standard MoE and MoE Market share the expert bank, gating architecture, data split, and initialization protocol on each task.Source: §5.1.3 · pp. 7–8

5.1.4 Experts Costs.

  • Expert costs are derived from measured inference latency, normalized to a profile with mean one, and held fixed across methods and seed repetitions for each dataset.The latency procedure uses warm-up passes followed by timed forward passes; costs therefore reflect expert architecture and request-specific input representation.Source: §5.1.4 · p. 8

5.1.5 Implementation and hardware configuration.

  • Experiments use a shared software and hardware environment, and methods use the same splits, independently fitted expert bank, and latency-derived cost profile unless otherwise stated.The implementation uses Python, PyTorch, scikit-learn, XGBoost, and CatBoost under CUDA on a Windows system.Source: §5.1.5 · p. 8

5.2 Baselines

MoE Market is compared with single-expert selection, non-adaptive averaging, and a predictive-only MoE gate. The methods share the same frozen expert bank and dataset conditions.

Welfare-aware Static Selection

  • Welfare-aware Static Selection chooses one expert for the entire request using validation performance and measured cost, without using test-set outcomes.It represents a conventional model market that selects an individual model rather than combining experts.Source: §5.2 · p. 8

Uniform Average

  • Uniform Average gives all frozen experts equal weight, representing non-adaptive collaboration that ignores input-dependent capabilities and execution costs.Source: §5.2 · p. 8

Standard MoE

  • Standard MoE learns input-dependent weights over the frozen experts but optimizes predictive performance without incorporating expert costs.It serves as a cost-unaware gating baseline.Source: §5.2 · p. 8

MoE Market

  • MoE Market trains the same gate and uses the same frozen experts as Standard MoE, but optimizes the market-aware objective.This distinguishes the proposed method from the predictive-only gate while holding the expert bank and gating architecture constant.Source: §5.2 · p. 8

5.3 Model Training

The training protocol fits the heterogeneous experts independently, freezes them, and trains separate gates for Standard MoE and MoE Market. Evaluation uses held-out instances from each request distribution across five random seeds.

5.3 Model Training

  • Neural and tree-based experts are fitted independently on each training split and then frozen, so coordination methods use an identical expert bank.Neural experts train for 10 epochs; XGBoost and CatBoost use 200 boosting iterations.Source: §5.3 · p. 8
  • Standard MoE and MoE Market train gates separately with the same architecture and initialization, using predictive loss and the market-aware objective, respectively.The measured cost profile is shared, and validation objectives determine the checkpoints used for test evaluation.Source: §5.3 · p. 8
  • Each dataset has its own gate, trained on request-specific data; test splits assess generalization to unseen instances from the same request distribution, not zero-shot transfer to new tasks.Experiments are repeated over five random seeds.Source: §5.3 · p. 8
  • The result tables report mean ± standard deviation over five random seeds for tabular classification, tabular regression, and image classification tasks.Source: Table 2 · p. 9, Table 3 · p. 9, Table 4 · p. 10

KMNIST

MoE Market leads the benchmark comparisons in mean welfare while reducing expected cost relative to Standard MoE. The cost coefficient controls the quality–cost trade-off, with real welfare highest near the deployment coefficient.

KMNIST

  • KMNIST results report sample standard deviations across runs for predictive metrics, expected service cost, and market welfare; WSS, UA, SM, and MM denote the four methods.The methods are Welfare-aware Static Selection, Uniform Average, Standard MoE, and MoE Market.Source: KMNIST · p. 10

5.4 Market Efficiency

  • MoE Market achieves the highest mean welfare on all 15 datasets and lower mean expected cost than Standard MoE in every case.Standard MoE is more accurate on some datasets, while WSS has the lowest cost but lower welfare than MoE Market.Source: §5.4 · p. 10

5.5 Sensitivity Analysis of β

  • Figure 3 evaluates β from 0 to 1 in 0.1 increments on one Adult Census run, retraining the gate while holding the expert bank and measured cost profile fixed.Its panels report training welfare, real welfare, utility, expected cost, cross-entropy loss, accuracy, Macro-F1, and AUC.Source: §5.5 · p. 10, Figure 3 · p. 10
  • Increasing β from 0.1–0.5 reduces expected cost from approximately 0.43 to 0.11–0.19 with limited predictive degradation; real welfare peaks near β=0.3.At β=0, routing closely matches Standard MoE; for β≥0.6, further cost reductions no longer compensate for declines in utility and predictive performance.Source: §5.5 · pp. 10–11

5.6 Revenue Allocation

The allocation comparison tests whether payments reflect both expert participation and execution cost. The proposed rule shifts shares toward lower-cost experts and has substantially lower runtime than exact Shapley allocation.

5.6 Revenue Allocation

  • The comparison uses five-seed results on Breast Cancer and Diabetes with λ=0.6, against participation-only allocation and exact Shapley allocation.Exact Shapley enumerates all 2^|E| coalitions using predictive utility after retaining and renormalizing each coalition’s experts; negative values are clipped before normalization.Source: §5.6 · p. 11
  • In both representative cases, the proposed rule shifts revenue toward lower-cost experts, including the lower-cost E1, and away from costly or higher-cost experts.E1 and E3 have similar participation in Case 1, but E1 receives a larger share; in Case 2, allocation shifts away from higher-cost E4.Source: §5.6 · p. 11
  • Mean allocation speedups over exact Shapley are approximately 99.9× on Breast Cancer and 82.4× on Diabetes.After participation statistics are available, the proposed rule is linear in the number of experts, whereas exact Shapley enumerates coalitions.Source: §5.6 · p. 11

6 Conclusion

The conclusion presents MoE Market as a market-level framework for coordinating heterogeneous experts, combining cost-aware routing with cost-adjusted revenue allocation. Experiments report welfare and cost advantages across the evaluated datasets, while dynamic market conditions remain future work.

6 Conclusion

  • The framework formalizes market participants, workflow, costs, and welfare, and combines cost-aware gating with cost-adjusted revenue allocation.The allocation rule is reported to satisfy budget balance, participation monotonicity, and cost sensitivity.Source: §6 · p. 12
  • Across five-seed experiments on 15 tabular and image datasets, MoE Market achieves highest mean welfare on every dataset and lower mean expected cost than Standard MoE in every case.The evaluation uses independently trained and frozen MLP, Transformer, XGBoost, and CatBoost experts with latency-based execution costs.Source: §6 · p. 12
  • Future work will examine dynamic model markets where expert availability, execution costs, and deployment distributions evolve over time.Source: §6 · p. 12

Artifacts

The paper provides artifacts for reproducing its experiments, including source code, scripts, and instructions.

Artifacts

  • The experiment artifacts include source code, experiment scripts, and reproduction instructions.The paper identifies the repository as https://github.com/MAYIZHOU/artifact-code_moe.git.Source: Artifacts · p. 12
Loading 2610.11908v1…