Source-linked AI summary
DataBright: Towards a Global Exchange for Decentralized Data Ownership and Trusted Computation
David Dao, Dan Alistarh, Claudiu Musat, Ce Zhang
TL;DR
DataBright addresses the costs and limited incentives around obtaining data and computation, as well as the need to trust outsourced training. It combines data ownership and usage-based dividends with a trusted computation market; in its testbed, SGX/TLS scheduling and transmission added almost negligible overhead.
Problem
Deep-learning training requires costly data and computation, while outsourced computation must be verifiable and existing data sharing lacks incentives.
Source: §1 · p. 1, COMPUTATION MARKET · p. 3Method
DataBright combines a data registry that gives contributors shareholder status with a computation market using trusted devices to schedule and verify work by untrusted workers.
Source: §2 · p. 2, COMPUTATION MARKET · p. 3Results
SGX/TLS scheduling and transmission introduced almost negligible overhead in the VGG-16 ImageNet testbed, while two-node computation without splitting incurred significant communication overhead.
Source: §3.1 · p. 3Takeaways & Limitations
Contributors retain initial ownership and receive dividends whenever their data is used, with trusted computation protecting against unauthorized disclosure of whole datasets.
Source: §2 · p. 2Triple modular redundancy adds roughly 3× overhead, and the current version adds roughly 6× when users also choose model splitting.
Source: §3.1 · p. 3
Abstract
from arXiv · showhide
It is safe to assume that, for the foreseeable future, machine learning, especially deep learning will remain both data- and computation-hungry. In this paper, we ask: Can we build a global exchange where everyone can contribute computation and data to train the next generation of machine learning applications? We present an early, but running prototype of DataBright, a system that turns the creation of training examples and the sharing of computation into an investment mechanism. Unlike most crowdsourcing platforms, where the contributor gets paid when they submit their data, DataBright pays dividends whenever a contributor's data or hardware is used by someone to train a machine learning model. The contributor becomes a shareholder in the dataset they created. To enable the measurement of usage, a computation platform that contributors can trust is also necessary. DataBright thus merges both a data market and a trusted computation market. We illustrate that trusted computation can enable the creation of an AI market, where each data point has an exact value that should be paid to its creator. DataBright allows data creators to retain ownership of their contribution and attaches to it a measurable value. The value of the data is given by its utility in subsequent distributed computation done on the DataBright computation market. The computation market allocates tasks and subsequent payments to pooled hardware. This leads to the creation of a decentralized AI cloud. Our experiments show that trusted hardware such as Intel SGX can be added to the usual ML pipeline with no additional costs. We use this setting to orchestrate distributed computation that enables the creation of a computation market. DataBright is available for download at https://github.com/ds3lab/databright.
1 INTRODUCTION AND MOTIVATION
DataBright targets costly, opaque access to data and computation by proposing a decentralized market for both. Its motivation also includes computation resources being wasted on Bitcoin mining.
1 INTRODUCTION AND MOTIVATION
- Data access is opaque, sharing lacks incentives, and workers have little incentive to produce high-quality data.Cloud computation remains expensive, while computation resources are also being wasted on Bitcoin mining.Source: §1 · pp. 1–2
- DataBright proposes a decentralized market for data and computation.The proposal responds to limitations of existing crowdsourcing and cloud platforms.Source: §1 · pp. 1–2
- The paper identifies computation resources used for Bitcoin mining as wasted resources.Source: §1 · p. 2
2 DECENTRALIZED OWNERSHIP OF DATA
DataBright gives data creators initial ownership and dividends when their data is used, while a blockchain registry and voting process manage accepted proposals. Because uncontrolled disclosure can dilute data value, the design pairs the data market with trusted computation.
2 DECENTRALIZED OWNERSHIP OF DATA
- Creators retain initial ownership and receive dividends whenever their data is used, rather than only a one-time payment upon submission.The paper proposes this mechanism to encourage attention to data quality and utility over speed of creation.Source: §2 · p. 2
- Contributors submit dataset proposals with their wallet and dataset addresses, and accepted proposals enter an immutable blockchain registry.Curators holding computational tokens vote on proposals, and reaching the vote threshold accepts a proposal.Source: §2 · p. 2
- Accepted contributors become shareholders through newly minted tokens, while the registry supports dataset access and payment routing by the computation market.The DApp uses CuratorToken, DataAssociation, and Database contracts to manage voting, proposals, and accepted entries.Source: §2 · p. 2
- Uncontrolled disclosure can dilute data value, so DataBright pairs the data market with trusted computation to avoid disclosing whole datasets to unauthorized entities.Source: §2 · p. 2
COMPUTATION MARKET
DataBright combines trusted devices for scheduling and verification with untrusted workers for bulk computation, and uses redundancy, data limits, or model splitting as protections. In the reported testbed, SGX/TLS scheduling and transmission added almost negligible overhead, while model splitting and redundancy carried substantial overheads.
COMPUTATION MARKET
- The user must be able to trust that paid computation faithfully trains the requested model, but many deep-learning workloads run on GPUs without Intel SGX support.Source: COMPUTATION MARKET · p. 3
- The computation market uses a small set of trusted devices for scheduling and verification and a larger pool of untrusted workers, such as GPUs, for bulk work.This hybrid design avoids requiring Intel SGX support on all workers, an assumption the paper says is unsuitable for GPU-heavy deep-learning computation.Source: COMPUTATION MARKET · p. 3
- DataBright offers triple modular redundancy, periodic limits restricting each untrusted worker to at most 5% of the training set, and optional model splitting across workers.With model splitting, workers exchange activations and gradients for a single layer, and no worker accesses the full model.Source: COMPUTATION MARKET · p. 3
3.1 EXPERIMENTS
- SGX/TLS scheduling and transmission introduced almost negligible overhead in the VGG-16 ImageNet testbed, while two-node computation without splitting incurred significant communication overhead.The machines were connected by a 1Gbps network.Source: §3.1 · p. 3
- Triple modular redundancy adds roughly 3× overhead, and the current version adds roughly 6× when users also choose model splitting.The authors say the splitting overhead could be reduced through optimization, including asynchronous training, and that the current implementation does not always keep both GPUs busy.Source: §3.1 · p. 3