PaperlayerView exact revision on arXiv

Source-linked AI summary

Taurus: A Data Plane Architecture for Per-Packet ML

Tushar Swamy, Alexander Rucker, Muhammad Shahbaz, Ishan Gaur, Kunle Olukotun

arXiv:2002.08987v2cs.NIcs.LGcs.PF

TL;DR

Datacenter networks need complex decisions at per-packet line rate, but existing data planes lack the compute resources for modern ML and control-plane decisions are delayed. Taurus adds a SIMD MapReduce block to programmable switches; its evaluation reports low area and latency overhead alongside full-accuracy FPGA inference and substantially greater event detection than a control-plane baseline.

  • Problem

    Datacenter networks must combine per-packet, line-rate decisions with computationally intensive policies based on network-wide state, while existing data planes lack resources for modern ML and control-plane decisions incur long delays.

    Source: §1 · p. 1

  • Method

    Taurus extends PISA with a spatial SIMD MapReduce block, while the control plane trains and updates models that the data plane applies to per-packet decisions.

    Source: §1 · pp. 1–2

  • Results

    3.8% more area and 122 ns average latency accompany Taurus’s line-rate ML models; the FPGA prototype achieves full accuracy and detects two orders of magnitude more events than the control plane.

    Source: §1 · p. 2, §8 · p. 12

  • Takeaways & Limitations

    Taurus brings complex decision-making to the data plane, enabling per-packet reaction while operating at line rate with minimal overhead to a programmable switch pipeline.

    Source: §8 · pp. 11–12

    For larger datacenter networks, training reaction time is likely higher, so instantaneous problems cannot be handled through training and instead must be learned gradually.

    Source: §6 · p. 11

Abstract

from arXiv · show

Emerging applications -- cloud computing, the internet of things, and augmented/virtual reality -- demand responsive, secure, and scalable datacenter networks. These networks currently implement simple, per-packet, data-plane heuristics (e.g., ECMP and sketches) under a slow, millisecond-latency control plane that runs data-driven performance and security policies. However, to meet applications' service-level objectives (SLOs) in a modern data center, networks must bridge the gap between line-rate, per-packet execution and complex decision making. In this work, we present the design and implementation of Taurus, a data plane for line-rate inference. Taurus adds custom hardware based on a flexible, parallel-patterns (MapReduce) abstraction to programmable network devices, such as switches and NICs; this new hardware uses pipelined SIMD parallelism to enable per-packet MapReduce operations (e.g., inference). Our evaluation of a Taurus switch ASIC -- supporting several real-world models -- shows that Taurus operates orders of magnitude faster than a server-based control plane while increasing area by 3.8% and latency for line-rate ML models by up to 221 ns. Furthermore, our Taurus FPGA prototype achieves full model accuracy and detects two orders of magnitude more events than a state-of-the-art control-plane anomaly-detection system.

1 INTRODUCTION

Taurus addresses the gap between line-rate packet processing and complex network decisions by adding a MapReduce-based compute block to programmable data planes. Its evaluations report low switch overhead and substantially more event detection than a control-plane baseline.

1 INTRODUCTION

  • Data-plane ASICs react in nanoseconds but lack compute resources for modern ML, while control-plane round trips take 10 µs or more.The paper frames the challenge as combining per-packet, line-rate execution with computationally complex decisions.Source: §1 · pp. 1–2
  • The control plane trains and periodically updates models from network-wide observations, while the data plane uses the latest model for per-packet decisions.Preprocessing MATs prepare packet, flow, cross-flow, and device features; post-processing MATs turn model output into forwarding decisions, while packets needing no ML can bypass the block.Source: §1 · pp. 1–2
  • Taurus extends PISA with a spatial SIMD MapReduce block that runs pipelined 8-bit operations across 16 independent data elements.Compute units can be arranged in parallel for wide layers or in series for multiple layers.Source: §1 · p. 2
  • 122 ns average latency, 3.8% area overhead, and 2.8% power overhead for the largest configuration accompany Taurus’s line-rate ML models.The FPGA prototype achieves full model accuracy and detects two orders of magnitude more events than the control plane.Source: §1 · p. 2

2 THE NEED FOR PER-PACKET ML

Network applications need ML decisions quickly, but current data-plane and control-plane approaches each have limitations for per-packet inference. Existing switch abstractions lack suitable compute operations, while accelerator and caching approaches incur latency or resource costs.

2 THE NEED FOR PER-PACKET ML

  • A few microseconds’ delay can miss millions of packets, making reaction speed important for per-packet queue management, detection, mitigation, and traffic classification.The paper notes that slow decisions can lead to sub-optimal behavior and missed security and service-level objectives.Source: §2 · p. 2

2.1 Limitations of Data-Plane ML

  • Current MAT pipelines lack key ML operations such as loops and multiplication, and implementations can consume substantial resources relative to model quality.Binary neural networks on MATs are imprecise, while an SVM implementation uses eight additional tables on a NetFPGA reference switch.Source: §2.1 · p. 2
  • Separate accelerators need unbatched inference for deterministic latency, but can consume switch ports or duplicate packet-processing functions, adding area, reducing throughput, and consuming power.Batching improves throughput but would make data-plane packets wait for a batch to fill.Source: §2.1 · p. 3

2.2 Limitations of Control-Plane ML

  • Cache misses send unseen features to the control plane, where unbatched DNN inference takes 0.67 ms even on the fastest CPU, and TCAM rule installation takes 3 ms.The paper states that rule-installation time increases with flow-table size.Source: §2.2 · p. 3

3 TAURUS ARCHITECTURE

Taurus adds per-packet ML inference to switches and NICs, using model outputs for forwarding decisions. Its pipeline preprocesses packet and network-state features, performs MapReduce inference, and turns results into packet actions.

3 TAURUS ARCHITECTURE

  • Taurus runs ML models for every packet on switches and NICs, using their outputs to make forwarding decisions.The control plane installs both model weights and flow rules in switches.Source: §3 · p. 3

3.1 Parsing & Preprocessing

  • Preprocessing MATs prepare packet features, while feature engineering and telemetry can add domain, flow-history, and network-wide context.MATs can canonicalize headers and aggregate information across packets and flows; INT provides measurements embedded in packets.Source: §3.1 · p. 4

3.2 Postprocessing & Scheduling

  • Postprocessing MATs translate model outputs into packet actions, and compiled safety and liveness constraints can bound model decisions.The actions include dropping, flagging, or quarantining packets; inference can also connect to scheduling abstractions such as PIFO.Source: §3.2 · p. 4

3.3 MapReduce for Per-Packet ML Inference

  • MapReduce expresses ML’s linear computation as element-wise map operations and associative reductions, which can combine hierarchically into neural networks.The P4 control block uses nested maps for neurons and applies an activation function after each neuron’s linear operation.Source: §3.3 · pp. 4–5
  • The MapReduce abstraction also covers applications beyond ML, including stream processing, gradient aggregation, consistent hashing, and sketching.The paper presents this as support for a broader class of applications than match-action tables alone.Source: §3.3 · p. 5

4 TAURUS IMPLEMENTATION

Taurus integrates a MapReduce block with existing packet-processing stages and provides bypass paths for packets that need no inference. Its implementation combines pipelined compute units, on-chip SRAM, and compilation that maps programs onto the hardware.

MapReduce: Compute Unit (CU)

  • Taurus’s compute units use pipelined functional units, reduced-precision fixed-point arithmetic, and customized lane-to-stage ratios for resource efficiency.The CU stages support map and sparse reductions; the current CU/MU architecture supports dense ML models.Source: §4 · p. 5, Figure 8 · p. 6

MapReduce: Memory Unit (MU)

  • Banked on-chip SRAM stores model weights for single-cycle access, avoiding DRAM’s hundreds-of-cycles access time but limiting supported model size.Memory units are interspersed with compute units to provide locality and coarse-grained pipelining.Source: §4 · p. 5

4 TAURUS IMPLEMENTATION

  • Pipelining within compute units and the interconnect supports a 1 GHz clock frequency for matching high-end switch line rates.The physical pipeline combines parsing, MapReduce, forwarding MATs, scheduling, and a non-ML bypass path.Source: §4 · p. 5, Figure 7 · p. 6

Pre/Postprocessing MATs

  • Taurus combines VLIW match-action tables for preprocessing with SIMD MapReduce for inference, and compiles ML applications and other primitives into MapReduce.The architecture is based on Plasticine, modified for latency-optimized streaming inference in the network.Source: §4 · p. 5, Figure 5 · p. 5

Non-ML Traffic Bypass

  • Non-ML packets bypass MapReduce without added latency, while only feature headers enter the block and other headers continue to postprocessing.The pipeline divides packet queues across preprocessing, MapReduce, and postprocessing stages.Source: §4 · p. 6, Figure 6 · p. 6

Target-Independent Optimizations

  • MapReduce loop unrolling can provide one iteration per cycle when resources suffice, yielding deterministic throughput at line rate or a known fraction of it.The design supports target-independent resource-aware optimizations and target-dependent compilation from nested loops to a placed-and-routed streaming dataflow graph.Source: §4 · p. 6

5 EVALUATION

Taurus’s evaluation examines ASIC design choices, application performance and resource costs, and end-to-end anomaly detection and online training. It reports line-rate ML with modest ASIC area overhead and substantially more anomaly detection than the control-plane baseline.

5.1 Taurus ASIC Analysis

  • The ASIC targets full model accuracy and per-packet throughput, using 8-bit fixed point, 16 lanes, and four compute stages.Eight-bit precision has negligible quantization error and reduces resources by 4×; 16 lanes fit the largest layer of the anomaly-detection DNN, while four stages support both inner-product and ReLU microbenchmarks.Source: §5.1 · p. 7
  • At 1 GPkt/s, Taurus adds 3.8% chip area; KMeans, SVM, and DNN add 6.1%, 8.3%, and 22.1% latency, while bypassed packets add none.The latency percentages assume a datacenter switch latency of 1 µs.Source: §5.1 · p. 8
  • The application benchmarks cover IoT KMeans classification, SVM and DNN anomaly detection, and Indigo LSTM congestion control.The evaluated models include an 11-feature, five-category KMeans model; the anomaly detectors use eight and six input features, respectively.Source: §5.1 · p. 8
  • Indigo can produce a decision every 805 ns in Taurus, compared with its 10 ms software update interval.The paper says this permits faster reaction to load changes and better control of tail latency.Source: §5.1 · p. 8
  • The 16-element inner product runs at line rate in a single CU; the iterative loop-based convolution runs at one-eighth line rate, and reaching line rate by unrolling increases area 8×.ReLU and Leaky ReLU use a single CU, while Taylor-series and piecewise sigmoid/tanh approximations require 2–5× as much area.Source: §5.1 · pp. 8–9
  • For the anomaly-detection DNN, Taurus uses 3 MATs versus N2Net’s 48; for SVM and KMeans, it uses one MAT versus 8 and 2, respectively.The comparison is with MAT-only ML implementations.Source: §5.1 · p. 9

5.2 End-To-End Performance

  • Taurus detects two orders of magnitude more events than the control-plane baseline while matching the isolated model’s F1 score.The baseline misses packets during rule installation and other processing delays, reducing its effective F1 score.Source: Table 8 · p. 10, §5.2 · p. 10
  • Online training converges in tens to hundreds of milliseconds at higher sampling rates; batch size 64 and 10 epochs reach the highest reported F1 score.The paper describes the smaller-batch, more-epoch configuration as producing less frequent but more substantial updates.Source: §5.2 · p. 10

6 LIMITATIONS & FUTURE WORK

Taurus’s model capacity is bounded by available hardware, and training may be too slow for transient network events. The paper points to model compression, bounded postprocessing, and lower-latency telemetry as relevant approaches.

6 LIMITATIONS & FUTURE WORK

  • Model sizes remain limited by available hardware resources, and fitting larger models requires model-compression techniques.The paper also identifies provably correct model assertions and faster training as areas needing further research.Source: §6 · p. 11
  • Smaller networks can let Taurus run multiple models simultaneously, and quantization, pruning, and distillation can further reduce model size.The paper cites structured control nets with four neurons per layer as performing almost as well as 512-neuron DNNs for nonlinear control.Source: §6 · p. 11
  • ML is best suited to heuristic applications such as congestion control, load balancing, and anomaly detection, not tasks requiring definitive correctness guarantees.When an ML decision could affect network correctness, postprocessing rules can bound the final decision.Source: §6 · p. 11
  • Training reaction time would likely rise in larger datacenters, so training cannot address instantaneous problems such as load on a specific link.Training instead gradually learns what causes problems; lower-latency techniques such as INT can provide network status as model features.Source: §6 · p. 11

7 RELATED WORK

Prior networking platforms offer programmable packet processing or specialized ML acceleration, but the cited work identifies constraints in flexibility, scale, or latency. Taurus is positioned as a flexible data-plane approach, while its MapReduce abstraction can also support numeric aggregation.

7 RELATED WORK

  • FPGAs offer reconfigurability but face interconnect-power and clocking challenges, while specialized ML accelerators can be too rigid for evolving algorithms.The paper says CGRAs’ fast, fixed clock can ease integration between Taurus’s MapReduce block and MATs.Source: §7 · p. 11
  • MAT-based data-plane switches support diverse network tasks, but MATs alone are ill-suited to ML algorithms and can consume unnecessary area and power.The paper presents Taurus as a first step toward more efficient hardware for data-plane ML.Source: §7 · p. 11
  • SmartNICs and middleboxes remain a poor fit for network-wide per-packet decisions because packets must visit the server or NIC, adding a round trip.The paper also notes that middleboxes’ fixed-function hardware limits flexibility and prompts frequent upgrades.Source: §7 · p. 11
  • Although Taurus is not designed specifically for distributed training, its MapReduce block can aggregate packet-carried numeric weights more efficiently than MATs.Prior work cited in the passage reports that switches can aggregate gradients in-network to accelerate training by up to 300%.Source: §7 · p. 11

8 CONCLUSION

Taurus moves complex, data-driven decisions from the slower control plane into per-packet data-plane processing. It operates at line rate with modest reported switch overhead, and the authors view its hardware as a foothold for future in-network ML.

8 CONCLUSION

  • Taurus brings complex decision-making to the data plane, reducing reaction time to per-packet and increasing management-policy specificity.The conclusion contrasts this with networks partitioned between a line-rate data plane and a slower control plane.Source: §8 · p. 11
  • 3.8% more area and 122 ns average latency accompany Taurus’s line-rate operation while accelerating several recently proposed ML networking benchmarks.The conclusion describes the added overhead relative to RMT.Source: §8 · pp. 11–12
  • The authors believe Taurus hardware could provide a foothold for in-network ML in next-generation data planes to improve performance and security.Source: §8 · p. 12

A.1 Abstract

The artifact includes Taurus’s MapReduce block and anomaly-detection application code. It also describes their integration and the components used in the end-to-end evaluation testbed.

A.1 Abstract

  • The artifact provides source code for Taurus’s MapReduce block and the anomaly-detection application used in end-to-end performance evaluation.Source: §A.1 · p. 12
  • The MapReduce block is integrated with the Xilinx OpenNIC Shell and programmed using the Spatial high-level DSL.Source: §A.1 · p. 12
  • The anomaly-detection testbed application combines P4 programs, Python scripts, an ONOS application, and Spatial code.Source: §A.1 · p. 12

A.2 Scope

The artifact adds the MapReduce block and anomaly-detection code needed to implement the paper’s end-to-end testbed. Other required hardware and software components are existing tools and platforms.

A.2 Scope

  • The two new artifact contributions are the MapReduce block and anomaly-detection code, intended to supply missing components for the complete end-to-end evaluation testbed.Source: §A.2 · p. 12
  • The remaining testbed components include a Barefoot Tofino switch and Xilinx Alveo board, alongside P4, ONOS, Spatial, and other software codebases.Source: §A.2 · p. 12
  • The artifact states that the remaining hardware and software components are available for purchase and download.Source: §A.2 · p. 12

A.3 Contents

The artifact provides build and replication resources for the MapReduce block and anomaly-detection application, while relying on third-party hardware and software dependencies.

A.3 Contents

  • The MapReduce Block in FPGA repository provides source code and instructions for building an FPGA implementation.Source: §A.3 · p. 12
  • The anomaly-detection repository shares P4, Python, ONOS, and Spatial code and instructions for replicating the end-to-end testbed.Source: §A.3 · p. 12

A.5 Dependencies

  • The listed dependencies include an EdgeCore Wedge 100BF-32X switch, an Alveo U250 FPGA board, and tools such as P4 Studio, Vivado, and ONOS.Source: §A.5 · p. 12
  • Users are expected to obtain the required hardware and individual access to necessary tools and licenses; setup instructions for these components are not expanded.Source: §A.5 · p. 12
Loading 2002.08987v2…