Source-linked AI summary
Coco: An Agentic Copilot for the Hardware--Software Co-Design Lifecycle
Samuel Kushnir, Kavya Sreedhar, Yeshwanth Reddy Pogula, Amir Yazdanbakhsh, Narges Shahidi, Ming Liu, Varun Gohil, Ravi Iyer, Parthasarathy Ranganathan, Christina Delimitrou, Suvinay Subramanian
TL;DR
Architects must reason about high-stakes accelerator designs using fresh simulation data and changing assumptions. Coco combines structured retrieval, composable tools, analysis agents, and interface context; early deployment reports more trusted evidence, richer comparisons, and less prompt-writing.
Problem
Method
Results
Early deployment reports that SQL-backed analyses were treated as grounded evidence, iso-execution surfaced configuration mechanisms, and captured navigation reduced prompt-writing.
Source: §4 · p. 3Takeaways & Limitations
Coco provides a practical system-architecture-level foundation for the autonomous co-designers of tomorrow.
Source: §6 · p. 4
Abstract
from arXiv · showhide
Co-designing ML models and the accelerators that run them is an unusual reasoning task: architects must draw confident, high-stakes conclusions about systems that do not yet exist, and the pace of both model evolution and hardware cadence means the analysis burden grows every quarter. The evidence behind each decision--hundreds of gigabytes of fresh simulation sweeps over novel design points--is by construction absent from any LLM's pretraining corpus, and there is no external literature to retrieve; naive "chat-with-your-data" approaches hallucinate exactly where correctness matters most. We present Coco (Copilot for Codesign), an agentic platform deployed with TPU architects that accelerates the co-design lifecycle of setting up experiments, sweeping simulators, and deriving insights. Coco is built as four layers: (i) a datastore that automatically registers every simulation sweep into a normalized relational schema, so agents ground every number in a SQL query rather than scraping heterogeneous files; (ii) a library of tools with typed APIs that agents compose without human orchestration; (iii) agents that encode recurring analysis workflows--most notably iso-execution analysis, which compares systems at matched execution configurations, including swept-but-dominated points off the Pareto frontier; and (iv) a platform UX whose navigation state doubles as agent context. We report early deployment experience toward a reduction in time-to-simulation and time-to-insight, and argue that co-design is a distinct agentic domain: its data must be retrieved rather than memorized, its workflows are recurring but context-dependent, and expert adoption hinges on UX that balances IDE-style control with interactive exploration.
1 Introduction
Co-design demands high-stakes analysis of proposed systems whose projected performance shifts as specifications and model assumptions change. Coco targets this burden with an agentic platform spanning experiment setup, simulation, and analysis.
1 Introduction
- Architects study proposed chips, interconnects, and packaging through simulation and analytical models, while committing to decisions with long lifetimes and high stakes.Hardware cadence, product complexity, and workload diversity are increasing.Source: §1 · p. 1
- A candidate system’s projected advantage can shift by tens of percent, requiring architects to separate specification, model, and sharding changes before acting.Comparative questions also require manual sweep archaeology, from locating runs to assembling plots.Source: §1 · pp. 1–2
- Coco is an agentic platform for setting up experiments, sweeping simulators, and analyzing results, with a near-term goal of halving time-to-simulation and time-to-insight.Its longer-term goal is to let the underlying LLM iterate on and explore architectural ideas.Source: §1 · p. 2
2 Why co-design needs its own copilot
Co-design needs a copilot because its evidence is fresh and organization-specific, its workflows recur with changing particulars, and its artifacts are collaborative objects that convey user intent.
2 Why co-design needs its own copilot
- Each design cycle regenerates hundreds of gigabytes of novel simulation data, so quantitative claims must be grounded in traceable queries and tool calls rather than model memory.There is no external literature to retrieve for these organization-specific sweeps, specifications, and documents.Source: §2 · p. 2
- Recurring analyses share a workflow skeleton, but use case, sequence-length regime, and metric vary; tools and encoded workflows can standardize the skeleton while the LLM handles those differences.Examples include rerunning a sweep with changed HBM bandwidth and retrieving peak memory for Pareto-optimal points.Source: §2 · p. 2
- Pareto curves, per-op breakdowns, timelines, and reports are collaborative artifacts to create, navigate, diff, and share; a user’s visited artifacts also signal intent.Source: §2 · p. 2
3 The Coco stack
Coco combines a relational datastore, composable typed tools, agents for recurring analyses, and a platform whose navigation captures context for those agents.
3 The Coco stack
- Simulation sweeps are registered in a normalized relational schema, and agents query it rather than scraping CSV, proto, or JSON artifacts.The schema includes test plans, workloads, systems, execution strategies, Pareto frontiers, and serving points with latency and throughput metrics.Source: §3 · p. 3
- Agents compose documented tools behind typed APIs into retrieval chains without human orchestration or bespoke glue for each analysis.A representative chain finds valid system/model combinations and Pareto curves, then fetches a curve, a point, and its breakdown.Source: §3 · p. 3
- Coco’s agents automate cross-system and cross-model comparisons; iso-execution analysis compares systems at a latency target and can retrieve swept configurations dominated off the Pareto curve.Sub-agents parallelize analysis across latency targets, reviewer agents check integrity, and an orchestrator synthesizes a report.Source: §3 · p. 3
- The platform attaches the viewed chart, use case, sequence-length regime, and question trajectory to a query, giving the agent context about the point under discussion.Its interface supports drilling from Pareto points into breakdowns and timelines, and diffing breakdowns across systems or strategies.Source: §3 · p. 3
4 Early experience
Coco is deployed with TPU architects for daily serving-focused analyses, and early use reports changes in trust, comparison depth, and interaction overhead.
4 Early experience
- Architects treat analyses that pair quoted numbers with their generating SQL as grounded evidence, placing data correctness on the schema rather than the model.Source: §4 · p. 3
- Iso-execution comparisons routinely identify chosen configurations and dominated alternatives, helping architects connect results to intuition or flag simulator assumptions behind a delta.Source: §4 · p. 3
- Encoding the user’s navigation trail as agent context eliminated most prompt-writing that had made earlier assistants feel slower than manual analysis.Source: §4 · p. 3
- The headline target is a 2× reduction in time-to-simulation and time-to-insight.Source: §4 · p. 4
5 Case studies
Two end-to-end case studies use public TPU generations, including deployed Ironwood and the announced TPU 8i. Coco reports a regime-specific SRAM benefit and analyzes how Boardfly changes network bottlenecks under highly sharded MoE serving.
5 Case studies
- The case studies compare deployed Ironwood systems with the announced eighth-generation TPU 8i inference chip.Source: §5 · p. 4
5.1 Generation-over-generation analysis: Ironwood to TPU 8i
- Tripling scratchpad from 128 MB to 384 MB benefits MoE models with fewer than 64 experts under a sub-2-second latency constraint.At those model shapes, larger SRAM allows expert tensors to be pinned and prefetched, removing HBM refetch from the critical path; outside that regime, the benefit washes out.Source: §5.1 · p. 4
5.2 Network topology what-if: torus versus Boardfly
- With sharding and batch size fixed, Boardfly reshapes dispatch and combine latencies, shifting execution from network-bound toward compute-bound.Side-by-side timeline breakdowns let the architect visually verify the bottleneck shift.Source: §5.2 · p. 4
6 Conclusion
Coco addresses co-design’s need for fresh-data reasoning and context-dependent workflows with a platform that shifts routine data archaeology and analysis from experts to agents. Its four-layer stack grounds insights in a relational datastore, automates recurring comparisons, and targets system-architecture decisions.
- Coco shifts data archaeology and routine analysis from experts to an agentic platform that grounds insights in a relational datastore and automates iso-execution comparisons.The interface is built for active architectural exploration.Source: §6 · p. 4
- Coco brings assistance to system-architecture decisions, where the paper says the highest-stakes choices are made.The authors position the four-layer stack as a practical foundation for autonomous co-designers.Source: §6 · p. 4