Source-linked AI summary
Pairit: A Platform for Live Experiments on Human-AI Collaboration
Harang Ju, Sinan Aral
TL;DR
Researchers lack a convenient way to declare auditable live experiments in which AI agents communicate and act on shared work alongside human participants. Pairit addresses this gap with YAML-declared executable experiment graphs for configuring and running mixed human–AI sessions. Multiple live deployments, including peer-reviewed studies, demonstrate operational use while broader adoption still depends on external execution requirements.
Problem
Researchers cannot easily declare auditable protocols combining AI communication, shared work, protocol actions, team assignment, and routing in one configuration.
Method
Pairit uses a single YAML configuration to declare executable experiment graphs containing mixed human–AI participants, routing, randomization, matchmaking, chat, workspaces, and server-hosted agents.
Results
Pairit has supported multiple live deployments, including peer-reviewed research, with process logs from human–AI collaboration and negotiation studies.
Takeaways & Limitations
Pairit provides reusable infrastructure for specifying, deploying, and sharing live human–AI organizational experiments through declarative experiment graphs.
Takeaways & Limitations
Wider adoption remains dependent on institutional review, participant recruitment, model access, and hosting beyond the Pairit configuration.
Abstract
from arXiv · showhide
Organizational design in the era of artificial intelligence requires experimental methods that can test how human-AI groups coordinate, delegate, and make decisions. Programmable platforms coordinate live human-to-human sessions or real-time human-AI chat, but researchers cannot easily declare experiment protocols in which AI participants both communicate and act on shared work within one auditable configuration. Here we introduce Pairit, an online platform that facilitates the design, testing, and deployment of experiments that test human-AI organizational designs and interventions. Through a single YAML configuration file, researchers declare an executable experiment graph (pages, routing, randomization, matchmaking, chat, shared workspaces, server-hosted agents, surveys, timers, and custom HTML components) and combine any number of humans and AI agents in live sessions. We have validated the feasibility of the platform through multiple live deployments, including peer-reviewed published studies, capturing high-resolution process traces of communication, negotiation, and collaborative work in live human-AI dyads. By representing complex interactive protocols as standardized, auditable configuration files, Pairit provides reusable infrastructure for specifying, deploying, and sharing live human-AI organizational experiments.
1 Introduction
Pairit addresses the difficulty of declaring auditable live experiments that combine human and AI participants, communication, shared work, and protocol actions. It provides a configurable platform for testing how AI roles affect coordination and collective performance.
- 1 Introduction: Existing platforms do not easily express protocols where AI agents converse, co-edit shared documents, and execute actions within one auditable configuration.The missing configuration also includes team assignment, communication channels, and routing.
- 1 Introduction: Pairit supports live sessions containing any mix of human and AI participants, including human–human baselines, human–AI dyads, facilitators, and negotiation assistants.Sessions combine matchmaking, chat, shared workspaces, and server-hosted AI actions.
- 1 Introduction: Researchers can dynamically configure AI roles, including persona prompts based on intake surveys and assistance restricted to particular task stages.These controls support experiments on communication, coordination, delegation, and collective performance.
- 1 Introduction: Figure 1 illustrates that studies can combine any n humans and m AI agents with built-in components.The figure presents team composition as a configurable part of the study design.
2 System Architecture and Declarative Experiment Graphs
Pairit represents interactive experiments as executable directed graphs declared in YAML. This declarative representation separates experimental design from software engineering while supporting routing, randomization, collaboration, and AI behavior.
- 2 System Architecture and Declarative Experiment Graphs: Researchers specify pages as graph nodes and participant actions or routing rules as edges in a single YAML configuration file.The configuration can define randomization, matchmaking, chat and workspace permissions, and agent models, prompts, and triggers.
- 2 System Architecture and Declarative Experiment Graphs: Built-in components provide surveys, stimulus displays, timers, matchmaking pools, chat rooms, and collaborative workspaces as core experimental primitives.Session variables drive conditional routing and real-time interface updates.
- 2 System Architecture and Declarative Experiment Graphs: The declarative experiment graph decouples experimental design from software engineering.This abstraction makes the study protocol the primary reusable specification rather than a collection of implementation files.
A. Declarative config
The supplied configuration excerpt shows a component-oriented declaration of matchmaking, layout, and live agents. These entries encode interactive study structure rather than participant-facing prose.
- A. Declarative config: The configuration declares a matchmaking component with the identifier discuss.This names a reusable matchmaking element in the study specification.
- A. Declarative config: The layout uses a split arrangement for components.The excerpt represents page presentation through a layout declaration.
- A. Declarative config: The configuration includes a live agent named facilitator alongside matchmaking.This combines an agent role with the session’s interactive coordination structure.
C. Participant experience
Pairit compiles declared pages, matchmaking, and AI facilitation into a graph that is rendered as a participant-facing study. Its broader design addresses limitations of systems that cannot coordinate shared work and acting AI participants.
- C. Participant experience: Figure 2 traces the path from YAML declarations through a compiled node-and-edge graph to a rendered study with a facilitator and shared workspace.The stages connect configuration, execution structure, and participant experience.
- C. Participant experience: Existing behavioral platforms generally do not coordinate synchronous matchmaking, collaborative drafting, and AI participants that both converse and act on shared work within a declared protocol.This limitation motivates Pairit’s participant-facing integration of communication and collaborative work.
3 Validation and Reproducibility
Pairit has been validated across multiple live deployments, including peer-reviewed research, with public configurations and export schemas designed to support auditability and reproducibility.
- Validation: Pairit supported multiple live deployments in peer-reviewed research and working papers, including published studies of human–AI collaboration.These deployments included randomized human–human and human–AI team compositions and AI personality conditions.
- Validation: Experiments with 2,234 Prolific participants produced process logs used to analyze gender gaps, diversity collapse, and the jagged frontier.Participants collaborated in synchronized pairs to produce advertising campaigns later tested in a live market.
- Reproducibility: Researchers can inspect, adapt, and run public configurations and export schemas without access to prior-study participant-level data.The platform separates reproducible protocol objects from prior-study participant data.
- Reproducibility: Pairit makes multi-agent designs auditable and directly reproducible by publishing executable protocol structure alongside the platform.The reproducible objects are intended for reuse across laboratories.
4 Implications for Organizational Research
Pairit turns live human–AI organizational experiments from bespoke infrastructure projects into declarative, repeatable protocols that can be audited, shared, and pretested.
- Implications: Pairit converts live coordination, communication, and delegation infrastructure into a declarative experiment graph.Researchers can vary participants, AI roles, and intervention timing within the same protocol framework.
- Implications: Researchers can test facilitation, delegation, and bargaining assistance in repeatable, customizable, and shareable live experiments.These designs replace one-off custom builds with reusable experimental protocols.
- Reproducibility: Executable configurations let reviewers and laboratories audit prompts, routing, timeouts, and agent settings directly.Shared lab links support reproducing or adapting protocols, while process logs record interaction history.
- Pretesting: Simulated participants can populate the same declared configurations used for live sessions, opening a path to directly comparable in silico pretesting.The paper describes this extension as under development.
5 Outlook
Pairit is being extended beyond its current deployments through beta access for outside laboratories, while broader adoption remains dependent on factors beyond protocol configuration.
- Outlook: Pairit is rolling out beta access so outside laboratories can run published human–AI protocols from shared configurations.This is intended to avoid bespoke live-infrastructure development by each group.
- Outlook: Wider use still depends on review, recruitment, model access, and hosting choices beyond Pairit protocol configuration.The paper identifies adoption of executable configurations as an open question.
A Specification and Runtime Architecture
Pairit represents experiments as validated, executable graphs whose components coordinate participant flow, AI actions, shared work, custom interfaces, and telemetry during live sessions.
- Specification: Researchers declare pages and participant actions as a directed graph in YAML, which the Manager CLI validates, compiles, and uploads.The compiled graph is then consumed by the runtime server.
- Runtime: The Lab Server coordinates sessions, matchmaking, and AI-agent calls while persisting state and events and streaming updates to participant browsers.Exported data come from the underlying event and session stores.
- Runtime: Strongly typed session state drives conditional routing, automated onEnter actions, treatment assignment, intake scoring, and matchmaking.Transition rules determine the next page from participant inputs and current session state.
- Components: Pages compose first-class matchmaking, chat, workspace, and server-managed agent components, with when expressions controlling conditional rendering.The component-first architecture mirrors hierarchical React JSX structure.
- Built-in components: Built-in components cover matchmaking, randomization, agents, media, timers, live workspaces, and custom HTML interfaces.These components provide reusable primitives for synchronous assignment, autonomous interventions, collaborative editing, stimuli, temporal constraints, and bespoke tasks.
- Custom interfaces: Custom HTML components run in sandboxed iframes, use the pairit helper API for state and telemetry, and can block navigation until completion.The API connects bespoke interfaces to matchmaking, agents, workspaces, routing, and event logging.
- Agents: Agents can activate on message, room entry, or message intervals and use server-side tools to assign state, end chats, or edit shared workspaces.This supports agents acting as facilitators, bargaining counterparties, and co-authors rather than passive conversational partners.
- Shared work: Live workspaces support freeform markdown or structured forms, participant or group scope, and configurable editing permissions alongside chat.The workspace can be displayed in a split layout with chat.
H Data Export and Protocol Auditing
Pairit makes experimental data and protocols auditable through standardized exports, temporally ordered traces, configuration linting, and human-readable executable specifications. Its live deployments demonstrate scalable workflows, while adoption, model dependence, and simulated-participant fidelity remain bounded by practical constraints.
- Data Export: Pairit exports complete experimental records in CSV, JSON, and JSONL, converting MongoDB collections into six discrete relational files.The standard export formats and six-file structure support structured downstream analysis.
- Protocol Auditing: Exact temporal ordering preserves step-by-step reconstruction of team deliberation, coordination, and task progress.AI-partner records also include model metadata, prompts, conversation history, hyperparameters, and raw outputs.
- Protocol Auditing: A single human-readable configuration file lets researchers and reviewers audit protocols without inspecting disparate server scripts or frontend code.The Manager CLI also checks page targets, graph edges, component properties, and agent specifications before deployment.
- Interactive Template Library: Runnable browser templates support reproducible study design and let researchers test complex multi-player and agent interactions without hosting accounts, databases, or API credentials.The interactive library includes fourteen standard demonstration templates with downloadable configurations and live browser demos.
- Scale and Empirical Precedent: Live deployments validated synchronized multi-party workflows at scale, including matchmaking, concurrent document editing, and server-managed AI interactions.The platform preserves functional primitives used in an earlier field experiment with 2,234 participants collaborating in pairs.
- Execution Risks and Open Questions: Wider adoption still depends on institutional review, participant recruitment, model access, hosting, and whether executable protocols travel across laboratories as routinely as survey instruments.Simulated language-model participants may reproduce conversational patterns without capturing full live-team social dynamics, and model revisions may hinder replication.
M Software Availability and Reproducibility
Pairit is operationally available through public source code, documentation, browser demos, and a template library. Its alpha deployments supported refinement, while beta rollout and public reproducible objects support external inspection and replication.
- Software Availability: Pairit’s public source code, CLI tools, and core server packages are available for inspection.The platform also provides public documentation and interactive browser demos.
- Software Availability: The stack uses Bun and Elysia, React with Vite and Tailwind CSS, MongoDB, and server-sent events for hosted real-time studies.Researchers run studies through the hosted platform.
- Reproducibility: Alpha testing with research laboratories helped deploy and refine core components, and beta rollout is supporting structural replications by external labs.Workshops continue refining the public template library and expanding experimental designs.
- Reproducibility: Reproducible objects for the technical submission comprise public example configurations and standardized export schemas, while participant-level data remain with the original publications under institutional review protocols.The platform source is public for inspection.