Source-linked AI summary

Rhythms of Work: Multi-Scale Interpretation of Human Behavioral Traces for Workplace Agents

Lin Ai, Scott Counts

arXiv:2609.04556v1cs.CL

TL;DR

Workplace agents need to interpret the human activity surrounding them, but dense behavioral traces are difficult to reason over directly and a single summary hides its temporal grain. This paper builds separately addressable operator, motif, episode, and rhythm views of workplace telemetry. It reports that different agent-facing questions are best answered at different resolutions, with next-episode forecasting outperforming a flat-operator baseline.

  • Problem

    Workplace agents must interpret the dense human activity around them, yet raw traces are not directly interpretable and a single summary obscures the temporal horizon of its interpretation.

    Source: §1 · pp. 1–2
  • Method

    The paper constructs separately addressable operator-, motif-, episode-, and rhythm-level interpretations of human behavioral traces.

    Source: §1 · p. 2, §3 · p. 3
  • Results

    Different questions favor different temporal grains, and the full representation improves next-episode forecasting by 17% relative macro-F1 over a flat-operator baseline.

    Source: §1 · p. 2
  • Takeaways & Limitations

    Behavioral context should be stored as addressable grains an agent can retrieve by time scale rather than as one flattened stream or summary.

    Source: §6 · p. 9, §7 · p. 9

    The corpus comes from one commercial productivity suite, and the clusters describe recurring patterns rather than productivity, intent, or cause.

    Source: §8 · p. 9

Abstract

from arXiv · show

Runtime traces are becoming a central substrate for understanding agentic systems, yet interpretation has focused largely on what the agent did. Workplace agents face the complementary problem: interpreting the human activity that surrounds them. Hours of low-level events carry rich evidence about a user's state but are too granular to reason over directly, and flattening them into one stream or compressing them into a single embedding both treat "summarize the user's behavior" as if it had one correct answer. We argue instead that behavioral interpretation is resolution-dependent: the same trace should admit multiple addressable interpretations at different temporal resolutions. We construct a multi-resolution vocabulary of semantically normalized operators, recurring motifs, coherent episodes, and day-level rhythms, each preserving the structure salient at its own horizon. Applied to 667 million human-attributed events from a large commercial productivity suite (50,000 users, 100 organizations), it yields 120 operator types, thousands of motifs, 25 episode types, and five day-rhythm archetypes. We validate it on real telemetry: re-running the entire pipeline on a disjoint 2,000-user sample recovers the same taxonomy (structural stability), and on held-out users the full representation forecasts a user's next episode more accurately than a flat-operator baseline, a 17% relative macro-F1 gain (predictive validity), so the abstractions preserve future-relevant information rather than merely describe it. A controlled resolution ablation then shows that no single level is optimal across questions: different agent-facing questions about the same trace are best answered at different resolutions. Behavioral trace interpretation for agents should therefore be multi-resolution and query-conditioned: an agent should access the temporal grain a question needs, not one universal summary.

1 Introduction

Workplace agents need to interpret the human activity around them, but long, low-level traces and single summaries each obscure useful structure. The paper proposes separately addressable interpretations at multiple temporal resolutions and reports evidence of predictive validity and question-dependent utility.

1 Introduction

  • Workplace agents face a less-studied task than interpreting their own trajectories: making sense of the dense human activity surrounding them.Raw events are too long and low-level to interpret directly, while a flat stream or single summary fails to expose longer-range structure or its temporal horizon.Source: §1 · p. 1
  • Operators, motifs, episodes, and rhythms offer distinct, addressable views of the same trace, each preserving structure salient at its temporal horizon.The agent-facing question determines which grain is relevant, such as the user's current work mode or whether a recurring pattern merits action.Source: §1 · p. 2
  • The study examines structural validity, predictive validity, and whether useful interpretations vary with the question; it describes behavioral state rather than an intervention policy.Its telemetry covers users' full workdays, and deciding whether or how to help also involves latent need, interruption cost, consent, and counterfactuals.Source: §1 · p. 2
  • A multi-resolution representation improves next-episode forecasting by 17% relative macro-F1 over a flat-operator baseline, while different questions favor different grains.The paper also reports replication on a disjoint 2,000-user sample and cautions that concatenating all levels can hurt.Source: §1 · p. 2

2 Related work

Prior work examines agent trajectories, event-log structure, prediction, and proactive assistance, but leaves the human trajectory around an agent comparatively underexplored. This paper develops separately addressable interpretations of that trajectory at multiple temporal resolutions.

Interpreting agent behavior

  • The paper shifts runtime interpretation from an agent's own actions to the longitudinal human trace around it, with operator-, motif-, episode-, and rhythm-level views.Figure 1 depicts the levels together on one clock, making their temporal containment and distinct invariances visible.Source: §2 · p. 2, Figure 1 · p. 3

Behavioral trace mining and process discovery

  • Unlike trace-mining approaches that discover units such as motifs or episodes, this work argues that no single unit suffices and keeps multiple resolutions addressable.Each level has a stated invariance rather than collapsing into one flat log or a single notion of a case.Source: §2 · pp. 2–3

Predictive user modeling

  • Next-episode forecasting tests whether adding a coarser scale preserves future-relevant information, complementing prior work on next activities, actions, and applications.The paper treats this as an external-validity test with the model class held fixed, not as a forecasting system.Source: §2 · p. 3

Proactive agents and user simulation

  • The paper places representation below intervention policy: behavioral state may inform assistance, while synthetic timelines serve as instrumented fixtures rather than faithful user simulations.Prior proactive-agent work includes inferring intent, timing assistance, and evaluating with simulated users.Source: §2 · p. 3

3 A multi-resolution vocabulary for human behavioral traces

The paper builds an unsupervised vocabulary of human work traces at several temporal horizons, with each level preserving different structure. Its pipeline discovers canonical operators, episodes, motifs, day rhythms, and population variation from workplace telemetry.

3 A multi-resolution vocabulary for human behavioral traces

  • The vocabulary treats operators, motifs, episodes, and rhythms as distinct views of a trace, and the pipeline segments episodes before mining motifs.The corpus contains 667 million events from 50,000 users across 100 organizations over two months; its structure is discovered without labels.Source: §3 · pp. 3–4

3.1 Canonical operators

  • Two LLM-assisted passes map product-specific action pairs to 120 canonical operators, resolving remaining events or discarding those that appear non-human.The catalog extends a production audit-log schema with semantic actions and interface primitives; operators below a privacy threshold are pooled into Other.Source: §3.1 · p. 4

3.2 Episodes and motifs

  • Session segmentation, within-application clustering, and merging adjacent same-cluster sessions yield 25 episode types; cluster-level n-gram mining produces 3,597 motif patterns.Motifs are contiguous two- to six-token sequences that meet an adaptive support threshold, and GPT-5.4-generated labels are attached only after mining.Source: §3.2 · p. 4

3.3 Rhythms and population variation

  • Clustering daily episode matrices and scalar summaries yields five day-rhythm archetypes, while a second clustering describes ten personas as mixtures over shared rhythms and operators.The personas are presented as evidence that one vocabulary describes a heterogeneous population, not as a taxonomy the paper relies on.Source: §3.3 · p. 5

4 Do the behavioral interpretations capture real structure?

The paper tests whether its behavioral vocabulary recovers stable structure and preserves information about what users do next. It finds independent replication and better next-episode forecasting with the full representation than with flat operators.

4.1 Structural validity: independent held-out replication

  • The disjoint 2,000-user rerun recovered 92% (23/25) of episode types, with gap-density Jensen–Shannon divergence of 0.0040.A same-scale split-half had divergence 0.021, supporting stability across samples and scale.Source: §4.1 · p. 5

4.2 Predictive validity: focused next-episode forecasting

  • The forecasting test holds the linear probe fixed and predicts the next episode type from same-day history, isolating the signal added by each interpretation level.The target is a 25-way classification at each episode boundary; the final condition uses only the day-so-far prefix.Source: §4.2 · pp. 5–6
  • Adding episode structure lifted macro-F1 by 11% (absolute +0.023; 95% CI [0.022, 0.023]), while episode context helped new-mode predictions but hurt same-type repeats.Operators produced the largest single jump; gains from added scales were uneven.Source: Figure 2 · p. 6, §4.2 · p. 6
  • 17% relative macro-F1 gain over flat operators (0.199 to 0.232) on 2,975,438 held-out same-day transitions shows the full representation improves next-episode forecasting.The Markov baseline reached 0.060; the evaluation used 10,000 held-out users.Source: §4.2 · p. 6

5 Which behavioral interpretation does an agent need?

The study varies which interpretation of a behavioral trace an agent sees while holding the question and answering model fixed. Different questions favor different temporal grains, and adding more context does not consistently improve answers.

5 Which behavioral interpretation does an agent need?

  • The evaluation uses 640 questions on synthetic, instrumented fixtures with known labels, rendered in eight context shapes and answered over three runs.The fixtures are calibrated to real-stream surface statistics but remain synthetic rather than real people.Source: §5 · p. 7

5.1 Where each capability finds its answer

  • No single interpretation wins across questions: the best observable context shifts among motif, episode, rhythm, and the stacked hierarchy.The hierarchy averages 0.67 macro-F1, but the level that works best depends on the question.Source: §5.1 · p. 7, Figure 3 · p. 8
  • Motif-scale recognition favors action ordering (0.44 versus 0.30 baseline), while behavior projection is best at the episode layer (0.63 versus 0.56 for the hierarchy).Adding day rhythm can introduce a misleading prior for behavior projection.Source: §5.1 · p. 7
  • Day-scale recognition rises from 0.08 at baseline to 0.77 with rhythm, while episode-scale recognition and anomaly detection are best answered by the episode layer.The observable episode layer reaches the ceiling attained by the full all shape for those latter questions.Source: §5.1 · pp. 7–8
  • Interruption timing reaches 0.60 only with the full hierarchy, and help selection improves from 0.82 with episode context to 0.85 with the hierarchy.Interruption timing draws on recent activity density, episode boundaries, and day load.Source: §5.1 · p. 8

5.2 Why one universal summary is not enough

  • Even declared workstream and workflow metadata never beats the best behavioral layer, and the larger all shape matches or trails the hierarchy on six of seven capabilities.The oracle drops interruption timing below baseline to 0.21, the study’s lowest cell.Source: §5.2 · p. 8
  • The authors argue that levels should remain separately addressable so an agent can retrieve the grain or combination a question requires.The best-performing level can shift even within a capability, across its subclasses.Source: §5.2 · pp. 8–9

6 Discussion

Behavioral interpretation is scale-selective retrieval: different questions need different temporal grains, so no fixed context wins everywhere. The representation supplies state for interpreting human activity, not a policy for whether to intervene.

6 Discussion

  • Different questions—and even sub-classes of one question—need different grains, so no fixed context wins everywhere and concatenating all levels can plant a misleading prior.The discussion argues for storing addressable grains that agents retrieve by time scale rather than topic.Source: §6 · p. 9
  • The human trajectory surrounding an agent complements work on interpreting the agent’s own behavior, and the same multi-resolution reading can apply before, during, and after interactions.That could let an agent examine whether its action changed what the user did next.Source: §6 · p. 9
  • Forecasting future behavior and deciding whether to act are distinct: intervention also weighs latent need, interruption cost, consent, and the counterfactual.The episode increment is positive on average but negative for same-label revisits, supporting decision-gated rather than automatic retrieval of added scale.Source: §6 · p. 9
  • Open problems include learning a query-to-grain router, testing transfer beyond one productivity suite, and evaluating action-facing questions with people rather than offline probes.Source: §6 · p. 9

7 Conclusion

A long human activity trace admits multiple valid readings, and which one an agent needs depends on its question. The paper builds and validates a multi-resolution vocabulary, concluding that behavioral context should remain addressable by time scale rather than flattened into one stream or summary.

7 Conclusion

  • The vocabulary reads behavior as operators, motifs, episodes, and day-level rhythms, each representing a different temporal scale.Source: §7 · p. 9
  • On workplace telemetry, the representation is stable across independent samples and predictive of users’ next behavior, while agent-facing questions do not have one universally best grain.Source: §7 · p. 9
  • Behavioral context should be kept as separately addressable grains that an agent can query by time scale, complementing interpretation of the agent’s own trajectory.Source: §7 · p. 9

8 Limitations and responsible use

The study’s corpus, forecasting test, and synthetic ablation bound what its findings establish. It selects next-episode prediction as a multi-scale target and explains why finer or coarser alternatives are less suitable.

8 Limitations and responsible use

  • The corpus comes from one commercial productivity suite, may not transfer to other workplaces or roles, and its clusters describe recurring patterns rather than productivity, intent, or cause.Source: §8 · p. 9
  • A single forecasting target and linear probe, plus synthetic fixtures and handpicked ablation moments, bound the evidence; interruption timing and assistance preference still need human evaluation.Source: §8 · p. 9
  • The telemetry is sensitive, and deployment requires purpose limitation, data minimization, transparency, user control, and organizational safeguards.Source: §8 · p. 9

A Why next-episode is the forecasting target

  • Next-episode type is the forecasting target because it is a coherent, objectively labeled work unit that can use every interpretation level; next action is too low-level, next motif is too close to motif construction, and next day-rhythm leaks future behavior.Source: A Why next-episode is the forecasting… · p. 12

B Synthetic test-fixture construction

The study constructs synthetic timelines with known labels to compare context shapes and test workflow validity. The fixtures are calibrated to real behavioral patterns, then evaluated for plausibility, execution, and answers to diagnostic questions.

B Synthetic test-fixture construction

  • Synthetic timelines reproduce the content-agnostic operator-and-timing structure, with latent labels fixed by construction and surface statistics calibrated to real data.The six-stage pipeline moves from source task material through workflows, workstreams, and episode specifications to matched real episode clusters and timestamped action sequences.Source: B Synthetic test-fixture construction · p. 12
  • Fixture checks rated workflow and episode coherence 4.5/5, behavioral fidelity 3.3/5, and interaction-gap divergence from the real gap model 0.26.The intended claim is plausibility with known labels, not fidelity to any specific person.Source: B Synthetic test-fixture construction · p. 12
  • 85% of workflows were executable in free mode and 80% in guided mode, with completeness of at least 7 defining success.Source: B Synthetic test-fixture construction · p. 12
  • The evaluation sampled diagnostic moments across 11 persona timelines for 640 questions rendered in eight shapes, with ground-truth answers fixed by construction and macro-F1 used to score recovery.The full design comprises 15,360 evaluations across three runs.Source: B Synthetic test-fixture construction · p. 13
  • Table 2 contrasts six observable context shapes, which add layers a deployed agent could compute, with oracle and all shapes, which add declared metadata as reference points.Source: Table 2 · p. 13
  • Table 3 organizes seven capabilities by family and reports sample counts for their sub-classes before the context-shape and run multipliers.Source: Table 3 · p. 13

C Inside each family: per-class analysis

Per-class results show that the most useful context grain varies within and across capabilities. The same question is tested across context shapes, and no single layer suffices for perception, inference, and action.

  • The seven probes use fixed multiple-choice questions and answer classes, so each context shape is evaluated against the same ground truth.The probe labels correspond to Figure 3 rows, with per-class sample counts reported in Table 3.Source: Table 4 · p. 14
  • Both anomaly-detection sub-classes peak at the episode grain, while behavior projection also favors episodes and can be hurt by a misleading day-rhythm prior.For anomaly detection, the observable episode context already matches the all ceiling; oracle workstream titles add nothing.Source: C Inside each family: per-class analysis · p. 14
  • Help selection favors abstracted context: motifs work best for drafting and scheduling, while the hierarchy works best for analysis, entry, and polish.Interruption timing shows a small, broad hierarchy lift, but no single context shape dominates every timing class.Source: C Inside each family: per-class analysis · p. 14
  • Perception varies by sub-class: day recognition favors rhythm, while motif recognition’s best context spans baseline, raw, motif, and rhythm.The per-class pattern means a capability-level score can conceal which grain best serves an individual question.Source: C Inside each family: per-class analysis · p. 13, Figure 4 · p. 15

D Predictive-validity sensitivity analyses

The sensitivity analyses show that episode context helps some next-episode predictions but can hurt same-label revisits. The accompanying figures depict independent structural replication and the discovered day rhythms and personas.

D Predictive-validity sensitivity analyses

  • The episode increment is +0.035 for state changes (+20% relative to that stratum’s S-C2 score) and +0.032 for 0–10 s gaps, but −0.098 (−27%) for same-label revisits.The increment is stable across five user-persona strata (+0.017 to +0.025) and essentially unchanged when always-on service accounts are excluded (+0.022); these descriptive strata were not part of the pre-registered primary comparison.Source: D Predictive-validity sensitivity… · p. 15, Table 5 · p. 15

E Structural-validity figure

  • Figure 7 visualizes independent replication by comparing log-gap densities and per-application episode-cluster matches between the disjoint 2,000-user sample and the 50,000-user run.Source: E Structural-validity figure · p. 16

F Day rhythms and personas

  • The full corpus yields five day-rhythm archetypes and ten personas, illustrating that the multi-resolution vocabulary describes a heterogeneous population.The personas are longitudinal mixtures over the same five rhythms and are not a taxonomy the paper’s claims depend on.Source: F Day rhythms and personas · p. 16

G Compute resources

The reported pipeline relies primarily on hosted LLM inference, while its non-LLM computation runs inexpensively on a commodity CPU workstation. The assets and software are used under their stated licenses or terms.

G Compute resources

  • Hosted GPT-5.4 inference is the dominant reported cost; the resolution ablation uses 15,360 answering calls, while classical computation is inexpensive on a commodity multi-core CPU workstation.The two-pass operator canonicalization labels roughly 14,000 distinct app-action pairs, and the one-time bottom-up mining is the largest non-LLM step.Source: G Compute resources · p. 16

H Assets and licenses

  • OdysseyBench is used under its MIT License, GPT-5.4 through a hosted commercial API under its terms, and the listed ML libraries under BSD-3-Clause licenses.The paper states that each asset is used in accordance with its license or terms.Source: H Assets and licenses · p. 16
Loading 2609.04556v1…