Rondo: Unsupervised Discovery of Recurring Temporal Structure

• Temporal streams contain reusable local units that form recurring motifs, even when instances differ in signal, duration, context, repetitions, omissions, or boundaries.This nested structure appears across domains including human motion, driving, and logistics.Source: §1 · p. 1
• Prior clustering and state-discovery, motif, and hierarchical methods typically represent temporal scales in isolation, which can fragment related patterns with different unit arrangements.The cited approaches include clustering and state discovery (Hallac et al., 2017), motif discovery (Minnen et al., 2006), and representation-learning or hierarchical segmentation (Liu & Chen, 2024; Kipf et al., 2019).Source: §1 · p. 2
• RONDO combines UnitAlign for reusable-unit discovery with MotifFormer for modeling higher-level recurring compositions.UnitAlign refines matched units or adds recurring novel ones; MotifFormer then updates existing motifs or adds compositions based on the updated unit sequence.Source: §1 · p. 2
• RONDO improves recurring-structure recovery across diverse datasets, with particularly strong gains when data are initially limited and later expanded.Ablations attribute improvements to both unit-level adaptation and motif modeling, and experiments find gains across different temporal representation backbones.Source: §1 · p. 2
• RONDO continually refines and expands unit and motif vocabularies as new recurring structure emerges in evolving streams.Learned unit vocabularies can also transfer across records to support motif discovery in new streams without relearning local structure.Source: §1 · pp. 2–3

2.1 UNSUPERVISED TEMPORAL STRUCTURE DISCOVERY

• Classical motif methods find repeated subsequences in raw data, but treating motifs independently can fragment related patterns that vary in duration, context, or composition.These methods extend to multidimensional, variable-length, and streaming settings.Source: §2.1 · p. 3
• State, segmentation, and contextual-representation methods capture richer temporal context, yet typically use flat states or segments that obscure reuse across temporal scales.Examples include TICC, Time2State, and E2USD (Hallac et al., 2017; Lai et al., 2024), unsupervised action segmentation (Kumar et al., 2022), and TNC, TS2Vec, and TimesURL (Tonekaboni et al., 2021; Liu & Chen, 2024).Source: §2.1 · p. 3

2.2 HIERARCHICAL AND COMPOSITIONAL TEMPORAL MODELING

• Hierarchical approaches compose longer patterns from reusable shorter structures, but existing unsupervised methods largely learn from fixed datasets or specific domains.Prior work groups lower-level units or reuses local patterns, while Rondo extends hierarchical discovery to continual streams where both local and higher-level structure can evolve (Gong et al., 2023; Spurio et al., 2025).Source: §2.2 · p. 3

2.3 CONTINUAL AND TRANSFERABLE STRUCTURE DISCOVERY

• Stream clustering, online motif discovery, and continual category discovery adapt to incoming data, but generally model novelty at one level rather than across compositional structure.Novelty in compositional streams can involve known motifs, new compositions of known units, or new units; Rondo maintains separate unit- and motif-level vocabularies to update and transfer recurring structure (Vaze et al., 2022; Park et al., 2024; Ma et al., 2024; Dai & Chauhan, 2025; Spurio et al., 2026).Source: §2.3 · pp. 3–4

3 METHODOLOGY

• Rondo combines UnitAlign, which builds a shared vocabulary of recurring local patterns, with MotifFormer, which discovers higher-level motifs from their unit sequence.Both vocabularies are progressively updated as new stream data arrive.Source: §3 · p. 4

3.1 PROBLEM FORMULATION

• The problem is to discover unlabeled recurring patterns with unknown boundaries and extent, including longer patterns composed from reusable units across temporal scales.Rondo represents the stream with sliding windows mapped to unit classes, then represents motifs as variable-length compositions of those units.Source: §3.1 · pp. 4–5

3.2 DISCOVERING REUSABLE TEMPORAL UNITS

• UnitAlign adapts self-supervised window representations and clusters them into reusable units, then smooths assignments to produce a stable unit sequence for MotifFormer.A consistency objective preserves similarity structure while clustering pseudo-labels encourage separability; adaptation and clustering alternate until assignments stabilize.Source: §3.2 · p. 5

3.3 DISCOVERING COMPOSITIONAL MOTIFS

• MotifFormer encodes variable-length unit subsequences with a Transformer and clusters them at each temporal scale to identify recurring motifs.Consistency training aligns subsequences across lengths and perturbations, while clustering pseudo-labels promote separability; a long-to-short soft coverage filter reduces redundant rediscovery before evidence across scales sets motif boundaries and assignments.Source: §3.3 · p. 5

3.4 CONTINUAL VOCABULARY MAINTENANCE

• For continual vocabulary maintenance, uncertain new windows and subsequences enter separate novelty buffers until sufficient evidence supports refining an existing entry or adding a persistent new unit or motif.Dynamically maintained micro-clusters summarize and adapt the current vocabulary structure.Source: §3.4 · p. 6

4.1 EXPERIMENTAL SETUP

• The evaluation covers six multivariate datasets and compares RONDO with unsupervised temporal discovery baselines, adding DenStream and DBSTREAM for continual discovery.Recordings are treated as independent sequences, and methods use the same recordings and common temporal axis while retaining their original learning and inference procedures where applicable.Source: §4.1 · p. 6

4.2 EVALUATION PROTOCOL AND METRICS

• Evaluation maps each discovered cluster to its duration-weighted overlap class and reports discovery, temporal-consistency, segmentation, and continual-adaptation metrics.The main text reports MoF, Recall, Edit, F1@50, and SegRatio; continual discovery also uses H-score and forgetting Mf.Source: §4.2 · p. 6

4.3 RECURRING STRUCTURE DISCOVERY

• RONDO achieves the highest Recall on all six datasets and the highest MoF on five, while segment-level results are mixed and PAMAP2 shows over-segmentation.It slightly trails AutoPlait in MoF on MoCap; RONDO otherwise lies between baselines that tend to produce fewer, longer segments and those that produce many short segments.Source: Table 1 · p. 7, §4.3 · p. 7

4.4 ADAPTATION AND TRANSFER ACROSS STREAMS

• On progressively revealed OPPORTUNITY and PAMAP2 streams, RONDO remains competitive in frame-level discovery while producing stronger segment-level quality and less fragmentation than streaming baselines.Its H-scores strengthen at later stream stages as unit and motif vocabularies are refined and expanded while previously discovered structure is retained.Source: Table 2 · p. 8, §4.4 · p. 8
• In cross-record transfer on LARa and Drive&Act, RONDO outperforms Time2State and E2USD, while RONDO-fit further improves MoF and Recall on both datasets.RONDO-fit transfers the learned unit vocabulary and adapts motifs on the target record, allowing personalization without relearning the entire representation.Source: Table 3 · p. 8, §4.4 · p. 8

4.5 UNDERSTANDING RONDO

• The full model achieves the highest MoF across four datasets; removing MotifFormer degrades MoF across datasets, while removing UnitAlign particularly reduces discovery quality on LARa and Drive&Act.Removing MotifFormer also substantially lowers segment-level performance on PAMAP2 and Drive&Act; UnitAlign’s effects are less uniform for F1@50 and Edit.Source: Table 4 · p. 9, §4.5 · p. 9
• RONDO’s performance remains generally stable across TS2Vec, Time-MAE, and MantisV2 representations, though the strongest encoder varies by dataset.TS2Vec achieves the highest MoF on OPPORTUNITY and PAMAP2, while Time-MAE performs best on Drive&Act and LARa and improves several segment-level metrics.Source: Table 5 · p. 9, §4.5 · p. 9

5 CONCLUSION

• RONDO models reusable local units and higher-level compositions, continually updating its vocabularies and improving discovery and temporal segmentation over existing unsupervised methods.The conclusion frames recurrence as compositional and evolving structure, with richer temporal hierarchies as future work.Source: §5 · pp. 9–10

AI USE STATEMENT

• Generative AI assisted with dataset preprocessing, baseline-experiment coding and scripts, and improving manuscript clarity.The assistance included adapting released baseline implementations, writing evaluation and scoring scripts, and running and aggregating experiments.Source: AI USE STATEMENT · p. 10
• The authors did not use generative AI to design the method or experiments, formulate hypotheses, interpret results, or generate research ideas..Source: AI USE STATEMENT · p. 10
• The authors reviewed and verified AI-assisted work, including cross-checking results, checking baseline settings, documenting baseline modifications, and editing AI-assisted text.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

• The work uses public datasets for research, involves no new data collection or human-subject studies, and reports no additional work-specific ethical concerns.The method performs unsupervised temporal structure discovery and does not infer sensitive personal attributes.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

• The paper and appendix describe preprocessing, evaluation, experimental settings, hyperparameters, metrics, implementation details, and baseline modifications; code and scripts are slated for release after review.Results across multiple random seeds are reported where applicable.Source: REPRODUCIBILITY STATEMENT · p. 10

A QUALITATIVE ANALYSIS

• RONDO generally preserves clearer temporal structure than baselines while capturing short transitions, though some long activities remain split into recurring units.Baselines can fragment segments through overly sensitive state changes or merge distinct activities through aggressive grouping.Source: A QUALITATIVE ANALYSIS · p. 14, Figure 3 · p. 14
• Across most recordings, 75% to over 90% of adjacent windows retain the same unit assignment, and units are repeatedly reused within recordings.In several datasets, individual units also align relatively strongly with activity classes, suggesting stable local structure can discriminate activities.Source: A QUALITATIVE ANALYSIS · p. 14
• Motifs improve activity consistency by a few to more than ten percentage points when units span several activities, but add marginal gains when unit purity is high.The motif layer provides longer-context disambiguation mainly when local patterns are insufficient; gains are especially limited for short actions below the modeled motif scale.Source: A QUALITATIVE ANALYSIS · p. 14

B EVALUATION METRICS

• Evaluation metrics assess discovery quality, temporal segmentation, and continual adaptation from complementary perspectives.Source: B EVALUATION METRICS · p. 15

B.1 TEMPORAL ALIGNMENT AND LABEL MAPPING

• Predictions and annotations are aligned on consecutive temporal intervals, and ground-truth labels are reserved for evaluation rather than training, clustering, or online updating.Source: §B.1 · p. 15
• Each discovered cluster maps to the ground-truth label with the greatest duration-weighted overlap, allowing multiple clusters to map to one semantic label.The mapping is estimated separately for each evaluation recording and fixed across continual stages, following prior over-clustering and action-cluster evaluation settings.Source: §B.1 · p. 15, Figure 4 · p. 15

B.2 CLASS AND SEGMENT METRICS

• MoF and discovery recall measure frame-level performance and the proportion of ground-truth class instances successfully discovered, respectively.The supplied passages introduce MoF as duration-weighted mean over frame but do not include its full computation.Source: §B.2 · p. 15
• Edit score compares ground-truth and predicted segment-label sequences using Levenshtein distance after merging consecutive intervals with the same mapped label (Lea et al., 2016).The exact score formula is not included in the supplied passages.Source: §B.2 · pp. 15–16
• Segmental F1 counts a predicted segment as a match when its mapped label and overlap satisfy the matching criterion, with one-to-one matching preventing duplicate predictions from repeated credit (Lea et al., 2017).The supplied passages do not include the overlap threshold.Source: §B.2 · p. 16

B.3 CONTINUAL DISCOVERY METRICS

• Continual evaluation measures performance after each update stage on previously known classes, newly discovered classes, and all classes.The passages denote stage-specific class-level performance with A(s).Source: §B.3 · p. 16
• The H-score measures the balance between retaining old classes and discovering new ones.The supplied passage does not include the formula.Source: §B.3 · p. 16
• Forgetting measures old-class performance degradation relative to the initial model; final forgetting is reported across stages, and lower Mf indicates better retention (Vaze et al., 2022; Park et al., 2024).The supplied passage does not include the forgetting formula.Source: §B.3 · p. 16

C.1 OVERVIEW

• The benchmark spans six datasets and diverse sensing modalities and behavioral domains, with 267 recordings and approximately 7.45 million frames overall.The datasets range from controlled human motion and daily activities to in-vehicle, sensor-rich routine, and industrial workplace activities.Source: §C.1 · p. 16, Table 6 · p. 17
• Discovery treats each recording as an independent continuous stream, with labels and class counts withheld until evaluation and segmentation metrics aggregated across recordings.Analysis windows remain within recording boundaries to avoid artificial transitions between unrelated sequences.Source: §C.1 · p. 16, §C.2 · p. 17

C.2 PREPROCESSING AND INPUT DIMENSIONALITY

• The common representation preserves multivariate recordings, annotation tracks, and recording boundaries, enabling a shared temporal organization and evaluation interface across original data formats.Source: §C.2 · p. 17
• Raw inputs range from 4 to 242 channels; Time2State and E2USD use native channels when tractable, while TICC, AutoPlait, and CLaP use shared per-dataset PCA projections for higher-dimensional data.Source: §C.2 · p. 17
• PCA choices were checked for sensitivity: PCA-reduced inputs yielded comparable results for Time2State and E2USD, and higher retained dimensionality did not meaningfully improve PCA-based methods.The reported encoder-based results therefore use native inputs, while selected projections were judged to retain sufficient information.Source: §C.2 · p. 17
• OPPORTUNITY uses a shared 54-dimensional PCA projection retaining 90% of variance; MoCap needs no reduction, and PAMAP2 uses nine ±16, g accelerometer channels from hand, chest, and ankle sensors.Table 7 summarizes the resulting preprocessing settings.Source: §C.2 · p. 17

D IMPLEMENTATION DETAILS

• Class annotations are reserved for post-hoc semantic mapping and evaluation, and hyperparameters are fixed across datasets unless noted.Preprocessing settings are specified by dataset in Table 7.Source: D IMPLEMENTATION DETAILS · p. 17, Table 7 · p. 18

D.1 TEMPORAL REPRESENTATION ENCODERS

• TS2Vec, Time-MAE, and MantisV2 encode sliding windows; their parameters stay fixed during discovery, and UnitAlign maps their different-dimensional outputs into a shared latent space.TS2Vec and Time-MAE are trained separately for each recording; MantisV2 uses a released pretrained checkpoint without target-domain fine-tuning.Source: §D.1 · p. 18

D.2 UNITALIGN IMPLEMENTATION

• UnitAlign adapts encoder representations, first preserving their geometry and limiting collapse, then alternating DeepDPM pseudo-unit clustering with adaptor refinement without semantic labels.The refinement uses discovered assignments as pseudo-labels, excludes clustering noise, and retains geometry and variance/covariance regularization.Source: §D.2 · pp. 18–19
• After assignments stabilize or refinement rounds run out, UnitAlign forms unit centroids and Viterbi-smooths assignments to discourage implausibly frequent neighboring transitions.The final DeepDPM fit defines the unit vocabulary; smoothed labels form the centroid timeline used for motif discovery.Source: §D.2 · p. 19

D.3 MOTIFFORMER IMPLEMENTATION

• MotifFormer encodes centroid subsequences with a Transformer and learns recurring compositions through corrupted views, reconstruction, contrastive alignment, and cross-length consistency.Longer motifs claim timeline intervals first; shorter candidates overlapping claimed structure receive lower sampling weight, and predictions across lengths are fused by motif, with unmatched windows assigned OTHER.Source: §D.3 · pp. 19–20

D.4 CONTINUAL UPDATE IMPLEMENTATION

• Continual updates buffer rejected windows and subsequences in separate evolving micro-clusters, then merge persistent, recurring candidates into compatible entries or promote them to new units or motifs.Micro-cluster support decays over time so transient or isolated novelty can disappear without entering either vocabulary.Source: §D.4 · p. 21

D.5 IMPLEMENTATION HYPERPARAMETERS

• The experiments’ key implementation hyperparameters are summarized in Table 8.Source: §D.5 · p. 21

E.1 OVERVIEW

• Six unsupervised segmentation and state-discovery baselines use their released procedures, while common timestamp-level outputs and evaluation code are applied across methods.Metrics are averaged across recordings except SegRatio, and results use recordings with valid outputs from every method.Source: §E.1 · p. 21

E.2 EXPERIMENTAL ADAPTATION AND COMPUTE

• Released implementations are used for AutoPlait, TICC, Time2State, E2USD, DenStream, and DBStream; TICC, CLaP, and CompILE inputs are standardized per channel within each recording.DenStream and DBStream are used only in continual-discovery experiments and retain their original online updates.Source: §E.2 · p. 21
• CLaP’s released implementation receives candidate change points from ClaSP, while the remaining implementation is used unchanged.Source: §E.2 · p. 22
• CompILE replaces discrete embeddings with linear projections and cross-entropy with squared-error reconstruction, while retaining boundary inference and its latent-variable model.Long recordings are split into 2,000-frame sequences, and segment representations are clustered with one dataset-wide Dirichlet process mixture for consistent state identities across recordings.Source: §E.2 · p. 22
• Experiments ran on a single machine with NVIDIA RTX A6000 and A40 GPUs; Time2State, E2USD, and CompILE use GPUs, while AutoPlait, TICC, and CLaP are CPU-bound.Source: §E.2 · p. 22

E.3 HYPERPARAMETER SELECTION

• No hyperparameter is selected using evaluation performance.Source: §E.3 · p. 23
• Baseline settings follow released configurations and dataset-specific settings where available; otherwise, settings remain fixed across datasets.Source: §E.3 · p. 23

• Time2State, E2USD, and RONDO are reported over five seeds; TICC’s spread reflects seed-dependent PCA and is zero on MoCap and PAMAP2, which use no PCA.AutoPlait and CLaP are each run once because they are deterministic or use a fixed random state; CompILE is run once with seed 0 because of computational cost.Source: F EXTRA RESULTS · p. 23
• Table 11 reports complete results in percent as mean ± standard deviation over five seeds; for SegRatio, values closer to 1 indicate better performance.Source: Table 11 · p. 24