PaperlayerView exact revision on arXiv

Source-linked AI summary

HiPLEX: Hierarchical Policy Factorization for Full Duplex Speech Language Models

Kyudan Jung, Hyunsin Park, Yoonhyung Lee, Jinhwan Park, Jinhyeok Yang, KiHyun Nam, Jaegul Choo, Jinkyu Lee

arXiv:2610.07727v1cs.SD

TL;DR

Full-duplex speech RL must coordinate when to speak with what to say, while existing approaches leave timing credit and content quality unresolved. HiPLEX separates per-frame control from conditional content selection and routes timing and semantic feedback to those factors; it improves restraint and response timing on the reported benchmarks.

  • Problem

    Full-duplex speech RL must coordinate timing and content, but existing methods leave their joint improvement unresolved.

    Source: §1 · p. 1

  • Method

    HiPLEX factorizes a pretrained text policy into per-frame control and conditional content policies, routing timing credit through event-causal masks and semantic feedback to content decisions.

    Source: §1 · p. 2

  • Results

    HiPLEX gives the strongest overall trade-off on Full-Duplex-Bench v1 for Moshi and improves restraint axes on PersonaPlex; on Moshi, judged response quality remains comparable to GRPO.

    Source: §1 · p. 2, §5.1 · p. 7

  • Takeaways & Limitations

    The reported results support separating timing and semantic feedback while using event-causal credit to shape turn-taking and interruption behavior.

    Source: §6 · p. 9

Abstract

from arXiv · show

As human--AI interactions become more conversational, full-duplex speech language models capable of natural real-time dialogue are growing in importance. Beyond generating appropriate responses, these models must coordinate turn-taking, backchanneling, and floor management in real time. Reinforcement learning (RL) provides a way to refine these behaviors through direct feedback on interaction outcomes. However, existing RL methods either apply timing feedback to a token policy or optimize semantic content, leaving the joint improvement of timing and content unresolved. We introduce HiPLEX, an RL framework that factorizes a pretrained full-duplex text policy into a control policy that decides when to emit content and a conditional content policy that decides what to emit. The first factor selects among 'pad', 'epad', and 'con'. The second selects a token only when 'con' is chosen. This hierarchy describes conditional actions within each frame and uses the model's existing text head. We route timing advantages to the token-group factor through event-causal masks derived from generated speech episodes, and route an LLM-judge semantic advantage to the conditional content factor. Across three Moshi seeds on Full-Duplex-Bench v1, HiPLEX reduces takeover rates during natural user pauses and backchannel opportunities, and shortens post-interruption response latency relative to GRPO, while maintaining comparable judged interruption-response quality. On Moshi and PersonaPlex, HiPLEX better matches pooled human turn-timing and backchannel-rate marginals than GRPO.

1 INTRODUCTION

Full-duplex speech models must coordinate what to say with when to speak, but existing RL approaches leave timing credit and content quality insufficiently coordinated. HiPLEX separates these decisions and routes feedback according to their roles.

1 INTRODUCTION

  • Existing approaches leave two gaps: fixed windows can credit the wrong frames, and timing-only RL can improve timing while degrading judged quality.The paper argues content need not be excluded from RL to protect it.Source: §1 · p. 1
  • HiPLEX factorizes the pretrained text policy into per-frame control over whether and when to emit, and conditional content selection when emission is chosen.Both decisions come from the existing text head and require no new parameters.Source: §1 · p. 2
  • Sign-dependent event-causal masks assign timing credit to decisions tied to generated speech episodes, rather than applying one advantage across a fixed window.Late-response penalties target preceding waiting decisions, while timely responses credit their onset.Source: §1 · p. 2
  • HiPLEX gives the strongest overall trade-off on Full-Duplex-Bench v1 for Moshi and improves restraint axes on PersonaPlex.The paper also reports a post-boundary word-activity diagnostic for mixed PersonaPlex turn results.Source: §1 · p. 2

2 RELATED WORK

Related work spans full-duplex speech RL and hierarchical credit assignment; HiPLEX adapts structured credit to streaming speech with conditional, per-frame actions.

2 RELATED WORK

  • Full-duplex systems target simultaneous listening and speaking, backchannels, and interruption handling; Moshi provides an inner-monologue text stream with streaming audio, while PersonaPlex adds role and voice control.The cited RL approaches include raw-token training, four-faceted GRPO, and timing-only GRPO within fixed event windows.Source: §2 · p. 2
  • Unlike option-based hierarchies, HiPLEX uses a conditional pad/epad/cont decision to determine whether a low-level content action occurs at that frame.It uses the pretrained text head without an added manager and replaces learned segment values with audio-grounded event masks.Source: §2 · p. 2
  • Figure 1 depicts semantic advantages routed to content decisions and timing advantages routed to control decisions through event-causal masks.Each cell represents an 80-ms frame.Source: Figure 1 · p. 3

3 METHOD

HiPLEX trains a full-duplex speech model by separating per-frame emission control from conditional content selection, then assigning timing and semantic feedback to the corresponding policy factors.

3.1 PROBLEM SETUP

  • The setup uses 80-ms frames conditioned on incoming audio, past model audio and text, and optional role or voice inputs; interaction windows cover turn-taking, pauses, backchannels, and interruptions.Generated speech spans are merged across gaps of at most 1 s into episodes, with episodes of at most 1 s treated as short.Source: §3.1 · p. 3

3.2 CONTROL AND CONTENT POLICY FACTORIZATION

  • The control policy selects among pad, epad, and cont at each frame, while the conditional content policy selects a token only when cont is selected.The factors reconstruct the temperature-scaled text distribution using the existing head and no new parameters.Source: §3.2 · pp. 3–4

3.3 REWARDS AND CREDIT ASSIGNMENT

  • Timing rewards score turn onset, delay, and early speech; pause intrusion and speech fraction; backchannel matching and errors; and interruption yielding and re-entry.Only reward components for the sampled interaction type are used.Source: §3.3 · pp. 4–5
  • Event-causal masks select control decisions using generated speech episodes, event timing, and advantage sign, targeting waiting, speech, yielding, or re-entry decisions as relevant.For turn taking, a late-response penalty targets preceding waiting without suppressing the eventual onset.Source: §3.3 · pp. 5–6
  • An ASR-plus-LLM judge scores response relevance and coherence, and its group-standardized semantic advantage applies only to content decisions where a content token was generated.Timing components are normalized separately within rollout groups.Source: §3.3 · pp. 4, 6

3.4 OBJECTIVE AND LEARNING RATE

  • PPO-style clipped losses update selected decisions for each factor, with separate normalization and reference-model regularization.A factor contributes no policy loss when it has no selected decisions.Source: §3.4 · p. 6
  • Event-causal masking selects about 20–30% of the frames used by a flat update, so the authors tune control and content learning rates separately.For Moshi, validation selects 4 × 10^-6 for HiPLEX and 1 × 10^-6 for flat GRPO; PersonaPlex uses 2 × 10^-6 for HiPLEX.Source: §3.4 · p. 6

4 EXPERIMENTAL SETUP

The experiments compare HiPLEX with a reproduced GRPO baseline on Moshi and PersonaPlex using interaction windows and multiple behavior-focused evaluation axes.

4 EXPERIMENTAL SETUP

  • Training uses fixed Moshi and PersonaPlex backbones and 73,561 approximately balanced annotated windows from four interaction behaviors.The windows come from conversation-only subsets of the Seamless-Interaction corpus.Source: §4 · pp. 6–7
  • Table 1 reports Full-Duplex-Bench v1 results, with trained Moshi rows for seed-42 terminal checkpoints and three-seed results reported separately.Its lower block contains calibration references, not training baselines.Source: Table 1 · p. 7
  • The primary comparison is HiPLEX versus a reproduced GRPO baseline using the same pretrained backbone, alongside the pretrained base model.Published results and released-checkpoint evaluations are also reported as calibration references.Source: §4 · p. 7
  • Ablations vary semantic reward and timing-credit rules, including all-frame, fixed-window, matched-sparsity random, and event-causal masking.Further experiments examine judge scale and stronger semantic feedback.Source: §4 · p. 7
  • Full-Duplex-Bench v1 evaluates pause silence, backchanneling, turn taking, and interruption response; judged response quality uses frozen ASR transcripts.The protocol uses real-time playback from t=0 without silence modification and reports multiple axes because no single metric captures all behaviors.Source: §4 · p. 7

5 RESULTS

HiPLEX improves several turn-taking and interruption measures over GRPO, though PersonaPlex’s official smooth-turn result and response latency are mixed. The results also show that training stability, semantic feedback, and timing-credit eligibility affect the balance between responsiveness and restraint.

5.1 MAIN COMPARISON

  • Across three Moshi seeds, HiPLEX reduces CANDOR pause and backchannel takeover rates and post-interruption response latency versus GRPO, with comparable mean judged quality; Figure 3’s displayed checkpoint also combines stronger pause restraint with higher judged interruption quality.Figure 3(a) uses post-boundary word activity, which may include speech initiated before the annotated turn end.Source: §5.1 · pp. 7–8, Figure 3 · p. 7
  • On PersonaPlex, HiPLEX reduces pause and backchannel takeovers but has lower official smooth-turn TOR and longer post-interruption latency; diagnostics show more post-boundary word activity and less pre-boundary speech.Post-boundary activity does not establish a higher rate of newly initiated responses.Source: §5.1 · p. 8

5.2 THE SAME RECIPE NEEDS A DIFFERENT STEP SIZE ON THE TWO FAMILIES

  • At Moshi’s 4 × 10−6 learning rate, PersonaPlex training collapses toward near-silence; halving the rate only delays the outcome, while rebalancing rewards toward initiation keeps both rewards healthy.The authors attribute the narrower optimization margin in part to PersonaPlex’s more talkative, floor-holding prompt prior.Source: §5.2 · p. 8, Figure 4 · p. 8

5.3 MATCHING HUMAN TIMING, NOT JUST PASSING A RULE

  • On Moshi, HiPLEX is closest to pooled human marginals for turn timing, backchannel rate, and backchannel length; PersonaPlex is closer on turn timing and backchannel rate, but not the other two.The comparison uses Wasserstein-1 distance, where lower values are closer.Source: §5.3 · p. 9, Table 2 · p. 9

5.4 EFFECT OF SEMANTIC FEEDBACK

  • Adding semantic feedback substantially improves HiPLEX’s judged interruption-response quality and also reduces pause and backchannel takeovers and shortens post-interruption latency.GRPO shows only a small judged-quality gain from the added reward.Source: §5.4 · p. 9, Table 3 · p. 9

5.5 WHY EVENT-CAUSAL MASKS MATTER

  • With factorization and rewards fixed, event-causal timing credit reaches official turn TOR of 1.0 while remaining competitive on pause handling, post-interruption latency, and quality; all-frame credit increases latency, while matched-random sparsity answers only 6.7% under the official turn-taking rule.A delayed-response penalty targets the waiting decisions before speech while leaving the eventual onset untouched by that component.Source: §5.5 · p. 9

6 CONCLUSION

HiPLEX frames full-duplex speech RL as credit assignment over when to speak and what to say. Compared with reproduced GRPO, it improves restraint in both model families and interruption latency on Moshi, while the conclusion emphasizes evaluating response activity and human timing as well as event-level success.

6 CONCLUSION

  • HiPLEX separates per-frame emission control from conditional content selection and uses event-causal masks to assign timing feedback to decisions tied to generated speech episodes.The conclusion describes the approach as credit assignment over when to speak and what to say.Source: §6 · p. 9
  • Compared with reproduced GRPO, HiPLEX improves pause and backchannel restraint in both model families and shortens post-interruption response latency on Moshi.The conclusion also notes that semantic feedback can improve judged response quality alongside timing.Source: §6 · p. 9
  • The conclusion cautions that event-level success can obscure early speech and excessive silence, so evaluation also needs response activity and comparisons with human timing distributions.The credit-routing ablation’s random-masking variant answered only 0.067 under the official turn-taking rule, indicating silence.Source: §6 · p. 9, Table 4 · p. 9

7 ETHICS STATEMENT

The passages place HIPLEX within work on full-duplex dialogue and language-policy optimization, then describe its event-based credit routing and separate timing rewards. They also state the data-handling boundary for the training corpus.

7 ETHICS STATEMENT

  • The training corpus uses consented dyadic recordings and interaction annotations, excludes speaker identification, and releases no raw participant audio.Source: §7 · p. 10

A EXTENDED RELATED WORK

  • Full-duplex systems generate and listen on the same clock, and HIPLEX changes RL credit assignment within a Moshi-style architecture rather than changing the architecture.The related-work discussion distinguishes full-duplex systems from single-sequence speech-and-text systems and positions HIPLEX as orthogonal to architecture.Source: A EXTENDED RELATED WORK · p. 14
  • Language-policy methods commonly apply one advantage across token log-probabilities, while HIPLEX breaks that flat-trajectory assumption through hierarchical credit assignment.Group-relative optimization standardizes rewards across a sampled group and is the estimator HIPLEX builds on.Source: A EXTENDED RELATED WORK · p. 14

B ALGORITHM

  • HIPLEX routes timing advantages to control decisions using event-causal masks derived from generated speech episodes, while routing the semantic advantage to content decisions.The training loop grounds episodes in decoded audio and distinguishes successful onset, late onset, intrusion, and correct silence cases.Source: B ALGORITHM · p. 15
  • Onset and delay rewards are separately normalized before being combined, so a raw-sum curve would not represent the optimized signal.The onset score retains a graded tail for late valid responses, while the delay score declines linearly to −1 at the window boundary.Source: B ALGORITHM · p. 15, Figure 6 · p. 16
  • A timely onset credits its onset decision, whereas a late response penalizes preceding waiting decisions rather than the eventual onset.The onset example selects the first 80 ms onset frame; the late-turn example selects the pre-onset control interval.Source: B ALGORITHM · p. 17

C VERIFYING THE EXACT POLICY FACTORIZATION

This appendix examines HiPLEX’s policy factorization and how timing rewards and event-causal masks assign credit across interaction events. It also specifies a narrowly gated escape for degenerate sparse-event groups and describes an ablation of temporal eligibility for control credit.

C VERIFYING THE EXACT POLICY FACTORIZATION

  • Median reconstruction error was zero for all three input dtypes; maximum ε was 7.6 × 10^-6 (float32) and 3.8 × 10^-6 (bfloat16 and float16).The factorization preserves the token distribution in real arithmetic, with numerical checks agreeing up to floating-point roundoff.Source: C VERIFYING THE EXACT POLICY… · pp. 17–18
  • The KL chain rule reconstructs token KL within 5.8 × 10^-7, but separately weighted factor penalties and clipped objectives are not generally equivalent to token-level KL or GRPO.For content tokens, control and content score gradients sum to the full token gradient; the same score-function estimator requires shared fixed advantages, equal weights, identical eligibility, and common normalization.Source: C VERIFYING THE EXACT POLICY… · p. 18

D REWARD COMPONENTS: DEFINITIONS, WEIGHTS, AND EDGE CASES

  • Timing components are standardized separately across rollouts for each window, weighted after standardization, and routed to control; the separately normalized semantic score is routed to content.A sparse-failure exception replaces selected component advantages with −1.5 for onset or miss and −1.0 for re-onset when the activation conditions hold.Source: D REWARD COMPONENTS: DEFINITIONS,… · pp. 19, 21
  • Turn-taking scores reward response onset and delay while penalizing early starts; pause handling penalizes sustained intrusion and speech fraction over the sampled pause window.Turn-taking uses onset, delay, and early-start components; the pause intrusion and speech-fraction scores cover the sampled pause-handling window.Source: D REWARD COMPONENTS: DEFINITIONS,… · p. 20
  • Backchannel rewards include F1, timing, miss, false-alarm, and overlong components, while interruption rewards cover yielding and post-interruption re-entry onset and delay.Backchannel opportunities are matched to short episodes, while interruption re-entry is anchored at the end of the barge-in.Source: D REWARD COMPONENTS: DEFINITIONS,… · pp. 20–21

E EVENT-CAUSAL MASKS IN DETAIL

  • Event-causal masks select realized frames of action types that can directly produce the target event, rather than assigning credit by temporal proximity alone.Episodes and masks are derived from generated speech and ground-truth event timing; the term “causal” means structural eligibility for policy-gradient credit, not a learned causal graph or counterfactual estimator.Source: E EVENT-CAUSAL MASKS IN DETAIL · pp. 21–22
  • For turn-taking, positive onset or delay advantages select the response onset, while non-positive advantages select the preceding wait; pause masks select generated speech or, for silent rollouts, ground-truth pause windows.Missing responses select the post-anchor interval, with a boundary-and-waiting rule when speech crosses the anchor.Source: E EVENT-CAUSAL MASKS IN DETAIL · p. 22
  • Backchannel masks use expanded opportunity windows and generated episodes, while interruption masks select overlap during the barge-in or apply turn-taking rules at its end.Overlong masks target speech after its first second, or a local stop interval after a short episode when no sustained episode occurs.Source: E EVENT-CAUSAL MASKS IN DETAIL · pp. 22–23
  • The degenerate-group escape replaces an otherwise-zero advantage with bounded −sk only for named components when rewards are finite, nearly identical (maximum deviation ≤ 10−8), and all ≤ −0.05.The replacement uses each component’s existing event-causal mask; backchannels are the most common sparse-event example, and other components retain ordinary group-relative normalization.Source: E EVENT-CAUSAL MASKS IN DETAIL · p. 23
  • The all-frame ablation row isolates the hierarchy and level-specific routing from event masking, with factorized rows sharing the reported Moshi configuration and 100-epoch endpoint.Rows differ in the temporal eligibility of control credit.Source: E EVENT-CAUSAL MASKS IN DETAIL · p. 23
  • The matched-random ablation controls for sparsity without aligning control credit to events.It is one of the Table 6 routing-ablation conditions.Source: E EVENT-CAUSAL MASKS IN DETAIL · p. 23

F TRAINING DATA: SUBSET SELECTION AND PREPROCESSING

The appendix describes how conversation windows were selected and split, the reported training configurations, and the separate protocols for semantic training rewards and response-quality evaluation.

F TRAINING DATA: SUBSET SELECTION AND PREPROCESSING

  • The source corpus is filtered to naturalistic and improvised conversation interactions, using audio and interaction annotations while discarding action-oriented types and video archives.The selected subset is the ipc conversation interaction type.Source: F TRAINING DATA: SUBSET SELECTION AND… · pp. 23–24
  • The training pool contains approximately 22.1k two-channel interactions and roughly 1.6k hours of dyadic conversation, split at the conversation level before window extraction.The split prevents speaker pairs or local context from appearing on both sides.Source: F TRAINING DATA: SUBSET SELECTION AND… · p. 24
  • The extracted manifest contains 73,561 training windows balanced across four axes and 2,048 validation windows, with strictly causal policy input.Future annotations are used only for window selection and reward computation.Source: F TRAINING DATA: SUBSET SELECTION AND… · p. 24

G FULL TRAINING CONFIGURATION

  • The two reported HIPLEX configurations share the recipe except for final-block adjustments, including a halved step size and a reward balance tilted toward initiation for PersonaPlex.The appendix attributes these adjustments to stability analysis and the prompt-conditioned prior; AdamW and a cosine learning-rate schedule are used.Source: G FULL TRAINING CONFIGURATION · pp. 24–25, Table 8 · p. 25

H SEMANTIC-REWARD AND EVALUATION-JUDGE PROTOCOL

  • The training semantic reward transcribes user context and generated audio, assigns a 0–2 rubric score, and standardizes it against rollouts for the same context.Empty, irrelevant, or unintelligible responses receive score 0 rather than being filtered from the rollout group.Source: H SEMANTIC-REWARD AND EVALUATION-JUDGE… · p. 25
  • Interruption-response quality is judged separately on a 0–5 relevance scale, only for clips with a qualifying agent turn, with takeover rate reported alongside it.The evaluation judge and rubric are separate from the training reward; the reported Moshi checkpoints have 192 HIPLEX and 191 GRPO scored responses out of 200 clips.Source: H SEMANTIC-REWARD AND EVALUATION-JUDGE… · p. 26

I SENSITIVITY OF SYNTHETIC PAUSE EVALUATION TO THE STREAMING PROTOCOL

Synthetic-pause scores are sensitive to streaming settings and clip construction, while natural CANDOR clips offer a complementary pause assessment with more variation across models.

I SENSITIVITY OF SYNTHETIC PAUSE EVALUATION TO THE STREAMING PROTOCOL

  • Under the reproduced protocol, base Moshi scores 0.416 versus the published 0.445, while Moshi +RL scores 1.000 versus the reported 0.307.The similar reproduced scores suggest limited discrimination under that protocol.Source: I SENSITIVITY OF SYNTHETIC PAUSE… · p. 27, Table 9 · p. 28
  • Synthetic clips may conflate mid-utterance pause handling with conversation-start behavior because the inserted hesitation occurs near the start of a short clip.With silence warm-up removed, base Moshi greets and scores 0.985, and that greeting is counted as a takeover.Source: I SENSITIVITY OF SYNTHETIC PAUSE… · p. 27
  • The tested serving settings vary silence warm-up and greeting exclusion, and the resulting scores suggest sensitivity to greeting treatment without identifying which settings produced published results.The base model is closer to its published score without greeting exclusion, whereas +RL is closer with it.Source: I SENSITIVITY OF SYNTHETIC PAUSE… · pp. 27–28, Table 9 · p. 28
  • Natural CANDOR clips show lower pause takeover rates for HIPLEX than for the base and reproduced GRPO models.Drawn from the middle of long conversations, they reduce greeting influence and preserve natural continuation cues.Source: I SENSITIVITY OF SYNTHETIC PAUSE… · p. 28

J OFFICIAL TURN METRIC AND POST-BOUNDARY WORD ACTIVITY

The official smooth-turn metric can count speech before a user finishes, limiting its ability to distinguish turn-boundary behavior. A post-boundary word-activity diagnostic offers a complementary measure but does not establish that a new response began.

J OFFICIAL TURN METRIC AND POST-BOUNDARY WORD ACTIVITY

  • The official score counts a sample as a successful turn when any speech span reaches one second or exceeds three words anywhere in the clip, and clamps reported latency at zero.In CANDOR clips, user turns end roughly halfway through a ten-second excerpt and early speech is frequent, so takeover rates and latency have limited discrimination in the reproduced protocol.Source: J OFFICIAL TURN METRIC AND… · p. 28
  • The complementary diagnostic transcribes the same 119 outputs and applies an elapsed-time-or-word-count criterion to ASR word intervals starting at or after the annotated user turn end.It reports mean post-boundary word activity and, for qualifying clips, the median offset to the first post-boundary word.Source: J OFFICIAL TURN METRIC AND… · p. 28
  • Post-boundary activity can include continuation of earlier speech and elapsed time between words, so it is not interpreted alone as the rate of newly initiated responses.The criterion does not require a new speech episode to begin after the boundary.Source: J OFFICIAL TURN METRIC AND… · p. 28

K EVALUATION SAMPLE SIZES, PAIRED UNCERTAINTY, AND TRAINING SEEDS

Across the evaluated Moshi checkpoints, HIPLEX improves both restraint metrics relative to GRPO, with paired clip-bootstrap intervals favoring HIPLEX. The three-seed replication supports the timing result, while a GRPO learning-rate sweep shows metric-dependent trade-offs.

K EVALUATION SAMPLE SIZES, PAIRED UNCERTAINTY, AND TRAINING SEEDS

  • The evaluation contains 137 synthetic-pause, 216 CANDOR-pause, 55 backchannel, 119 smooth-turn, and 200 interruption clips; judged interruption quality covers 192 HIPLEX and 191 GRPO responses.Systems in paired comparisons use the same clip identifiers.Source: K EVALUATION SAMPLE SIZES, PAIRED… · p. 28
  • The post-boundary table measures pre-turn-end speech separately from qualifying post-boundary word activity, which may include continuation of earlier speech.Its reported offset is the median time to the first post-boundary word among qualifying clips.Source: Table 10 · p. 29
  • Paired bootstrap resampling of matched clip identifiers over 10,000 iterations estimates clip-sampling uncertainty for fixed 100-epoch checkpoints, not variation across training runs.The resampling uses bootstrap seed 0.Source: K EVALUATION SAMPLE SIZES, PAIRED… · p. 29
  • CANDOR pause TOR was 0.306 for HIPLEX versus 0.454 for GRPO (difference −0.148, 95% CI [−0.222, −0.074]); Backchannel TOR was 0.127 versus 0.309 (difference −0.182, 95% CI [−0.327, −0.036]).The paired-bootstrap table identifies negative differences as favoring HIPLEX for both takeover-rate metrics.Source: K EVALUATION SAMPLE SIZES, PAIRED… · p. 29, Table 11 · p. 29
  • Across three seeds, event-causal routing retains lower CANDOR pause and backchannel takeover rates than GRPO, preserves a perfect smooth-turn response rate, and reduces mean post-interruption latency.Mean judged quality is close to flat GRPO with greater seed variation; fixed-window routing has the highest conditional judged quality but gives up restraint and turn response.Source: K EVALUATION SAMPLE SIZES, PAIRED… · p. 29, Table 12 · p. 29

L SENSITIVITY TO THE GRPO LEARNING RATE

  • At 4 × 10^-6, GRPO has lower backchannel takeover than HIPLEX (0.000 versus 0.127), but higher pause TOR (0.431 versus 0.306) and lower judged quality (3.890 versus 4.083).Across tested learning rates, GRPO trades off metrics differently; reducing its rate from 10^-6 to 5 × 10^-7 worsens most reported metrics while increasing backchannel frequency.Source: L SENSITIVITY TO THE GRPO LEARNING RATE · p. 30, Table 13 · p. 30

M MECHANISM DIAGNOSTICS: WHERE EACH OBJECTIVE MOVES THE POLICY

The diagnostics show differentiated policy updates under the tested settings, but the control and content factors share model parameters. Semantic feedback is associated with lower repetition, and PersonaPlex exhibits a narrower stability margin at the tested rates.

M MECHANISM DIAGNOSTICS: WHERE EACH OBJECTIVE MOVES THE POLICY

  • For HIPLEX, recorded control KL exceeds the probability-weighted content KL contribution, while the two factors use distinct regularization terms and policy-loss eligibility rules.The diagnostics average KL over the final 30% of training; flat-baseline rows report total token-policy KL.Source: M MECHANISM DIAGNOSTICS: WHERE EACH… · p. 30, Table 14 · p. 31
  • Event-causal updates select approximately 20–30% of the frames used by a flat update, with a few dozen control decisions and under ten content decisions touched per rollout.The different decision frequencies motivate tuning learning rates separately, though selected-frame counts alone do not determine policy movement.Source: M MECHANISM DIAGNOSTICS: WHERE EACH… · p. 30
  • Timing-only HIPLEX repeats trigrams an order of magnitude more often than other systems, while routing judge reward to content brings repetition below flat baselines and leaves control-side KL essentially unchanged.The pattern is consistent with level-specific routing but does not establish behavioral independence because both losses update the same backbone.Source: M MECHANISM DIAGNOSTICS: WHERE EACH… · pp. 30–31, Table 14 · p. 31
  • At 4 × 10^-6, PersonaPlex collapses after roughly 1.75k steps into continuous audio with an empty text stream; halving the rate avoids the abrupt transition but reaches a near-silent endpoint.Rebalancing toward initiation keeps both suppression and initiation healthy; the authors suggest PersonaPlex's more talkative, floor-holding prompt may contribute to a narrower stability margin.Source: M MECHANISM DIAGNOSTICS: WHERE EACH… · p. 31

N WHAT INTERACTION LOOKS LIKE IN THE TRAINING DATA

Timing distributions vary across the examined corpora, and the section compares model behavior against pooled human marginals, tests judge-rubric and semantic-feedback choices, and identifies listener-specific adaptation as an open question.

N WHAT INTERACTION LOOKS LIKE IN THE TRAINING DATA

  • A third to two-thirds of a second separates Seamless floor transfers, versus essentially zero median offset for Fisher; overlap occurs on roughly a third of Seamless transitions and slightly more than half of Fisher transitions.The two Seamless subsets also differ, with roughly twice the backchannel rate in the improvised subset.Source: N WHAT INTERACTION LOOKS LIKE IN THE… · p. 31
  • The Wasserstein diagnostic pools sampled corpora into one-dimensional marginals; its fixed 6 s ceiling for no-response turns and omitted interruption-yield marginal constrain what the distances capture.The passage says these pooled marginals do not measure the full joint or context-conditioned distribution of human interaction.Source: N WHAT INTERACTION LOOKS LIKE IN THE… · p. 32
  • On Moshi, turn-timing distance is 4.70 for the base model, 1.78 for GRPO, and 0.90 for HiPLEX; HiPLEX is also more than four times closer on backchannel rate.On PersonaPlex, HiPLEX is closer on turn timing and backchannel rate, while GRPO is closer on backchannel length and the base model on pause intrusion.Source: N WHAT INTERACTION LOOKS LIKE IN THE… · p. 32

O HOW COARSE SHOULD THE JUDGE’S SCALE BE?

P SENSITIVITY TO STRONGER SEMANTIC FEEDBACK

  • 4.488 judged quality coincided with 58.3% backchannel takeovers versus 1.8% for the reported configuration; turn-response rate fell from 46.7% to 3.3% ten epochs later.The tested variants illustrate sensitivity to relative content-objective strength but do not establish a general limit on semantic feedback.Source: P SENSITIVITY TO STRONGER SEMANTIC… · p. 33

Q DISCUSSION

  • Human timing preferences vary by person and situation, which fixed-outcome benchmarks and pooled marginals do not capture; preference conditioning would require varied-preference data and profile-specific evaluation.The proposed conditioning includes explicit settings or spoken requests about backchannel frequency and readiness to take the floor.Source: Q DISCUSSION · pp. 33–34

Related papers

Loading 2610.07727v1…