Source-linked AI summary
Dear Algo: A Precision-First Agentic Intent Layer for Unified Search and Recommendation
Rui Wang, Jiazhou Wang, Zheng Wei, Chenglin Lu, Fangcheng Sun, Ivy Sun, Jin Sun, Hui Geng, Lillian Zhang, Chao Yang, Lei Chen, Shahin Sefati, Reem Helou, Joe Zhou, Babak Shakibi, Yiyi Pan, Bi Xue, Hong Yan, Shujian Bu
TL;DR
Search and recommendation encode intent differently, and explicit requests are rarely converted into durable, executable state for feed recommendation. Dear Algo compiles natural-language intent into grounded retrieval plans and, in production-path evaluation, reduced judge-Irrelevant share by 1.97 percentage points while increasing Exact-Relevant share by 2.24 points.
Problem
Explicit requests are rarely converted into durable, executable state for feed recommendation, while production retrieval still requires grounded identifiers, constraints, and inspectable execution.
Method
Dear Algo compiles explicit and negative natural-language constraints into grounded STQL retrieval plans, with optional semantic or multimodal reranking over bounded candidate sets.
Results
1.97 percentage points lower user-weighted judge-Irrelevant share and 2.24 points higher Exact-Relevant share were observed for the reranker-on path among judged admissions.
Takeaways & Limitations
The results support representation-level unification and precision-first evaluation, but not yet causal cross-task transfer.
Takeaways & Limitations
The explicit-to-feed behavioral analysis is observational and uses hour-level proxy sessions, so user self-selection and temporal factors may contribute to observed differences.
Abstract
from arXiv · showhide
Search and recommendation serve a shared discovery objective but encode intent differently. We study this boundary through Dear Algo on Threads, a deployed product where open-ended requests such as \emph{more NBA news} or \emph{less politics} steer subsequent feed recommendations rather than return a one-shot result list. Its agentic intent layer compiles explicit, inferred, negative, and compound intent into a grounded executable plan, then invokes conventional retrieval and optional semantic or multimodal reranking. The layer shares an intent-to-retrieval contract without requiring one model or serving path across search-like and recommendation-like modes. We evaluate Dear Algo under a precision-first objective. In a blinded audit of 300 public request-item pairs (296 evaluable), a strict categorical LLM-as-a-judge gate achieved 94.4\% exact-Relevant precision [88.8\%, 98.9\%]. Across 72 normalized request clusters, the full configuration produced 7.73 judge-qualified candidates per 20 slots versus 6.61 for an LLM-derived-query baseline, a gain of 1.11 [0.12, 2.12]. In a candidate-randomized serving-path study restricted to the reranker path's first 72 eligible hours, the user-weighted judge-Irrelevant share among judged admissions was 2.80\% versus 4.78\% off (-1.97 points [-3.02, -0.94]), while Exact-Relevant share was 2.24 points higher [0.08, 4.41]. Together, these studies show how explicit natural-language intent can be carried into feed recommendation under a precision-first evaluation framework
1 Introduction
Dear Algo addresses the gap between explicit search intent and longer-lived feed recommendation by converting natural-language requests into grounded, executable plans for feed steering. Its shared intent layer supports precision-first evaluation while allowing interfaces and serving paths to remain mode-specific.
- Motivation: Search uses explicit query intent, whereas recommendation relies mainly on implicit actions and rarely converts requests into durable, executable feed state.Production retrieval additionally requires valid identifiers, explicit constraints, bounded execution, fallback, and inspectable telemetry.
- Product: Dear Algo lets Threads users request preferences such as more NBA news, less politics, or content that will make them laugh, steering subsequent feed retrieval.The interface changes feed recommendations rather than returning only a one-shot ranked list.
- Method: The agentic intent layer maps instructions into typed positive and negative constraints, grounds open-ended phrases to controlled vocabulary, and compiles reusable STQL retrieval plans.Optional semantic or multimodal reranking operates on bounded candidates; STQL is an internal JSON-serialized retrieval DSL, not relational SQL.
- System design: The compiler and plan are shared across modes, while interfaces, operating points, and serving paths remain mode-specific because irrelevant admissions can cause repeated unwanted exposure.The evaluation separates admission precision, recall, ranking, engagement, and report actions.
- Contributions: The paper contributes a shared executable intent layer, a layered account of unification, and a precision-first evaluation bridge that keeps distinct estimands separate.It combines human calibration, offline candidate evaluation, randomized admitted-set analysis, behavioral analysis, and logging lessons for causal cross-mode evaluation.
2 Related Work
Related work distinguishes search from recommendation by their evidence sources, while prior systems explore cross-task transfer, interactive steering, semantic parsing, and agentic control. Dear Algo contributes an operational intent layer that grounds language into typed, inspectable serving plans while preserving mode-specific serving and requiring careful policy and judge validation.
- Search and recommendation: Search conditions decisions on explicit, often transient queries, whereas recommendation infers preferences from interaction history over shared catalogs.Prior work unifies behavior sequences, graphs, encoders, transition models, and item representations across the tasks.
- Search and recommendation: Cross-task gains require aligned histories, compatible popularity distributions, and related item co-occurrence, so Dear Algo shares a contract but retains mode-specific serving choices.The shared contract consists of a controlled vocabulary and typed executable plan; retrieval, ranking, thresholds, and latency budgets remain mode-specific.
- Interactive recommendation: Dear Algo grounds positive, negative, and compound instructions to production feature identifiers and compiles them into an inspectable plan that can persist beyond the initiating request.Its persistent lifecycle must handle preference drift, correction, provenance, and deletion.
- Intent representation: The intent layer keeps the catalog external: an LLM proposes intent, grounding resolves supported values, and a typed DSL expresses predicates and retrieval parameters.This separates identifier validity from semantic correctness and differs from free-form query expansion or direct generative retrieval.
- Evaluation: Deployable cross-mode policy effects require randomization, while observational studies, short experiments, and behavioral metrics have distinct limitations for estimating intent fulfillment.The paper therefore separates request relevance, admitted-set quality, user behavior, and reported actions.
3 Dear Algo: A Layered Intent Architecture
Dear Algo uses a layered intent architecture that compiles heterogeneous requests and context into grounded, executable STQL plans while sharing orchestration, catalogs, vocabulary, and interpretation across modes. Candidate configuration, ranking, and interaction surfaces remain mode-specific, and the implementation supports immediate execution or refreshed feed recommendation through STAR.
- Layered claims: Dear Algo shares infrastructure and representation layers across discovery modes, but does not establish causal empirical unification.Common components include orchestration, catalogs, vocabulary identifiers, intent interpretation, and STQL plans; ranking and interaction remain mode-specific.
- Intent compilation: The intent layer converts explicit, passive, negative, and compound inputs into normalized intent, an executable plan, candidates, and ranking.Its input combines an optional request, eligible user context, immediate interaction context, and discovery mode.
- Intent compilation: Schema-constrained features specify semantic types, polarity, candidate values, and valid operators before grounding maps values into controlled identifiers.The planner composes inclusion, exclusion, conjunction, disjunction, language, location, and freshness constraints into STQL.
- Execution: Typed, grounded plans drive existing high-throughput candidate generation, optional text or multimodal LLM reranking, bounded expensive stages, and refreshable plan execution.Unsupported plans abstain or follow the existing fallback path, while persisted plans can refresh against new content without rerunning every reasoning stage.
- Implementation: STAR implements the intent layer on SilverTorch, allowing explicit requests to execute immediately or persist for refreshed intent-conditioned feed candidates.SilverTorch integrates filtering, approximate nearest-neighbor retrieval, and scoring as model components.
4 Evaluation
The evaluation separates measurement calibration, offline candidate yield, and randomized serving-path quality rather than combining them into one significance claim. Each study uses distinct estimands, controlled comparisons, and explicitly stated limitations.
- Evaluation design: The studies address separate questions: evaluator calibration, offline qualified-candidate yield, randomized admitted-item quality, and behavioral change after explicit requests.The evaluation does not combine these studies into a single significance claim.
- Human calibration: 300 sampled request-item pairs were audited with a strict gate admitting only judge-Relevant items; sampling oversampled Irrelevant and Partial strata for audit precision.The 300-pair design used 90 Irrelevant, 120 Partial, and 90 Relevant pairs, with inverse selection weights for cross-stratum estimates.
- Offline study: 72 normalized request-text clusters supported the offline comparison, with fixed inventory, infrastructure, and Top-20 budget while configurations retrieved different slates.The primary outcome is expressed as both Yield@20 and qualified candidates@20 using the same fixed denominator.
- Candidate-randomized study: The serving-path study randomized candidates within requests between reranker-off and reranker-on sources and restricted the primary comparison to the common-support window below 72 hours.The restriction followed operational TTL validation rather than quality-outcome selection.
- Candidate-randomized study: Quality metrics average candidates within requests and requests within users, reporting paired user contrasts with 20,000 user-cluster bootstrap replicates.Because rejected candidates and a frozen pre-reranking slate are unavailable, the study estimates admitted-item quality and reports arbitrary-MNAR bounds.
5 Results
Dear Algo achieved high exact-Relevant precision under a strict admission gate, increased fixed-denominator judge-qualified yield over comparison configurations, and improved judged-admission quality with reranking. Explicit-request initiation was followed by higher Threads app usage, while report-rate changes were not statistically significant.
- Strict-gate audit: 94.38% exact-Relevant precision (84/89) was achieved under the strict gate, with 100.00% at-least-Partial precision (89/89).The gate admitted 84 of 176 human-Relevant pairs, trading coverage for admission precision.
- Interpretive limitations: The result is descriptive, not a preregistered pass, because the precision-first estimand and threshold were clarified after label inspection.Initial human reliability was weak, with pairwise exact agreement of 0.677 and quadratic weighted kappa of 0.250.
- Configuration comparison: 7.73 qualified candidates@20 at the 0.8 threshold exceeded configuration B by 1.11 [0.12, 2.12] and configuration A by 2.15 [0.59, 3.73].Across all three thresholds, both C–B and C–A intervals excluded zero; the supported result is configuration-level yield improvement.
- Serving-path comparison: -1.97 points [-3.02, -0.94] in Judge-Irrelevant share and +2.24 points [0.08, 4.41] in Exact-Relevant share favored reranker-on over reranker-off.These primary differences were measured among judged admissions within the common-support window.
- All-eligible cohorts: All three cohorts showed higher post-period usage, while every two-sided report-rate interval included zero.Excluding users with a repeat request retained the session pattern, and report-rate intervals again included zero.
6 Discussion
Dear Algo uses a modular shared intent layer across explicit and recommendation-derived modes without forcing identical operating points. The discussion distinguishes offline candidate quality, serving-path effects, and behavioral analysis while motivating a causal follow-up study.
- Architecture: Dear Algo implements shared vocabulary, plan, and candidate interfaces for explicit and recommendation-derived intent.The architecture supports shared interfaces across modes while leaving operating points distinct.
- Architecture: A broad candidate stage may suit explicit requests, whereas persistent recommendation intent may require a more conservative admission gate.The modular shared layer is intended to limit negative transfer while enabling cross-mode state.
- Evaluation scope: The offline candidate study holds request-time inventory and a 20-slot budget fixed, while the candidate-randomized study estimates serving-path effects only on eligible, admitted, judge-labeled impressions.The serving-path study cannot recover recall over the pre-reranking universe.
- Future experiment: The next study should randomize stored-plan availability while holding inventory, retrieval budgets, and downstream ranking fixed, with intent fulfillment primary and usage and report rate secondary.The proposed design targets causal effects of carrying explicit requests into feed recommendation.
7 Limitations and Responsible Use
The study’s evidence is limited by observational and proxy-based behavioral analyses, incomplete offline and judgment coverage, post-hoc evaluation choices, and unadjusted behavioral intervals. Responsible deployment requires protecting sensitive intent, minimizing data, controlling access and retention, enabling deletion and correction, and auditing negative-preference suppression.
- Study limitations: Behavioral differences are observational and based on hour-level proxy sessions, so self-selection, temporal trends, regression to the mean, and feed-composition changes may contribute.The paired candidate study also uses grounded tags and STQL across all offline configurations, limiting its scope.
- Study limitations: Offline human calibration covers public request-item pairs and does not establish private, per-intent, or multimodal validity.Post-hoc latent-label sensitivity depends on the prior and raters.
- Study limitations: Precision-first endpoints were clarified after initial result inspection, while behavioral intervals are not multiplicity-adjusted.These choices affect interpretation of the reported evaluation evidence.
- Study limitations: The reranker study observes only admitted candidates, with TTL-aligned request age serving as an eligibility proxy and differing judgment coverage and candidate composition by arm.Its arbitrary-MNAR bounds span both directions, and recall or reranker validity is not established in the supplied passage.
- Responsible use: Persisted explicit or inferred intent may expose sensitive interests or amplify unwanted feedback loops, while confidentiality and privacy constraints prevent releasing implementation details, user-level logs, or reproduced requests.Reported results are aggregate.
- Responsible use: Deployable use requires data minimization, access control, expiration, deletion, and user correction, plus collateral-suppression audits for negative preferences.These safeguards address privacy and the potential effects of negative intent.
8 Conclusion
Dear Algo demonstrates a shared, grounded, executable intent layer for explicit and recommendation-like discovery while preserving task-specific serving choices. Its results support representation-level unification and precision-first evaluation, but not yet causal cross-task transfer.
- System contribution: Dear Algo unifies explicit and recommendation-like discovery through a shared, grounded, executable intent layer while preserving task-specific serving choices.The system supports a common intent layer without requiring identical serving paths.
- Evaluation: 1.97 percentage points lower user-weighted judge-Irrelevant share was observed in the candidate-randomized serving-path study’s TTL-aligned delivery window.The study used candidate-randomized serving paths within a TTL-aligned delivery window.
- Scope of evidence: The evidence supports representation-level unification and precision-first evaluation, not causal cross-task transfer.This conclusion limits the scope of the reported evidence to representation and evaluation, rather than causal transfer across tasks.