Source-linked AI summary
An Interactive Agent for Requirement-Driven Candidate Sourcing
Yuanpeng He, Fangjing Li, Xiangyu Ru, Kexin Sun, Kun Yang, Lijian Li, Chi-Man Pun, Qingsong Wen, Wenpin Jiao, Mingkai Guo, Yirong Feng, Daiheng Gao, Zhi Jin
TL;DR
Vague natural-language candidate requests are under-determined requirements with implicit constraints, many valid answers, and no acceptance criterion, so sourcing requires more than retrieval. The paper presents an interactive requirements-driven agent and a benchmark that operationalize elicitation, validation, retrieval, and verification. Across 21 systems and 691 requirements, the agent recalls the most relevant real people while serving as a deep sourcing engine complementary to precision-ranking LLMs.
Problem
Vague candidate requests are under-determined requirements with implicit constraints, many valid answers, and no acceptance criterion, but existing retrieval systems assume finished specifications.
Method
The paper presents DINQ TalentScout and TalentTrace to elicit, validate, retrieve, verify, and evaluate candidate-sourcing requirements through grounded evidence and lifecycle-based benchmarking.
Results
Across 21 systems and 691 requirements, DINQ TalentScout recalls the most relevant real people, reaching 0.241 of the union pool and 1.9× the next system.
Takeaways & Limitations
DINQ TalentScout is positioned as a deep sourcing engine, while precision-ranking LLMs serve as complementary verifiers.
Takeaways & Limitations
The evaluation uses model-constructed oracles, and truthfulness judgments are noisy for concise profile candidates, making precision and evidence results approximate.
Abstract
from arXiv · showhide
Finding people from a natural-language description (``ML engineers transitioning to research roles in biotech'') is increasingly delegated to LLM agents and framed as information retrieval. We argue that it is fundamentally a requirements engineering task: such a request is an under-determined requirement with implicit constraints, many valid answers, and no acceptance criterion, so useful answers require eliciting, validating, and verifying the requirement before search can matter. We present \sys{}, to our knowledge the first interactive, requirements-driven candidate-sourcing agent (it elicits, validates, retrieves, and verifies a vague people-request into a justified slate through bounded elicitation, workflow templates, a two-stage commit protocol, and bidirectional termination guards) and \bench{}, a benchmark that runs the requirements lifecycle (criteria-anchored validation, multi-model evidence-grounded oracle construction, and cost-aware verification). Across $21$ systems and all $691$ requirements, \sys{} dominates breadth ($100%$ coverage at $2.5\times$ the yield) and is \emph{near-orthogonal} to the field, with $90%$ of the people it returns are surfaced by \emph{none} of $20$ strong LLM-plus-web baselines combined. Beyond breadth, an evidence-grounded judging of every system shows \sys{} \emph{recalls} the most relevant real people: $0.241$ of the union pool, $1.9\times$ the next system, with a bootstrap $95%$ interval disjoint from every baseline. \sys{} is thus the strongest \emph{sourcing} engine (the deepest real, reachable candidate pool), while precision-ranking LLMs serve as~complementary verifiers.
I. INTRODUCTION
The paper reframes vague candidate sourcing as requirements engineering: systems must elicit, validate, retrieve, and verify an under-determined people-request rather than rank profiles against a finished query. DINQ TalentScout and TalentTrace operationalize this lifecycle, and DINQ TalentScout finds a deeper, largely distinct pool than strong LLM-plus-web baselines.
- Motivation: A vague people-request leaves constraints implicit, permits many valid answers, and lacks an acceptance criterion for checking returned candidates.Classical retrieval systems generally assume that the request is already a well-formed query.
- Agent: DINQ TalentScout is an interactive, requirements-driven agent that elicits latent constraints, validates requirements, retrieves candidates, and verifies each person with grounded evidence.Its design applies one principle across four sources of agent divergence.
- Benchmark: TalentTrace is a benchmark that runs validation, evidence-grounded oracle construction, and cost-aware verification to evaluate sourcing agents at scale.The benchmark addresses the absence of an existing benchmark for this setting.
- Results: 90% of DINQ TalentScout’s returned people are surfaced by none of 20 strong LLM-plus-web baselines combined.Across 21 systems and 691 requirements, this near-orthogonality indicates that the agent accesses a distinct candidate pool.
- Results: 0.241 of the union pool is recalled by DINQ TalentScout, 1.9× the next system, under evidence-grounded judging of relevant real people.The result is reported as bootstrap-significant, positioning DINQ TalentScout as a deep sourcing engine and precision-ranking LLMs as complementary verifiers.
- Motivating example: For one benchmark requirement, DINQ TalentScout returns 46 named, evidence-backed candidates, whereas a strong GPT-class web agent returns 5 with zero overlap.The reported gap is attributed to baseline failures in verification and elicitation, including unsupported evidence and binding-constraint violations.
III. CANDIDATE SOURCING AS REQUIREMENTS ENGINEERING
Candidate sourcing is framed as requirements engineering: the system elicits and validates an under-specified request, operationalizes it for retrieval, and verifies delivered candidates against evidence. DINQ TalentScout implements this lifecycle with bounded workflows, a two-stage commit, and explicit termination controls.
- Requirements perspective: A sourcing request is an under-determined requirement whose constraints, valid answers, and acceptance criteria are initially implicit.The lifecycle therefore distinguishes elicitation, validation, operationalization, and verification rather than treating sourcing as query ranking alone.
- Requirements perspective: The architecture separates validation of the requirement from verification that delivered people satisfy it with grounded evidence.Bounded elicitation and criteria-anchored validation address the requirement; an evidence-grounded oracle and commit–defer scoring address delivered candidates.
- System workflow: A three-way intent router and seven-dimension interview constrain elicitation to at most four questions followed by confirmation.The dimensions include domain, compensation, location, role, education, experience, and headcount, with a missing headcount defaulting to twenty.
- System workflow: Two-stage commit first drafts candidates, then enriches and verifies all of them before refinement and delivery.Each candidate is classified as a verified match, verified mismatch, or unverifiable, with evidence and confidence attached.
- System workflow: A fixed five-step workflow orchestrates parallel retrieval across internal, professional, academic, open-source, and company sources.The operationalization and retrieval stage is presented as a structured alternative to exploratory, irreproducible search.
- System workflow: Guardrails and bidirectional termination boundaries regulate continued search, verification, and delivery.The workflow includes explicit thresholds, a headcount target, session resumption, and self-healing parallel tool calls.
IV. DINQ TALENTSCOUT: AN INTERACTIVE REQUIREMENT-DRIVEN SOURCING AGENT A. Problem Formulation
The paper formulates open-ended candidate sourcing as returning a bounded, ranked, evidence-backed list of real people for a natural-language request. It represents the requirement and candidate set explicitly, including constraints, requirement kind, target cardinality, evidence URLs, and calibrated confidence.
- Problem Formulation: Open-ended sourcing must return a bounded, ranked, evidence-backed list of real people from a natural-language request.The formulation targets people who satisfy the request rather than a single retrieved artifact.
- Problem Formulation: The requirement is represented as R = (D, K, n), with constraint dimensions D, requirement kind K, and target cardinality n.Dimensions include properties such as domain, location, and seniority; K distinguishes open-ended from closed requirements.
- Problem Formulation: The system produces C = {c1, . . . , cm}, where m ≤n and each candidate includes identity, organization, role, evidence URLs, and calibrated confidence.This output structure operationalizes the requirement into a bounded candidate set.
B. Design Principle: Four Divergences, Four Constraints
DINQ TalentScout addresses four orthogonal divergence axes in unconstrained LLM agents: under-determined input, wandering search, incomplete or conflicting output, and premature or endless termination.
- Four Divergences, Four Constraints: LLM agents guess missing constraints when the input requirement is under-determined.The input divergence is addressed by constraining how the requirement is elicited.
- Four Divergences, Four Constraints: Exploratory and irreproducible search causes process divergence during retrieval.The design principle treats workflow structure as the constraint on this axis.
- Four Divergences, Four Constraints: Fast results can conflict with complete, verified fields at the output.This output divergence motivates controls that reconcile delivery speed with candidate verification.
- Four Divergences, Four Constraints: Agents may stop before verifying candidates or continue indefinitely when a query produces no hits.The termination constraint addresses both premature stopping and unbounded spinning.
C. Eliciting the Requirement (Clarification Contract)
DINQ TalentScout turns vague requests into structured requirements through a bounded clarification contract. It routes intent, asks about one requirement dimension per turn, and confirms after at most four questions.
- Eliciting the Requirement: A system prompt enforces a structured output contract and routes requests into specific-person, by-conditions, or other intent types.The open-ended branch uses the bounded elicitation interview.
- Eliciting the Requirement: The open-ended branch asks about one dimension per turn across seven requirement dimensions.The dimensions are domain, compensation, location, role, education, experience, and headcount.
- Eliciting the Requirement: Elicitation stops after at most four questions and then requires a confirmation turn, even when dimensions remain unknown.A missing headcount defaults to twenty, keeping the interview bounded while completing the structured requirement.
D. Operationalizing the Requirement (Workflow Template)
TalentScout operationalizes sourcing through a fixed workflow that surfaces candidates quickly, gathers evidence by chaining sources, and submits candidates through a structured pipeline.
- Workflow template: The retrieval-kickoff recipe combines skill loading with a search-and-submit call so initial candidates appear within seconds.Subsequent turns plan sources, search broadly in parallel, and draft each surfaced person immediately.
- Evaluation pipeline: The TalentTrace pipeline validates requirements, builds acceptance pools, verifies candidates, and deterministically aggregates per-system scores.
E. Decoupling Speed and Quality (Two-Stage Commit)
The agent separates rapid candidate surfacing from later refinement and uses bounded stopping safeguards to balance responsiveness, completeness, and stable execution.
- Two-stage commit: Every candidate is submitted twice: an immediate draft provides basic identity and snippet evidence, while refinement adds organization, role, links, and recalibrated confidence.The front end merges both submissions using the stable candidate_id.
- Termination guards: A full-coverage guard prevents stopping while candidates remain draft-only, with retries supporting resubmission of searched-but-unsubmitted candidates.
- Termination guards: Termination is bounded between an 80-call narrow threshold and a 100-call halt that reports honestly.
- Runtime support: The runtime adds session resumption, stuck detection, credential isolation, and bounded tool concurrency to support stable execution.
- Evaluation context: TalentTrace evaluates sourcing with generated and validated requirements, evidence-backed oracles, and candidate verification rather than queries and answer keys alone.
- Evaluation context: The benchmark includes open and closed requirements across six task types and three difficulty levels, with task types deterministically assigned after generation.
- Evaluation context: 1,007 of 1,500 generated requirements were admitted, including 691 open and 316 closed requirements.
B. Validating Requirements with Criteria-Anchored Debate
CADA validates requirements by anchoring heterogeneous model judgments to explicit criteria, admitting well-formed requests and classifying them as open or closed for appropriate evaluation.
- Criteria-Anchored Debate: CADA uses three heterogeneous LLMs to score criteria before adjudicating requirement quality, reducing model-specific optimism and making decisions auditable.A GPT-class judge rated 86.8% of closed requirements perfect, compared with a Claude-class mean of 3.62/5.
- Quality admission: Requirements enter the benchmark only when they satisfy seven binary quality tests and reach a quality score of at least 3.5/5.The binary tests cover ambiguity, feasibility, consistency, verifiability, scope, temporal validity, and fabricated premises.
- Kind classification: CACJ classifies admitted requirements as OPEN or CLOSED according to whether their constraints determine an enumerable answer set or an open pool.
- Acceptance oracle: Open requirements receive graded, evidence-backed acceptance pools built from independent web searches and cross-family judging of source reasonableness, evidence depth, and usefulness.
- Quality admission: CADA’s seven binary tests correspond to classical requirements-quality attributes applied to people-search requirements.
- Quality admission: The admission criteria include appropriate scope, temporal validity, and non-existent-entity checks alongside intent, search, type, and constraint verification.
D. Verifying Satisfaction (CGPS)
CGPS verifies delivered candidates against requirements and acceptance oracles using cost-aware staged judgment, then reports deterministic recall and reachable yield over relevant real people.
- Cost-aware verification: CGPS fixes requirement-level constraints, applies a cheap relevance gate, and defers expensive evidence fact-checking to a calibration subset.This mirrors the agent’s commit–defer discipline while controlling verification cost.
- Candidate verdicts: Each candidate receives a structured verdict with one primary issue bucket: OFF-TOPIC, THIN-SOURCES, BROKEN-URL, LIKELY-HALLUCINATION, or NONE.
- Scoring: CGPS reports recall and reachable yield over the union pool of relevant real people, reducing structured verdict labels deterministically to scores.
- Scoring: A candidate counts as a relevant real person when it is not OFF-TOPIC, with relevance based on name, organization, and title rather than evidence thickness.
- Experimental scope: All empirical results use 691 open requirements, while 316 closed requirements validate the open/closed boundary and are not run against systems.
- Experimental scope: The benchmark spans ten difficulty levels, sixteen sourcing task-types, and seven languages; grounded quality grades every candidate from 21 systems against acceptance oracles for 203 open requirements.The 203 oracle-backed requirements have a mean pool of 44.6 people and mean reliability of 0.60.
A. RQ1: Coverage and Yield at Scale
DINQ TalentScout combines complete coverage with the deepest candidate yield across 691 requirements, while retrieving a largely distinct candidate pool from 20 baselines. Evidence-grounded evaluation further finds it recalls the most relevant real people across 21 systems.
- Coverage and yield: 100% coverage and 49.9 candidates per requirement make DINQ TalentScout the only system pairing complete coverage with deep yield.One GPT-class LLM also reaches 100% coverage but yields 19.8 candidates per requirement; Exa-augmented systems yield only 4–12 and bare/Grok-class models miss up to 45% of requirements.
- Robustness: 100% coverage and approximately 46–57 candidates persist across all ten difficulty levels, sixteen task-types, and seven languages.Baselines degrade on hard and non-English requirements, including cross-verify, temporal, niche, Chinese, career-trajectory, and stability cases.
- Complementarity: 90.0% of DINQ TalentScout’s 34,406 requirement-person mentions appear in none of 20 pooled baselines, despite their union being 2.5× larger.Mean pairwise Jaccard is 0.010, and stricter rematching changes the unique fraction only to 89.7%.
- Grounded quality: 0.241 recall of the relevant-real-person union pool is 1.9× the next system, with a bootstrap 95% confidence interval disjoint from all baselines.DINQ TalentScout also surfaces 14,267 reachable relevant candidates and 32.8 relevant people per requirement.
- Grounded quality: DINQ TalentScout ranks first on every CGPS sourcing-depth leaderboard, most by approximately 2×.The leaderboard marks bootstrap intervals disjoint from all 20 baselines with an asterisk.
E. RQ5: Efficiency and Cost
The system concentrates computation on difficult decisions while using bounded clarification to recover requirements that single-shot reading misses. This yields faster grounded retrieval and substantially stronger recovery of latent constraints.
- Efficiency and cost: 0.92 seconds to first result makes the hybrid backbone 38% faster than web-only retrieval.The two sources run in parallel and short-circuit.
- Efficiency and cost: 95.3–97.5% of requirement-kind decisions skip the expensive arbitrator under the commit–defer cascade.A no-early-exit control gives identical verdicts at approximately 20× the cost.
- Ablations: Web-only retrieval loses 7.6 people per requirement, while internal-only retrieval drops to 56% coverage.Mix retrieval also supplies fact-checkable evidence from the internal index.
- Clarification: 67.5% latent-constraint recovery versus 13.8% under single-shot reading represents a 4.9× gain from the bounded clarification interview.The gain appears on 33 of 40 requirements and holds across every constraint dimension.
- Clarification: 0%→75% geography recovery and 4%→48% organization-type recovery are the steepest per-dimension improvements from clarification.These dimensions represent implicit where and what-kind constraints that vague requests often omit.
VIII. DISCUSSION AND IMPLICATIONS FOR RE
The paper treats open-ended candidate sourcing as requirements engineering, using traceable acceptance criteria and evidence-grounded evaluation to separate retrieval depth from verification precision. Its results position DINQ TalentScout as a sourcing engine, with precision-ranking models as complementary verifiers, while noting important validity and availability boundaries.
- Discussion and implications: Acceptance decomposes sourcing into recall, reachable yield, relevance, and evidence-support to localize failures to retrieval or verification.TalentTrace makes criteria validation first-class through auditable rubrics and deterministic scoring.
- Discussion and implications: Recall is the sourcing objective because a missed candidate is unrecoverable, whereas precision can be applied as a downstream human filter.The paper therefore assigns deep candidate discovery to DINQ TalentScout and pruning to precision-ranking LLMs.
- Implications for RE: The requirements lifecycle is presented as transferable to literature and dataset discovery, vendor selection, and compliance evidence.Candidate sourcing is described as the demanding case because it involves open-set oracles and adversarial evidence.
- Threats to validity: The judge’s truthfulness assessments are noisy for concise profiles, so precision and evidence-support results are treated as approximate.Model-constructed evaluation oracles are not exhaustively human-verified at scale, and some pool members may be stale or missing.
- Related work: Existing people-search benchmarks and recruiting tools generally rank against finished queries without elicitation or well-formedness validation.The paper contrasts this stance with requirements-driven agents for software engineering and its own people-sourcing formulation.
- Scope boundary: The studied system is proprietary, and its source code and data cannot be publicly released because of confidentiality restrictions.The paper states that re-implementation algorithms and settings are specified in the paper.