PaperlayerView exact revision on arXiv

Source-linked AI summary

HiddenLens: A Multi-Layer LLM Framework for Detecting and Resolving Hidden Assumptions in Software Requirements

Tanu Singh, Aashna Suman, Vinod Patidar

arXiv:2610.29996v1cs.SEcs.AIcs.CL

TL;DR

Natural-language requirements can appear clear while leaving assumptions unstated, a gap that existing ambiguity-focused work has less often addressed. HiddenLens combines BERT and LLM stages with analyst validation to detect and resolve those assumptions; it reports improved detection over two baselines and evaluates NFR classification and corpus-based performance.

  • Problem

    Natural-language requirements can appear complete while stakeholders interpret terms differently, and less attention has been given to assumptions hidden in apparently reasonable requirements.

    Source: §1 · pp. 2–3

  • Method

    HiddenLens is a six-layer BERT–LLM pipeline that localizes candidate spans, articulates and classifies assumptions, generates clarification options, supports analyst validation, and rewrites requirements.

    Source: §1 · pp. 3–4, §4 · p. 8

  • Results

    An assumption-detection F1 of 0.79 represents a 71% relative improvement over the keyword-based baseline and a 61% improvement over QuARS-Adapted; BERT NFR classification achieves an F1 of 0.847.

    Source: §7.1 · p. 16, §10 · p. 21

  • Takeaways & Limitations

    The authors conclude HiddenLens can assist analysts, developers, and organisations in identifying potential problems before they become costly or contribute to late-stage project failures.

    Source: §10 · p. 21

    The 312-requirement corpus is smaller than real-world industrial specifications, so transfer to other domains requires domain-specific validation.

    Source: §9 · p. 20

Abstract

from arXiv · show

Natural-language software requirements often omit assumptions about performance, operating conditions, user behaviour, and domain constraints, affecting architecture, testing, and deployment. If undiscovered until implementation, resolving them is far more costly than addressing them early. Although ambiguity and vagueness are widely studied, less attention has been given to assumptions embedded in requirements that appear clear and reasonable. We propose HiddenLens, a novel six-layer BERT-LLM pipeline and, to our knowledge, the first framework engineered to detect, classify, question, and resolve hidden assumptions rather than merely flag ambiguity. Layer 1 uses BERT attention entropy to localise unresolved spans; Layer 2 uses CoT-prompted LLM reasoning to articulate each assumption and its failure risk; Layer 3 uses fine-tuned BERT to classify assumptions into six non-functional requirement categories: performance, security, scalability, usability, reliability, and domain; Layer 4 generates ranked clarification options using FS LLM prompting; Layer 5 enables human validation through confirmation, editing, or rephrasing; and Layer 6 rewrites the requirement using the confirmed resolution, producing an explicit, testable, and traceable requirement. We also introduce the HiddenLens Corpus (HLC), comprising 312 requirements annotated with 847 hidden assumptions across six NFR categories. Inspired by PROMISE and PURE, HLC addresses the lack of annotated data for this task and is, to our knowledge, the first dataset developed specifically for it. HiddenLens outperforms keyword-based and QuARS-Adapted baselines by 71% and 61%, respectively, achieving an end-to-end F1-score of 0.79; its BERT-based NFR classifier achieves an F1-score of 0.847. These results indicate that HiddenLens can help analysts resolve hidden assumptions before they propagate through software development.

1. Introduction

Requirements shape downstream software decisions, yet natural-language requirements can leave stakeholders with different interpretations of apparently clear terms. Hidden assumptions may remain unnoticed until they produce mismatches between specified and expected behavior; HiddenLens is introduced to detect and resolve them.

1. Introduction

  • Requirements problems include incompleteness, inconsistency, and ambiguity, with ambiguity described as pervasive and difficult to detect.Requirements defects have been identified among the top three root causes of systemic failures.Source: §1 · p. 2
  • Natural-language terms such as “quickly” and “securely” can appear clear while stakeholders assign them different meanings.The same gap may go unchallenged in review because the requirement appears complete and well written.Source: §1 · pp. 2–3
  • Hidden assumptions can create divergent stakeholder expectations and deviations between specified and expected system behavior.Figure 1 illustrates how such assumptions may go unnoticed during specification and contribute to project failure.Source: §1 · p. 3, Figure 1 · p. 3
  • Existing ambiguity and quality tools identify poorly specified symptoms but do not articulate the specific information missing behind hidden assumptions.The introduction positions HiddenLens as addressing this gap by identifying assumptions and supporting their validation and resolution.Source: §1 · p. 3
  • HiddenLens is presented as a six-layer BERT–LLM pipeline for detecting, classifying, and resolving hidden assumptions in software requirements.The introduction also describes analyst validation, requirement rewriting, and a corpus of 312 requirements with 847 annotated assumptions.Source: §1 · pp. 3–4
  • The introduction reports an assumption-detection F1-score of 0.79 and a BERT NFR-classifier F1-score of 0.847.It describes improvements over rule-based and keyword-based baselines, and identifies prompt-strategy ablation and error analysis as part of the evaluation.Source: §1 · p. 4

2. Related Work

Requirements engineering research has progressed from linguistic rules to learned classifiers and LLM reasoning, but prior approaches largely analyze information stated in the text. HiddenLens is proposed to detect, classify, and resolve the assumptions that remain implicit.

2. Related Work

  • Pattern-based tools such as ARM and QuARS detect linguistic defects, while Smella reports requirements smells without diagnosing what information is missing (Wilson et al., 1997; Lami et al. 2005; Femmer et al., 2017).Berry et al. (2003) distinguish deliberate from accidental ambiguity, a nuance most tools overlooked.Source: §2 · pp. 4–5
  • Neural requirements classifiers learn richer representations, but remain limited to classification or defects discernible in the text.The review also notes that preprocessing choices could substantially shift classifier performance.Source: §2 · p. 5
  • BERT classifiers, prompt-based classification, and LLM quality assessment extend requirements analysis, while the reviewed work still focuses on stated information or reports domain dependence.A review of 137 NLP-in-RE studies likewise found less research on implicit requirements and underlying presumptions (Necula et al., 2024).Source: §2 · p. 5
  • LLMs support few-shot prompting and inferential reasoning, but prior generation work still operates on information explicitly stated in requirements (Brown et al., 2020; Wei et al., 2022; Almonte et al., 2025).HiddenLens is proposed as a six-layer BERT–LLM pipeline for detecting, classifying, and resolving hidden assumptions.Source: §2 · pp. 5–6
  • Prior approaches are described as not directly identifying and resolving hidden assumptions, despite advances in requirements tooling and LLM-based tasks.The review notes that unvalidated assumptions can translate into architectural flaws, while a scalable systematic tool remained unavailable.Source: §2 · p. 6

3. Problem Definition

The section defines hidden assumptions as implementation necessities absent from a requirement, distinguishing them from ambiguity and vagueness. It also identifies why they are difficult to detect and presents a six-category taxonomy.

3. Problem Definition

  • The problem-definition section separates defining hidden assumptions from distinguishing them from vague or unclear requirements and describing their six categories.Source: §3 · p. 6

3.1 Formal Definition

  • A hidden assumption is necessary for correct implementation but absent from the requirement, unlike ambiguity or vagueness that concern propositions stated imprecisely or with conflicting interpretations.The section notes that hidden assumptions can be invisible in surface text, depend on domain context, and span multiple NFR categories.Source: §3.1 · p. 6

3.2 Assumption Taxonomy

  • The paper identifies six principal assumption categories from analysis of 312 HLC requirements and NFR-classification literature.Table 2 presents the taxonomy with definitions and illustrative examples.Source: §3.2 · p. 7, Table 2 · p. 7

4. The HiddenLens Framework

HiddenLens is a six-layer BERT–LLM pipeline that detects, classifies, and resolves hidden assumptions with analyst input. It moves from candidate spans and assumption hypotheses to clarification, validation, and rewritten requirements.

4. The HiddenLens Framework

  • HiddenLens combines six layers to detect, classify, and resolve hidden assumptions, engaging an analyst and rewriting requirements to make assumptions explicit.Source: §4 · pp. 7–8

4.1 Layer 1: BERT Attention Entropy (Ambiguity Localisation)

  • Layer 1 uses BERT attention entropy to flag high-entropy tokens as candidate anchors when their scores exceed 60% of the sentence maximum.The approach requires no additional training for this step and can be applied across requirement domains, unlike keyword methods relying on static lexicons.Source: §4.1 · pp. 8–9

4.2 Layer 2: LLM Chain-of-Thought Assumption Detection

  • Layer 2 prompts an LLM with candidate terms and the requirement to produce structured assumptions, failure risks, and analyst clarifying questions.Source: §4.2 · p. 10

4.2.1 Prompt Engineering Strategy

  • Among zero-shot, few-shot, and chain-of-thought prompting, CoT achieves the best precision-recall balance and is adopted as the default.Few-shot examples include requirement-assumption pairs; CoT examples add explicit reasoning traces.Source: §4.2.1 · p. 10

4.3 Layer 3: Fine-Tuned BERT NFR Classifier

  • Layer 3 classifies each assumption with fine-tuned BERT using the concatenated requirement and assumption as context.Source: §4.3 · p. 11

4.4 Layer 4: LLM Clarification Suggestion Generation

  • Layer 4 generates three to five ranked, actionable clarification options for each classified assumption, with plain-English rephrasing available for non-technical stakeholders.Source: §4.4 · p. 11

4.5 Layer 5: Analyst Validation Loop

  • Layer 5 lets analysts select a suggested answer, enter a relevance- and coherence-checked free-text answer, or request a layman rephrasing.Source: §4.5 · p. 12

4.6 Layer 6: Requirement Rewriter

  • Layer 6 replaces ambiguous terms with analyst-confirmed resolutions through boundary-aware matching, producing an explicit requirement.The example rewrites “quickly” as a 200 ms 95th-percentile threshold under 1,000 concurrent users.Source: §4.6 · p. 12

5. Dataset and Annotation

The HiddenLens Corpus (HLC) provides annotated requirements for hidden-assumption research, with a workflow that samples and labels requirements across six NFR categories. The dataset is split into training, validation, and test sets.

5. Dataset and Annotation

  • HLC comprises 312 requirements containing 847 annotated assumptions across six NFR categories.The categories are performance, security, scalability, usability, reliability, and domain.Source: §5 · p. 13
  • The annotation workflow shuffles requirements, skips items with no assumptions or a full category, and caps each NFR category at 60 items.The workflow sends requirements to a language model and parses its inferred category and assumptions.Source: §5 · pp. 13–14
  • The dataset uses stratified 70%/15%/15% training, validation, and test splits that preserve category proportions.Classifier samples are requirement-assumption pairs; T5 fallback samples pair requirements with structured assumption strings.Source: §5 · p. 14

6. Experimental Setup

The experimental setup specifies HiddenLens’s component configurations and three baselines, then evaluates it using held-out annotations and RE practitioners. Automated evaluation covers classification and assumption detection, while practitioners rate assumptions on three criteria.

6.1 Model Configuration

  • Detection uses temperature 0.2 for factual consistency, while clarification generation uses 0.7 to increase lexical diversity in analyst options.BERT-base-uncased uses a 256-token input length, a 2e-5 learning rate, and early stopping on validation F1.Source: §6.1 · p. 15

6.2 Baseline Methods

  • The baselines are a vague-term keyword system, QuARS extended with assumption-class labels, and a zero-shot LLM without pre-filtering, CoT traces, or few-shot examples.The zero-shot comparison is intended to isolate prompting and architectural integration choices.Source: §6.2 · p. 15

6.3 Evaluation Metrics

  • Evaluation combines automated metrics against held-out annotations with human evaluation by requirements-engineering practitioners.Source: §6.3 · p. 15

6.3.1 Automated Metrics

  • Layer 3 reports macro-averaged Precision, Recall, and F1; Layer 2 counts an assumption as correct when ROUGE-L overlap with a gold annotation is at least 0.4.The overlap criterion accepts semantic rather than verbatim matches.Source: §6.3.1 · pp. 15–16

6.3.2 Human Evaluation

  • Practitioners who did not participate in annotation rated test-split assumptions on three criteria using a five-point scale.The criteria are depicted in Table 6.Source: §6.3.2 · p. 16, Table 6 · p. 16

7. Results and Discussion

HiddenLens with CoT improves assumption detection over the reported baselines, while its classifier performs well across NFR categories. The paper contrasts the task with prior work and illustrates how vague terms become assumptions and clarification questions.

7.1 Quantitative Results

  • F1 of 0.79 on assumption detection represents a 71% relative improvement over the keyword-based baseline and a 61% improvement over QuARS-Adapted.BERT attention pre-filtering alone contributes approximately 3 F1 points in the reported zero-shot comparison.Source: Table 7 · p. 16, §7.1 · p. 16, Figure 12 · p. 17
  • Macro F1 Score of 0.847 on the held-out test split is highest for Security (F1 Score 0.89) and lowest for Domain (F1Score 0.79).The paper attributes the lower Domain result to that category’s breadth and context-dependence.Source: §7.1 · p. 16

7.2 Comparative Analysis with Prior Work

  • Prior work addresses requirements smells, NFR classification, and defect prediction, but none of the cited works targets implicit content as its main object of analysis.The comparison distinguishes HiddenLens’s focus on implicit omissions from work on defective explicit content.Source: §7.2 · p. 17, Table 8 · p. 17

7.3 Input and Output - HiddenLens

  • Examples span reliability, performance, and security, showing vague terms alongside inferred assumptions and analyst clarification questions.The examples illustrate the move from a vague word to a concrete, answerable specification gap.Source: §7.3 · p. 18, Table 9 · p. 18

8. Case Study – Hospital Patient Management System

The hospital case study processes natural-language requirements containing vague terms that lack measurable acceptance criteria. HiddenLens turns the assumptions behind those terms into analyst-validated specifications with concrete thresholds, standards, and scope.

8. Case Study – Hospital Patient Management System

  • Across six layers, HiddenLens identifies vague terms, hypothesizes assumptions, classifies them into NFR categories, generates clarification suggestions, incorporates analyst validation, and rewrites the requirements.For example, “quickly” prompts a suggested response time of 200 ms under normal load and 500 ms under peak load.Source: §8 · p. 18
  • The hospital requirements use six fuzzy terms, including “quickly,” “real time,” and “regularly,” without measures, scope, or testable acceptance criteria.Without a shared verification point, developers would determine their own criteria for those terms.Source: §8 · p. 18
  • The rewritten requirements specify response times with load conditions, AES-256 and TLS 1.3, RBAC with audit logging, a three-second notification SLA with named roles, and scoped scheduling.Scheduling is bounded by staff volume and a conflict-resolution deadline.Source: §8 · pp. 18–19
  • The rewrite meets ISO/IEC/IEEE 29148 criteria for unambiguity, verifiability, and completeness without losing original requirement intent.The process resolves hidden assumptions rather than replacing stakeholder decisions.Source: §8 · p. 19

9. Threats to Validity

The threats to validity span subjective annotation, the scope of the 312-requirement corpus, approximate evaluation criteria, and the possibility of unnecessary LLM-generated assumptions.

9. Threats to Validity

  • Annotation subjectivity remains a risk: agreement on assumption presence was κ = 0.74, and systematic annotator bias could over- or underestimate NFR categories.Formal definitions reduce but do not eliminate disagreement between hidden assumptions and implicit stakeholder expectations.Source: §9 · p. 20
  • Transferability beyond the three-source, 312-requirement HLC, including to embedded systems, telecommunications, and intelligent systems, requires domain-specific validation.Industrial requirements may be more concise than scholarly examples, potentially increasing assumptions per sentence.Source: §9 · p. 20
  • ROUGE-L overlap ≥ 0.4 is only an approximate true-positive criterion and can count semantically identical assumptions as false negatives when specialized terms differ.This risk applies when gold assumptions use domain-specific terminology absent from the predicted wording.Source: §9 · p. 20
  • LLMs may generate assumptions that seem logical but are unnecessary for implementation, leaving analysts responsible for validation through Layer 5.The validation loop is described as a safety valve, not an automatic guarantee against such assumptions.Source: §9 · p. 21

10. Conclusion and Future Work

HiddenLens is presented as a six-layer framework for resolving hidden assumptions, with reported results on the HLC and a goal of making tacit engineering judgment explicit and testable. The conclusion also identifies future integration work and provides data and manuscript-preparation disclosures.

10. Conclusion and Future Work

  • HiddenLens combines BERT attention entropy, LLM reasoning, fine-tuned BERT NFR classification, context-aware suggestions, and human validation to detect, classify, question, and resolve hidden assumptions.The paper frames these implicit beliefs as drivers of implementation design choices.Source: §10 · p. 21
  • On 312 HLC requirements, HiddenLens achieved an F1 score of 0.79 for assumption identification and 0.847 for BERT classifier-based NFR classification.The paper also describes a Hospital Management System case study as an illustration of relevance to safety-critical systems.Source: §10 · p. 21
  • The authors aim to make tacit engineering judgment an explicit, testable artifact by closing the gap between what a requirement states and what it silently assumes.They propose future integration with IDEs and requirements-management tools to investigate influence on projects over time.Source: §10 · p. 21
  • The HiddenLens Corpus and study code are stated to become publicly available in a GitHub repository upon manuscript acceptance.Source: §10 · p. 21

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

Related papers

Loading 2610.29996v1…