Source-linked AI summary
Think-Probe-Respond: Improving Large Language Models as Judges of Research Idea Novelty
Tim Schopf, Tobias Schreieder, Akiko Aizawa
TL;DR
Manual novelty judgment is difficult to scale, and LLMs can generate human-like reasoning while their final novelty scores diverge from human judgments and favor medium novelty. The paper introduces TPR, which probes hidden states during reasoning and conditions the final response on the extracted judgment. Across strong baselines, TPR improves performance by 22.30% and reduces medium-novelty bias.
Problem
Manual novelty judgment requires substantial expertise and literature knowledge, while LLM final judgments can diverge from human judgments despite human-like rationales.
Method
TPR probes latent novelty judgments from hidden states during reasoning and uses them to condition the final response, with frozen LLM parameters and a lightweight probing classifier.
Results
22.30%: TPR improves novelty judgment performance over strong baselines and reduces the prevalent medium-novelty bias.
Takeaways & Limitations
TPR yields less biased and more accurate quantitative novelty judgments by exploiting novelty beliefs expressed in model reasoning.
Takeaways & Limitations
Evaluation focuses on machine learning ideas in RINoBench, and TPR requires hidden-state access unavailable for direct use with closed-source LLMs.
Abstract
from arXiv · showhide
Automated novelty judgment can accelerate scientific discovery by enabling efficient evaluation, refinement, and comparison of research ideas. While large language models are increasingly adopted for this task, we investigate a previously overlooked limitation in their judgment capabilities: despite generating reasoning rationales that closely mirror those of human experts, their final novelty judgments often diverge substantially. We demonstrate that this miscalibration stems from a systematic bias towards judging ideas as "medium novel". To mitigate this, we propose Think-Probe-Respond (TPR), a lightweight approach that probes latent novelty judgments from hidden states during the reasoning phase and uses the probed judgments to condition the final response. Across strong baselines, TPR improves novelty judgment performance by 22.30% and successfully mitigates the prevalent "medium novelty" bias.
1 Introduction
Automated novelty judgment is valuable but difficult to scale manually, and LLMs can produce human-like rationales while final judgments diverge from human assessments. TPR addresses this miscalibration by probing latent novelty beliefs and conditioning final responses on them.
- Manual novelty judgment requires substantial expertise and broad literature knowledge, making it time-consuming, subjective, and difficult to scale.
- LLMs are increasingly used to judge research idea novelty as scientific output continues to grow.
- Despite producing plausible, human-like novelty arguments, LLMs’ final judgments remain disconnected from their reasoning rationales and misaligned with human judgments.
- 22.30%: TPR improves novelty judgment performance on average over strong baselines while producing less biased judgments.TPR probes latent novelty beliefs from hidden states during reasoning and conditions the final response on those extracted beliefs.
2 Related Work
Automated novelty judgment methods have progressed from citation, lexical, and embedding approaches toward LLM-based methods. Figure 1 introduces TPR’s parameter-freezing and parameter-training design choices.
- Citation- and lexical-based methods evolved toward semantic embeddings, but these approaches largely remain limited to surface-level similarity estimation.
- Figure 1 marks frozen parameters with a snowflake and trainable parameters with a flame.
3 Benchmark
RINoBench is a publicly available benchmark for research idea novelty judgment built from expert-authored ideas, related works, novelty scores, and written justifications. Models predict scores from 1 to 5 and justify them through comparisons with prior work.
- RINoBench contains 1,381 expert-authored ideas paired with related works, five-point human novelty scores, and expert-written justifications.
- Models receive a research idea and related work, then predict novelty from 1 (not novel) to 5 (highly novel) and generate a comparison-grounded justification.
4 Miscalibration in LLM Novelty Judgments
LLMs often reason about novelty in ways that resemble human analysis but express biased numerical judgments. Their predictions concentrate on middle categories, revealing a mismatch between latent beliefs and final outputs.
- Six state-of-the-art LLMs perform poorly, with MAE values around one and consistently low macro-F1 scores; the best model achieves only 17.1.
- LLMs Avoid Extreme Novelty Judgments: Models predominantly predict novelty classes 3 and 4, while the lowest and highest categories are rarely predicted correctly.
- Latent Belief vs. Expressed Novelty Judgment: Models achieve high recall for human-annotated justification arguments, often identifying the same overlaps, differences, and novelty aspects as experts.
- Takeaway: The results characterize miscalibration as a mismatch between latent novelty beliefs and expressed judgments biased toward safe middle categories.
5 Probing Latent LLM Judgments
TPR probes novelty beliefs from hidden states during reasoning, then conditions the final response on the extracted judgment. Across open-source LLMs, it improves macro-F1, remains robust across models and sizes, and produces more balanced novelty-class predictions.
- TPR Method: TPR uses three stages: qualitative reasoning, hidden-state probing with logistic regression, and response generation conditioned on the predicted novelty class.The method keeps LLM parameters frozen and trains only a lightweight probing classifier.
- Experimental Setup: TPR probes the final hidden layer because it offers strong probing performance and is easier to extract in common open-LLM frameworks than earlier layers.The choice follows prior work cited by the paper.
- Evaluation Results: 22.30% is TPR’s average improvement over the best competing approach in macro-F1 across evaluated LLMs.TPR also surpasses the computationally expensive FineTune approach.
- Evaluation Results: TPR delivers strong, model-agnostic performance across all investigated LLMs, including reasoning and non-reasoning models.The evaluated models include Qwen3, GPT-OSS-20B, Gemma 3, and Llama 3.1 families.
- Evaluation Results: TPR produces more balanced predictions across novelty classes and recognizes very low and very high novelty ideas more often than prompting.Qwen3 remains relatively unlikely to assign class 1 even under TPR.
6 Probing over Time
The study probes last-layer hidden states throughout thinking and response generation to identify when novelty information is most reliable. Novelty signals generally peak at the final thinking token, while response generation can dilute them.
- Probing over Time: The analysis probes last-layer hidden states at multiple time steps during both thinking and response generation.Table 3 compares first, last, and intermediate tokens in each phase.
- Novelty Signals: The final thinking token, t_n, yields the strongest or near-strongest novelty judgment performance across nearly all models.This pattern holds for both reasoning and non-reasoning models.
- Novelty Signals: Earlier thinking probes generally perform substantially worse, indicating that novelty representations emerge progressively during reasoning.The comparison includes intermediate points such as t25% and t50%.
- Reasoning Length: Probing at t_n remains consistently strong across models with widely varying numbers of thinking tokens.The paper reports no clear correlation between absolute reasoning-chain length and probing performance.
- Novelty Signals: Response-generation representations become increasingly influenced by surface realization and linguistic planning, diluting the underlying novelty signal.The final thinking token remains the most reliable and stable probing point overall.
7 Conclusion
The paper finds that LLMs’ novelty judgments are miscalibrated: their rationales may align with human reasoning, while final judgments favor medium novelty. TPR probes latent judgments during reasoning to improve alignment and reduce this bias.
- LLMs’ rationales often align with human reasoning, but their final research-idea novelty judgments are biased toward medium novelty.
- TPR probes latent novelty judgments from hidden states during reasoning and conditions the final response on the probed judgment.
- 22.30% improvement over strong baselines demonstrates TPR’s gain in novelty judgment performance while reducing the prevalent medium-novelty bias.
8 Limitations
The evaluation uses RINoBench, which focuses on machine learning research ideas and may not represent novelty judgments in other scientific domains. TPR also requires hidden-state access, limiting direct use with closed-source LLMs, while expert annotations remain subjective.
- RINoBench focuses on machine learning research ideas and may not fully reflect novelty judgments in other scientific domains.
- TPR requires access to model hidden states, limiting its direct applicability to closed-source LLMs.
- Novelty judgments are inherently subjective, and expert annotations may reflect individual preferences or incomplete literature knowledge.
9 Ethical Considerations
The paper frames automated novelty judgment as support for research and education rather than a replacement for human expertise. Its probing approach compares information encoded during reasoning with information available during response generation.
- Novelty judgments are inherently subjective and should not be used for formal or high-stakes decisions.
- Automated novelty predictions should support, rather than replace, human judgments of research ideas.
- The paper compares representations during reasoning and response generation, finding more novelty-judgment information during thinking.
- RINoBench evaluates numerical novelty scores and textual justifications, including overlaps with prior work and new contributions.
- The experiments use hidden-state extraction, probing classifiers, and LoRA-based fine-tuning across specified computational infrastructure.
E On the Choice of Novelty Score Metrics
The paper uses macro-F1 as its primary novelty-score metric because MAE can favor conservative middle predictions on the five-point scale. Macro-F1 instead evaluates novelty classes more evenly, including extreme categories.
- Macro-F1 is the primary metric because MAE is less informative when predictions concentrate around the middle of the 1–5 novelty scale.The paper still reports MAE for completeness in Table 6.
- A model repeatedly predicting score 3 can achieve deceptively low MAE even when its novelty judgments are less accurate overall.
- Macro-F1 treats all novelty classes equally and rewards correct predictions for low, medium, and high novelty judgments.
F Justification Evaluation
Using TPR, open-source LLMs generate textual justifications that remain broadly aligned with human rationales, with reasoning-capable models approaching proprietary-model alignment and competitive recall.
- TPR evaluates textual justifications generated by open-source LLMs in addition to numerical novelty prediction.The evaluation is reported in Table 7, while Table 8 illustrates alignment and miscalibration cases for Claude Opus 4.5.
- Alignment: ALI values around 0.5 show that the strongest reasoning-capable open-source models approach the alignment range observed for Claude and GPT under zero-shot prompting.
- Recall: Smaller open-source models achieve relatively high recall, with results slightly below top-performing OpenAI models but competitive with Gemini Pro.
- Additional Ratio: Reasoning-capable models generally produce more elaborate justifications than non-reasoning models, although OpenAI models reach higher Additional Ratios.The best-performing Qwen3 models remain competitive with Gemini Pro on this measure.
- Evaluation Setup: The zero-shot setup supplies the research idea, related works, and RINoBench novelty rubric before requesting a numerical score and textual justification.
- Evaluation Setup: TPR instructions include step-by-step reasoning only for models that do not generate think tokens by default.