PaperlayerView exact revision on arXiv

Source-linked AI summary

Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search

Jimmy Lin, Sahel Sharifymoghaddam, Lingwei Gu, Nour Jedidi

arXiv:2610.11922v1cs.IRcs.CL

TL;DR

The paper asks whether a competitive pointwise decoder-only reranker can be built from scratch with commonly available data and a handful of GPUs, without open-weight backbones. The authors pre-train a decoder-only reranker from scratch and then fine-tune it with supervised learning, using commonly available datasets and no existing open-weight backbone. The authors report that their from-scratch, supervised-fine-tuned pointwise reranker is competitive, establishing an initial milestone toward their broader thesis.

  • Problem

    The paper asks whether a competitive pointwise decoder-only reranker can be built from scratch with commonly available data and a handful of GPUs, without open-weight backbones.

    Source: §1 · p. 1

  • Method

    The authors pre-train a decoder-only reranker from scratch and then fine-tune it with supervised learning, using commonly available datasets and no existing open-weight backbone.

    Source: §1 · p. 1

  • Results

    The authors report that their from-scratch, supervised-fine-tuned pointwise reranker is competitive, establishing an initial milestone toward their broader thesis.

    Source: §1 · p. 1

  • Takeaways & Limitations

    A small model trained end to end can provide educational value that is difficult to obtain by fine-tuning a more capable model whose development is inaccessible.

    Source: §3.5 · p. 10

    A competitive reranker does not establish a path to a competitive agent, and some agentic-search aspects may still require non-open models.

    Source: §9 · p. 30

Abstract

from arXiv · show

Project Greenhouse represents our exploration of a simple thesis: We believe that it is possible to build fully open and sovereign models for agentic search with only modest computational resources. As a first milestone, we describe how to build a competitive pointwise decoder-only reranker using a simple two-step recipe comprising pre-training from scratch followed by supervised fine-tuning, starting only from commonly available datasets. Contrary to the dominant approach in the literature, we do not rely on existing open-weight backbones from third parties, and thus we are fully in control of model training, from end to end. We were able to accomplish the bulk of our experiments using no more than a handful of GPUs. This report articulates the importance and benefits of our approach, and we share artifacts that enable transparent, independent reproduction of all aspects of model training. Beyond data, code, and configurations that capture our efforts, we also release checkpoints for our family of Gaggle models, demonstrating the feasibility of our approach and providing a first step toward validating our broader thesis.

1 Introduction

Project Greenhouse frames the feasibility of fully open and sovereign agentic-search models as its thesis, with a competitive pointwise reranker as its first milestone. It emphasizes end-to-end control, reproducibility, and incremental progress.

1 Introduction

  • A competitive pointwise decoder-only reranker is built by pre-training from scratch and then supervised fine-tuning on commonly available datasets, without an existing open-weight backbone.The bulk of the experiments used only a handful of GPUs.Source: §1 · p. 1
  • The released Gaggle checkpoints and reproduction artifacts make the work an initial existence proof and foundation for further exploration.The artifacts include code, data, and configurations.Source: §1 · p. 1
  • The project scopes agentic search and full openness through definitions in later sections, and presents this report as one milestone in a longer-running effort.Its intuitive account of full openness emphasizes publicly available artifacts for data, code, and configurations.Source: §1 · p. 2
  • The stated contributions include a directed property hypergraph for characterizing openness and sovereignty, a reranker as a stepping stone, and decoder-only Gaggle models.The hypergraph formulation is proposed as extendable to an open ecosystem and distributed research efforts.Source: §1 · p. 2

2 Toward Agentic Search

The report scopes agentic search as information seeking with tools and presents pointwise reranking as a focused, useful first component. Its rationale combines incremental development with a simple interface, standalone utility, and potential roles in broader search agents.

2 Toward Agentic Search

  • Agentic search is framed as models using search tools in loops to find, assess, and synthesize information for user requests.The scope includes heterogeneous sources and may involve non-search tools as well.Source: §2.1 · p. 2, §2 · p. 2

2.1 Scoping Our Efforts

  • The effort focuses on information seeking rather than frontier-model capabilities such as coding or general reasoning, while leaving system architecture open.The authors favor incremental components and allow that early systems may combine open and proprietary components.Source: §2.1 · pp. 2–3

2.2 Pointwise Reranking

  • A pointwise reranker scores each query–document pair independently, then sorts the initial candidate list to seek better ranking quality.The milestone is pointwise rather than pairwise or listwise reranking.Source: §2.2 · p. 3, §2.3 · p. 3
  • A fully open and sovereign decoder-only pointwise reranker competitive with open-weight alternatives is the report’s first project milestone.The authors share the artifacts needed to reproduce the work end to end.Source: §2.2 · p. 3

2.3 Why Pointwise Reranking?

  • Independent query–document scoring supports rapid iteration and checkpoint evaluation without repeatedly processing the full corpus.The authors say this setup helps isolate changes to data, recipes, architectures, and configurations, and is more tightly scoped than training embedding or agentic-search models.Source: §2.3 · p. 4
  • The same query-and-candidate interface lets rerankers work with varied retrieval sources and prioritize results when search agents receive more content than they can inspect.The authors also describe rerankers as potential relevance judges for search agents, including agents using weaker tools.Source: §2.3 · p. 4
  • The authors present reranking as immediately useful and as a stepping stone toward more sophisticated agentic-search models.Planned directions include listwise reranking, query reformulation, relevance feedback, and steering search trajectories.Source: §2.3 · p. 5

3 Fully Open and Sovereign Models

The section defines fully open and sovereign models through public, non-proprietary training provenance, distinguishing openness and sovereignty from reproducibility while describing their relevance to organizational control, academic inquiry, and collaboration. It frames agentic search as a deliberately scoped initial contribution that can make progress with modest computational resources rather than compete across broad capabilities.

3.1 LLM Training as Hypergraphs

  • A fully open and sovereign model has publicly available, non-proprietary upstream artifacts and computational steps, so its weights can be independently derived.Under this definition, open-weight models with unavailable pre-training corpora or recipes do not qualify, although they can provide useful backbones; the authors say such backbones are not necessary for a competitive reranker.Source: §3.1 · pp. 5–6

3.2 The LLM Training Ecosystem

  • A machine-readable, aggregated training hypergraph could trace model and dataset provenance across researchers and help agents identify, propose, and execute feasible experiments.Automatic evaluation could let agents grow the graph toward a specified goal, potentially supporting automated AI scientists within a narrow scope.Source: §3.2 · p. 6

3.3 Other Considerations

  • A fully public ecosystem lets people use available artifacts directly, trace them upstream, or branch training recipes, while openness and sovereignty remain distinct from reproducibility.Reproducibility is assessed separately: identical outputs are not required, and traversal should instead reproduce experimental claims substantively within reasonable variation. For synthetic data or teacher guidance involving non-open models, the report leaves compatibility unresolved and identifies one-time use versus continued access during training as a possible consideration.Source: §3.3 · p. 7

3.4 Why is this important? The Organizational Perspective

  • End-to-end provenance can help organizations control model lifecycles, while open-weight checkpoints may leave unresolved dependencies and risks around data, alignment, and future releases.Incomplete provenance can make licensing, privacy, quality, alignment, and backdoor risks difficult to assess; reliance on successive third-party checkpoints can also constrain experiments or leave organizations without a suitable path forward.Source: §3.4 · pp. 8–9

3.5 Why is this important? The Academic Perspective

  • Access to training data and transparent interventions supports rigorous study of model behavior, while end-to-end training provides educational opportunities for experimentation and collaboration.Researchers can remove problematic corpus or fine-tuning examples and retrain, though output quality is not guaranteed; students can inspect decisions, investigate reproduction failures, and share intermediate artifacts across projects and institutions.Source: §3.5 · pp. 9–10

3.6 Openness and Sovereignty

  • Openness and sovereignty concern resource availability and control, respectively, and neither guarantees the other.The authors distinguish these concepts from reproducibility, which concerns whether a traversal works as claimed; they allow that repeated training may yield non-identical weights while reproducing benchmark claims substantively. They leave open whether synthetic data or teacher guidance involving non-open models is compatible, noting the distinction between one-time use and continued access during training.Source: §3.3 · pp. 6–7, §3.6 · p. 10
  • Rather than compete with industry labs across capabilities such as reasoning, coding, and drug discovery, the work focuses on agentic search as a meaningful, intentionally scoped contribution.The authors argue that openness and sovereignty support rigorous understanding and educational opportunities, while modest computational requirements aim to make those opportunities feasible for others.Source: §3.6 · p. 11

4 Headline Results

Greenhouse’s first milestone builds a pointwise reranker from scratch using public data, then evaluates its effectiveness against retrieval and reranking alternatives on TREC DL and BEIR.

4 Headline Results

  • Gaggle is a 3B-parameter pointwise reranker pre-trained from scratch and fine-tuned on public relevance annotations, using a two-step recipe without an existing open-weight backbone.The authors report that the recipe required no intermediate instruction-tuning or alignment, mid- or post-training, teacher supervision, synthetic data, or SFT hyperparameter tuning.Source: §4 · p. 11, §4.2 · p. 14

4.1 Experimental Setup

  • Evaluation uses nDCG@10 on TREC DL 2019–2023 and seven BEIR collections, reranking top-100 BM25 candidates and comparing with an oracle and other rerankers.Scores are unweighted means within each benchmark group; four BEIR collections are out of domain relative to RLHN-250K fine-tuning data.Source: §4.1 · pp. 11–12

4.2 Reranker Effectiveness and Implications

  • Gaggle improves over BM25 on every collection and exceeds the mean nDCG@10 of all compared fine-tuned pointwise and listwise rerankers.On TREC DL, only GPT-6.1 Sol scores higher overall; on BEIR, Gaggle is comparable to Gemma-4 but appears lower than Qwen3.8 and GPT-6.1 Sol.Source: Table 1 · p. 13, Table 2 · p. 13, §4.2 · p. 14
  • Collection-level results vary: Gaggle scores highest among reported non-oracle systems on DL19 and beats GPT-6.1 Sol on TREC-COVID.It is comparable to Sol on SCIDOCS under the noise guideline; Sol scores higher on the other BEIR collections, while Gaggle is among the strongest on TREC-News as well.Source: §4.2 · p. 14
  • The comparisons establish feasibility but cannot isolate the effects of model size, data mixtures, or training and inference procedures.Reported run-to-run variability is about 0.001 for benchmark means and 0.003 on average for individual datasets, and the oracle gap may reflect either reranking headroom or imperfect relevance judgments.Source: §4.2 · p. 14

5 Pre-Training

The authors pre-trained causal and bidirectional checkpoints from scratch, then evaluated them against existing base models. Their causal model scored much higher than nanochat-d34, while the comparison does not isolate the effect of any single training factor.

5.1 Experimental Setup

  • The 3.29B-parameter backbone uses a 34-layer decoder-only transformer with value embeddings in alternating layers and mixed sliding-window and full-context attention.Value-embedding lookups make its per-token compute comparable to a dense model of about 2B parameters.Source: §5.1 · p. 15
  • Pre-training used one pass over 287B ClimbMix tokens, with batch size doubled three times while learning rates stayed fixed.Training used 8 H100 GPUs and consumed roughly 1,600 GPU-hours.Source: §5.1 · p. 15
  • The bidirectional variant starts from the causal checkpoint at 245B tokens and continues for 42B tokens with masked next-token prediction.It allows attention to context on both sides while predicting masked tokens from the preceding position.Source: §5.1 · p. 15

5.2 Model Evaluation

  • Base-model evaluation covers CORE, MMLU, and MMLU-Pro, with local scoring protocols adjusted for context length and bidirectional answer masking.The authors report that their CORE implementation was within 0.011 of published scores on seven reference models and their MMLU scores within 0.003 of the stock harness.Source: §5.2 · p. 16

5.3 Released Artifacts

  • They release the causal checkpoint selected at 283B tokens and a bidirectional variant, along with modified attention code for inference.The causal checkpoint had the highest CORE score; the final 287B-token model scored 0.004 lower.Source: §5.2 · p. 16, §5.3 · p. 17

5.2 Model Evaluation

  • Our causal model scores much higher than nanochat-d34, though most listed models were trained on 1.7T–34T tokens or have unreported budgets.The models differ in data, architecture, and training recipe, so the comparison places the checkpoint among existing models rather than isolating a single factor.Source: §5.2 · pp. 16–17, Table 3 · p. 17

6 Supervised Fine-Tuning

The section describes fine-tuning a pre-trained checkpoint into a pointwise reranker and averaging four training runs into the headline model. The soup slightly exceeded the individual-run averages, with gains close to the observed run-to-run noise.

6.1 Experimental Setup

  • Gaggle scores each query–passage pair independently by subtracting the ␣false next-token logit from the ␣true logit.The reranker retains the pretrained output rows for these two tokens and adds no classification head.Source: §6.1 · p. 18
  • Fine-tuning uses RLHN-250K, whose curation relabels verified false negatives as positives, with localized contrastive estimation over one positive and sampled hard negatives.The authors also report that their comparison with pointwise cross-entropy supports this choice under the tested configuration.Source: §6.1 · p. 18

6.2 Run-to-Run Variability and Model Averaging

  • The headline checkpoint is a model soup formed by averaging the final parameters from four runs with equal weight.All runs began from the same causal checkpoint and used the same RLHN-250K recipe, with differences in seed and execution settings.Source: §6.2 · p. 19
  • Four individual runs had mean nDCG@10 of 0.640±0.001 on TREC DL and 0.544±0.001 on BEIR.Across 12 collections, the average standard deviation was 0.003, which the authors use as their guideline for interpreting experimental noise.Source: §6.2 · pp. 19–20
  • 0.641 on TREC DL and 0.547 on BEIR: the soup model's mean nDCG@10 exceeded individual-run averages by 0.001 and 0.003, respectively.Its TREC DL mean matched the strongest individual run, and its BEIR mean exceeded all four; both gains were close to observed run-to-run noise.Source: Table 4 · p. 19, §6.2 · p. 20

7 Contrastive Conditions

The section compares Gaggle’s from-scratch reranker with open-weight backbones under a shared fine-tuning recipe, then tests whether lowering the learning rate changes Gemma and MiniCPM’s results. The retuning improves their reranking outcomes, while the report maintains that its own checkpoint is sufficient for a competitive reranker.

7 Contrastive Conditions

  • Comparisons used baseline run 1 of Gaggle, excluded the model soup as methodologically unfair, and treated differences near 0.001 in benchmark means or 0.003 per collection as noise.The authors used unrounded evaluation scores.Source: §7 · p. 20

7.1.1 Experimental Setup

  • The backbone comparison applied the same supervised fine-tuning recipe to nanochat-d34, Gemma-4-E2B, Qwen3.5-4B-Base, and MiniCPM5-2B-Base.Runs shared causal attention, the standard prompt, LCE loss, and the last non-padded token for scoring; Gemma and MiniCPM were also rerun at 1×10−5 after 5×10−5 proved too high.Source: §7.1.1 · p. 20
  • The multimodal backbones retained their native architectures and tokenizers, but their vision encoders—and Gemma’s audio encoder—were discarded for fine-tuning.This left approximately 4.6B trainable parameters for Gemma and 4.2B for Qwen3.5 under the paper’s counting convention.Source: §7.1.1 · p. 20

7.1.2 Results and Discussion

  • With the shared recipe, Gaggle’s baseline had higher mean nDCG@10 than the other tested backbones on both TREC DL and BEIR, beyond the run-to-run noise guideline.Table 5 reports mean nDCG@10 for backbone configurations reranking top-100 BM25 candidates.Source: §7.1.2 · p. 20, Table 5 · p. 21
  • At the shared 5×10−5 peak learning rate, Gemma and MiniCPM training losses rose on identical batches while Gaggle’s continued to fall.Figure 3 compares training loss at 5×10−5 and 1×10−5.Source: §7.1.2 · p. 21, Figure 3 · p. 21
  • The authors hypothesized that the shared rate produced larger relative updates for Gemma and MiniCPM because Gaggle’s weight matrices were approximately five times larger.Gaggle was pretrained with Muon; Gemma and MiniCPM were pretrained with AdamW.Source: §7.1.2 · p. 21
  • At a 1×10−5 peak learning rate, MiniCPM achieved higher BEIR effectiveness than Gaggle, while both retuned backbones reached comparable TREC DL effectiveness.The lower rate eliminated the training-loss increase; the authors say MiniCPM’s BEIR mean exceeded their noise guideline.Source: §7.1.2 · p. 21
  • The authors argue that MiniCPM’s stronger reranker does not negate their narrower claim: their own scratch-pretrained checkpoint can produce a competitive reranker and preserve end-to-end control.They describe their pretraining recipe as “good enough” for pointwise reranking.Source: §7.1.2 · pp. 21–22

7.2 The Effects of Training Data

The experiments test how training-data size and composition, attention, prompt design, and readout affect reranking. Larger RLHN training yields a small TREC DL mean gain, while RLHN-250K outperforms Tevatron on both benchmark groups; among the modeling variants, QPQ improves both benchmark means.

7.2.1 Experimental Setup

  • The larger RLHN dataset has 648,762 instances versus 247,534, increasing optimization steps from 1,931 to 5,060 under the same one-epoch recipe.The variants also change source distribution: MS MARCO rises from 55.2% to 70.2%, while Natural Questions falls from 12.0% to 8.9%.Source: §7.2.1 · p. 22
  • Tevatron provides 399,600 MS MARCO instances and 3,112 optimizer updates, versus 1,931 for RLHN-250K, but differs in composition and preparation.RLHN-250K combines seven collections and includes LLM-assisted false-negative relabeling; each dataset averages about 30 negative passages per query.Source: §7.2.1 · p. 22

7.2.2 Results and Discussion

  • 0.646 versus 0.641 mean nDCG@10 on TREC DL for larger versus smaller RLHN training; the BEIR mean changes from 0.544 to 0.545, treated as experimental noise.The TREC DL comparison does not isolate dataset size, source distribution, and additional optimization steps.Source: §7.2.2 · p. 22
  • 0.023 higher mean nDCG@10 on TREC DL and 0.074 higher on BEIR for RLHN-250K than Tevatron MS MARCO.The datasets differ in size, composition, and preparation; RLHN-250K covers two BEIR evaluation domains, while Tevatron has no BEIR training collections.Source: §7.2.2 · p. 23

7.3.1 Experimental Setup

  • The QPQ prompt repeats the truncated query before and after a passage under causal attention; Jina-Reranker-v3 uses a related query-before-and-after arrangement [Wang et al., 2025].The short context window prevents direct adoption of Jina’s multi-document recipe.Source: §7.3.1 · p. 23

7.3.2 Results and Discussion

  • Bidirectional and causal rerankers achieve comparable effectiveness with the standard prompt, and appending the mask token gives benchmark means comparable to the standard bidirectional prompt.The causal baseline suggests an alternative way to combine query and passage information; adapted models rely on bidirectional attention, but show no clear reranking advantage over the causal baseline.Source: §7.3.2 · p. 24
  • 0.647 versus 0.641 mean nDCG@10 on TREC DL and 0.551 versus 0.544 on BEIR for causal QPQ versus the standard causal prompt.Both changes exceed the noise guideline; repeating the query may provide passage-conditioned query representations at the final scoring position.Source: §7.3.2 · p. 24
  • 0.638 versus 0.643 on TREC DL and 0.538 versus 0.544 on BEIR for bidirectional mean pooling versus single-position readout; causal pooling differences are within the noise guideline.Mean pooling did not appear to improve reranking effectiveness.Source: §7.3.2 · p. 24

7.4 The Effects of Self-Filtering and Self-Distillation

The experiments test filtering noisy positives and distilling a baseline model’s scores on the filtered data. Self-filtering improves BEIR mean effectiveness, while adding self-distillation gives no clear additional benefit under the tested setup.

7.4.1 Experimental Setup

  • The baseline checkpoint flags a positive for removal when more than max(9, 0.375N) negatives score higher; queries with no remaining positive judgments are discarded.The procedure is not applied to ArguAna because its counterargument-retrieval task may not align with the model’s general relevance judgments.Source: §7.4.1 · p. 25
  • Filtering removes 14,887 positive passages and 1,484 queries, leaving 246,050 queries for retraining from the original pre-trained checkpoint.The new reranker uses the baseline run’s filtering decisions and the same LCE objective and training recipe.Source: §7.4.1 · p. 25
  • Distillation uses KL divergence to transfer the fixed teacher’s relative preferences over sampled negatives, while LCE retains supervision from positive documents.Teacher and student use τKD = 2; distillation is not performed for ArguAna.Source: §7.4.1 · p. 25

7.4.2 Results and Discussion

  • 0.548 versus 0.544 mean BEIR nDCG@10 for self-filtering versus baseline, while TREC DL remains within run-to-run noise.TREC-News increases from 0.510 to 0.529, exceeding the typical collection-level noise threshold.Source: §7.4.2 · p. 25
  • Self-distillation produces benchmark means comparable to self-filtering alone, with differences within the noise guideline and mixed collection-level changes.The results suggest useful filtering decisions from the trained checkpoint, but no clear additional reranking benefit from score supervision by a teacher of the same capability.Source: §7.4.2 · p. 26
  • The authors choose self-filtering and self-distillation with their own model variants to avoid relying on an external teacher they may not control.Alternative distillation formulations are left for future work.Source: §7.4.2 · p. 26

7.5 The Effects of Different Loss Functions

The paper compares LCE with pointwise cross-entropy under matched training exposure. LCE achieves higher mean nDCG@10 on both benchmarks in the tested decoder-based reranker recipe.

7.5.1 Experimental Setup

  • CE predicts a true or false answer token per query–document pair, while LCE retains two scoring rows and averages contrastive losses over query groups.The comparison matches sampled pairs, optimizer-step plan, initialization, prompt, optimization settings, seed, precision, and input-length limits.Source: §7.5.1 · p. 26

7.5.2 Results and Discussion

  • 0.641 versus 0.628 mean nDCG@10 on TREC DL and 0.544 versus 0.528 on BEIR for LCE versus CE.The absolute improvements of 0.013 and 0.016 exceed the run-to-run noise guideline; results support LCE for this decoder-based reranker recipe.Source: §7.5.2 · p. 26

8 Related Work

Related work frames AI sovereignty around institutional autonomy and transparency, then distinguishes Greenhouse’s narrowly scoped, reproducible training from other open-model efforts. The discussion emphasizes that its reranker can be reproduced and modified with modest compute.

8.1 AI Sovereignty

  • Concerns motivating organizational sovereignty include control of institutional knowledge and production, data exposure, and limited control over third-party inference pricing.The discussion also notes that organizations are exploring whether smaller task-specific models can be as effective as frontier models.Source: §8.1 · p. 27
  • Greenhouse makes transparency and reproducibility part of its sovereignty contribution by releasing what organizations need to inspect, reproduce, and modify the training pipeline.The authors also highlight transparency’s role in understanding model behavior and its educational value.Source: §8.1 · p. 27

8.2 Open Models and Related Efforts

  • Unlike broader frontier-model efforts, Greenhouse focuses on agentic search and makes end-to-end experimentation practical with a handful of GPUs.The authors describe this as complementary to OLMo’s transparency because researchers can modify data, test recipes, and run controlled ablations.Source: §8.2 · p. 28
  • DR Tulu targets deep research from an open-weight Qwen backbone, whereas Greenhouse’s first milestone is a narrower reranker trained without an upstream backbone whose pre-training details are unavailable.Under Greenhouse’s definition, DR Tulu’s available agent-training pipeline does not make it fully open and sovereign.Source: §8.2 · p. 28
  • The discussion contrasts Greenhouse’s resource requirements and scope with other efforts, including frontier-scale projects and Thomson’s continued training from open-weight models.The authors characterize their approach as complementary to efforts that pursue frontier capabilities.Source: §8.2 · pp. 28–29

9 Limitations

The authors frame the reranker as an initial milestone, not proof that fully open and sovereign agentic search is achievable. They also acknowledge limits in experimental coverage, generalization evidence, and the boundaries used to define sovereignty.

9 Limitations

  • A competitive reranker does not establish a path to a competitive agent, and some agentic-search functions may still require non-open or non-sovereign models.The authors specifically raise the possibility of a frontier model coordinating other search components.Source: §9 · p. 30
  • The sovereignty boundary depends on treating public datasets as trusted starting points, although their filtering, synthetic generation, annotation, and upstream dependencies may be undisclosed.The authors suggest explicitly declaring provenance boundaries as a possible refinement.Source: §9 · pp. 30–31
  • Pre-training from scratch is not claimed as uniquely necessary; fine-tuning an existing fully open backbone might offer the same transparency at lower cost or better effectiveness.The authors identify comparisons with other open backbones as future work.Source: §9 · p. 31
  • The report leaves many data and training conditions unexplored, balancing experimental breadth against completing its milestone.The authors invite community contributions and propose the training hypergraph as a coordination mechanism.Source: §9 · p. 31
  • Out-of-domain experiments were less thorough than the authors wanted, leaving generalization and possible overfitting as admitted weaknesses.They plan to examine out-of-domain generalization in subsequent work.Source: §9 · p. 31
  • The authors target competitive rather than best-in-class effectiveness and acknowledge that their individual methods are not particularly novel.They describe the combination of familiar methods as the distinctive aspect of the work.Source: §9 · pp. 30–31

10 Conclusion

The report presents a first Greenhouse milestone: a competitive pointwise reranker trained from scratch and then fine-tuned, without a third-party open-weight backbone. Its modest compute needs are intended to make reproduction and further work feasible.

10 Conclusion

  • The milestone is a competitive pointwise decoder-only reranker built through pre-training from scratch followed by supervised fine-tuning on commonly available datasets.The authors state that they do not rely on third-party open-weight backbones and retain end-to-end control of training.Source: §10 · p. 31
  • Modest computational requirements make it feasible for others to reproduce and build on the reported results.Source: §10 · p. 31
  • The authors invite academic groups, organizations, and independent researchers to join the ongoing project.Source: §10 · p. 31

A Listwise Reranking Prompt

The appendix documents the listwise reranking prompt used for three baselines and describes how templates and passage windows were assembled for each model.

A Listwise Reranking Prompt

  • Figure 4 reproduces the RankLLM prompt used for the Gemma-4, Qwen3.8, and GPT-6.1 Sol listwise reranker baselines.The prompt was taken from Sharifymoghaddam et al. [2025].Source: A Listwise Reranking Prompt · p. 39
  • The implementation substitutes each window’s passage count, query, and passage identifiers and texts into the prompt template.The user message concatenates the prefix, one body entry per passage, and the suffix.Source: A Listwise Reranking Prompt · p. 39
  • The models used different template handling: Qwen3.8 used the shown system message, Gemma-4 omitted the Qwen-specific line, and GPT-6.1 Sol received the assembled prompt as one user message.For GPT-6.1 Sol, tools, extensions, skills, and automatic context-file loading were disabled.Source: A Listwise Reranking Prompt · p. 39

B Jev Shared-Rubric Prompt

Jev’s TREC DL reranking request combined the query, top-100 BM25 candidates, and a four-level TREC relevance rubric. Passages were ranked by expected relevance grade, with ties preserving BM25 order.

B Jev Shared-Rubric Prompt

  • Jev received the query, all top-100 BM25 candidates in retrieval order, and the four-level TREC relevance-grade rubric in one request.The request included one score question per passage.Source: B Jev Shared-Rubric Prompt · p. 39
  • Passages were ranked by the API’s returned expected relevance grade, with score ties retaining the original BM25 order.The expected grade was computed from the probabilities assigned to the four relevance grades.Source: B Jev Shared-Rubric Prompt · p. 39
  • Jev used the same query and passage truncation limits as Gaggle (base reranker): 128 and 512 tokens, respectively, with cl100k_base tokenization.Source: B Jev Shared-Rubric Prompt · p. 39

C Supervised Fine-Tuning Configuration

This appendix documents supervised fine-tuning and evaluation settings, including baseline runs, variants, hardware, and prompt formats. It also records the common optimization recipe and evaluation harness used for comparisons.

C Supervised Fine-Tuning Configuration

  • The soup checkpoint was produced by parameter averaging the four individual causal rerankers and used for the headline results.The final checkpoint was used for each baseline evaluation; soup averaging required no further optimization.Source: C Supervised Fine-Tuning Configuration · pp. 39, 42
  • Causal Gaggle rerankers initialized from the climbmix causal checkpoint, while MNTP rerankers used the bidirectional checkpoint; MNTP names backbone adaptation, not a relevance objective.Source: C Supervised Fine-Tuning Configuration · p. 39
  • Across 247,534 baseline instances, each group paired one positive passage with 7, 15, or 23 hard negatives and contributed one joint contrastive loss.The negative counts occurred in 1.93%, 45.85%, and 52.21% of instances, respectively.Source: C Supervised Fine-Tuning Configuration · p. 39
  • The appendix includes zero-shot listwise prompts, RankLLM’s ranking instructions and rank-list output format, and a Jev prompt using the four-level TREC rubric.The listwise prompt figure notes that Gemma-4 and GPT-6.1 Sol omitted the opening /no_think directive.Source: Figure 4 · p. 40, C Supervised Fine-Tuning Configuration · p. 40, Figure 5 · p. 41
  • Gradient accumulation targeted 128 query groups per update, while gradient caching handled oversized groups in chunks without breaking their joint LCE loss.Baseline run 4 used execution chunks of up to 16,384 tokens on H100 GPUs.Source: C Supervised Fine-Tuning Configuration · p. 41
  • Unless specified otherwise, retained parameters were fine-tuned for one epoch with AdamW, a 5 × 10^-5 learning rate, and linear-warmup then cosine decay.The recipe used 150 warmup steps, β = (0.9, 0.95), weight decay 0.01, and global gradient clipping at 12; comparisons generally were not hyperparameter-tuned.Source: C Supervised Fine-Tuning Configuration · p. 41
  • SFT comparisons used top-100 BM25 candidates and nDCG@10, with means reported separately for TREC Deep Learning 2019–2023 and seven BEIR collections.The four baseline runs used the same harness and inputs as the soup checkpoint.Source: C Supervised Fine-Tuning Configuration · p. 41
  • Baseline runs used two RTX PRO 6000 Blackwell GPUs except run 4, which used two H100 GPUs; many variants used two H200 NVL GPUs.Runs 1, 3, and 4 completed 1,931 updates, and run 2 completed 1,933; variant hardware and update counts differed by run.Source: C Supervised Fine-Tuning Configuration · pp. 41–42
Loading 2610.11922v1…