Technical Paper Recommendation: A Study in Combining Multiple Information Sources

WHIRL

• WHIRL extends a conventional DBMS with information-retrieval similarity metrics, ranking tuples by cosine similarity between TFIDF-weighted text vectors rather than requiring exact field equality.This enables joins between reviewer descriptors and paper text with different terminology.Source: .
  .
  WHIRL · p. 8

Combining Information Sources by Query Expansion

• Multiple-source recommendations can combine similarity scores across sources or first combine source terms into one representation, with query expansion providing an analogy for the latter approach.The study compares expansion methods on the same inputs and estimates a source’s merit by comparing algorithms that include it with those that exclude it.Source: .
  .
  WHIRL · p. 9

The Concatenation Method

• The queryConcat method adds information from a new data source by appending terms; one descriptor uses the union of words appearing in paper abstract and keyword fields.The descriptor can represent different combinations of abstract, keyword, and title fields using their union.Source: .
  .
  WHIRL · pp. 9–10

The Conjunction Method

• The conjunction method adds separate source-comparison conditions to the WHERE clause, testing abstract, keyword, and title matches in different combinations.Reviewer data can come from a homepage or the first 100 words extracted from each PostScript file; another tested query adds a homepage-keyword conjunct alongside reviewer paper text.Source: .
  .
  WHIRL · pp. 10–11

Recommending with Reviewer Preferences

• The study simulates collaborative filtering from reviewer ratings using kNN and Extended Direct Bayes, although such preference information may be unavailable during real paper selection.After a negative rating, the simulated process supplies one relevant paper as a positive example for conditioning future recommendations.Source: .
  .
  WHIRL · p. 11

.
.
WHIRL

• Extended Direct Bayes ranks candidate papers using their estimated relatedness to papers a reviewer previously rated positively, assuming those prior papers are independent.Its score represents the probability that a candidate is related to at least one positively rated paper.Source: .
  .
  WHIRL · p. 12
• The evaluation measures precision among the Top 10 and Top 30 recommendations, reporting averages across reviewers; the table caption identifies those two cutoffs.The methods are evaluated on recommendation lists intended to reflect reviewer workloads.Source: .
  .
  WHIRL · pp. 12–13
• The evaluation crossed query-expansion methods with paper-abstract, home-page, or combined sources, measured precision at Top 10 and Top 30, and averaged results over 50 randomly chosen reviewers with both sources available online.Source: .
  .
  WHIRL · pp. 12–13
• The authors note that perfect precision is unlikely because reviewer choices reflect hard-to-model factors and human assignments may not reflect content relevance.Without an assessment of human judges performing the same task, the evaluation compares relative performance against reasonable baselines.Source: .
  .
  WHIRL · p. 13

Results

• Across queries using multiple sources, both query methods exceed random selection by factors of 2 to 5, and queryConjunct has higher precision in almost all cases.The authors identify queryConjunct as the dominant and preferred query-expansion method for this task.Source: . Results · p. 14
• For queryConjunct, queries using the most paper information usually perform significantly better than those using less, and adding reviewer information also often improves performance.The gains are not always monotonic; the authors suggest that added information such as fixed-list keywords may introduce noise and degrade retrieval.Source: . Results · pp. 14–15
• Among the tested sources, paper abstracts and reviewer homepages have the highest success rates, while the impact of a paper source depends on the reviewer representation.The observed trends for queryConjunct do not hold for queryConcat, so whether more information helps depends on the expansion method as well as the sources.Source: . Results · p. 16
• With all information sources, queryConjunct returns more papers of interest than random selection, collaborative filtering, and queryConcat, with almost four preferred papers among the Top 10.The collaborative methods are competitive with the best queryConcat performance for Top 10 papers.Source: . Results · pp. 16, 19

• Precomputed collection information about papers’ abstracts, keywords, and titles can support runtime lookup to identify promising query formulations.The passage asks whether varying the amount of this information in queries makes sense.Source: . Results · p. 20
• queryConjunct combines Boolean and vector-space query formulation by replacing terms in Boolean feedback expressions with vectors.Boolean feedback can produce expressions consisting of term conjuncts.Source: . Results · p. 20

Conclusions

• Conjunctive-query recommendations outperformed the other tested approaches, and using more information generally improved the optimal algorithm.The comparison included two content-based information-retrieval query formulations and one collaborative approach.Source: . Conclusions · p. 21
• The framework automatically gathers reviewer-interest information from the Web and combines reviewer interests and paper contents as information sources to route papers.Source: . Conclusions · p. 21
• The framework is more flexible than keyword search and less intrusive than collaborative methods, but assumes reliable, accurate, timely data; the authors are optimistic about credible Web sources.Source: . Conclusions · p. 21