PaperlayerView exact revision on arXiv

Source-linked AI summary

Exploring Author Gender in Book Rating and Recommendation

Michael D. Ekstrand, Daniel Kluver

arXiv:1808.07586v2cs.IRcs.HC

TL;DR

This paper examines how collaborative filtering recommenders interact with author-gender patterns in book-rating data, a social concern because recommender systems can propagate patterns from user behavior. Using public book ratings, the authors measure gender distributions in user profiles and recommendations, finding that algorithms differ in how they propagate those patterns and that balanced recommendation lists can be achieved with little accuracy loss.

  • Problem

    Recommender systems may propagate input-data patterns relevant to social concerns, but how their non-propagation relates to those concerns remains an open question.

  • Method

    The paper experimentally and observationally studies author-gender distributions in book data, user rating profiles, and collaborative-filtering recommendation lists, including accuracy effects of adjusting list gender composition.

  • Results

    Collaborative filtering algorithms differ in how they propagate users’ author-gender tendencies, while equal male–female recommendation lists can be achieved with little loss in recommendation accuracy.

  • Takeaways & Limitations

    The findings provide an initial basis for understanding recommender interactions with social concerns and support further fairness research using the paper’s data, methods, and code.

  • Takeaways & Limitations

    Book-rating data is extremely sparse, and the small BookCrossing dataset limits the picture of users’ reading histories and reduces some algorithms’ performance.

Abstract

from arXiv · show

Collaborative filtering algorithms find useful patterns in rating and consumption data and exploit these patterns to guide users to good items. Many of the patterns in rating datasets reflect important real-world differences between the various users and items in the data; other patterns may be irrelevant or possibly undesirable for social or ethical reasons, particularly if they reflect undesired discrimination, such as discrimination in publishing or purchasing against authors who are women or ethnic minorities. In this work, we examine the response of collaborative filtering recommender algorithms to the distribution of their input data with respect to a dimension of social concern, namely content creator gender. Using publicly-available book ratings data, we measure the distribution of the genders of the authors of books in user rating profiles and recommendation lists produced from this data. We find that common collaborative filtering algorithms differ in the gender distribution of their recommendation lists, and in the relationship of that output distribution to user profile distribution.

1 Introduction

This paper studies how collaborative filtering recommender systems interact with author-gender patterns in book data, user profiles, and recommendation lists. It measures whether these patterns propagate into recommendations and the accuracy cost of targeting list distributions.

  • The study examines author-gender representation in book catalogs, users’ reading histories, and collaborative-filtering recommendations.
  • Recommender systems may reproduce input patterns, but how those patterns propagate—and how that relates to social concerns—remains an open question.
  • Book rating data contains a higher proportion of female-authored books than the underlying Library of Congress book corpus, while GoodReads interactions are near gender parity.
  • Several collaborative filtering algorithms propagate much of users’ gender-related input tendency into their outputs.
  • Arbitrary recommendation-list distributions, including equal male–female balance, can be achieved with little loss in recommendation accuracy.
  • The paper describes current data and algorithm behavior without making normative claims about whether observed distributions represent under- or over-representation.

2 Background and Related Work

Related work places this study within research on recommender-system social impacts, machine-learning fairness, and provider-focused fairness. The paper specifically analyzes calibrated provider fairness through the gender composition of recommendation lists.

  • Recommender-system research has examined social effects such as filter bubbles, feedback loops, and socially desirable outcomes alongside traditional accuracy and diversity objectives.
  • Fairness research distinguishes consumer fairness from provider fairness, which concerns whether producers of recommended content receive fair treatment.
  • This study focuses on provider fairness and relates it to long-tail recommendation, item diversity, and calibration.
  • Its list-composition perspective examines gender representation in recommendation rankings rather than only provider exposure counts.
  • The paper analyzes calibrated provider fairness in classical collaborative filtering algorithms and their underlying training data using offline empirical methods.

3 Data Sources and Integration

The study integrates public interaction, book, and author data to analyze gender representation in reading histories and recommendations. The resulting composite dataset supports explicit- and implicit-feedback experiments across multiple book-rating sources.

  • Studying content-creator demographics requires consumption data, book records, and author data containing demographic characteristics.
  • The authors integrate six public datasets, linking heterogeneous records despite malformed identifiers and missing linking keys.
  • Three user-book interaction sources—BookCrossing, Amazon Books, and GoodReads—provide historical profiles and collaborative-filtering training data.
  • The datasets support explicit-feedback analyses using ratings and implicit-feedback analyses treating user-item interactions as positive signals.
  • BookCrossing contains 1.1M interactions, Amazon Books contains 22.5M reviews and ratings, and GoodReads contains 189M interactions.

3.3 Book Bibliographic Records

Book bibliographic records are pooled from Open Library and Library of Congress sources and linked to rating data through ISBNs. Related editions are grouped into creative-work items to improve coverage and reduce sparsity.

  • Book records, especially author lists, are pooled from Open Library and Library of Congress MARC records.
  • The authors link bibliographic records to rating data by ISBN, although Library of Congress ISBN fields require format handling.
  • Related ISBNs are grouped into single items representing creative works rather than individual editions.
  • The grouping process reduces interaction sparsity and increases data-linking coverage.
  • The reproducible linking process causes multiple ratings for a book in less than 1% of ratings; explicit-feedback duplicates are resolved with the median rating.

3.5 Author Gender Data

The paper links book and author records to resolve author gender, but the available VIAF data reduce gender identities to male, female, or unknown. The resulting workflow reports coverage and groups unresolved cases as unlinked.

  • Author information comes from VIAF, whose MARC Authority Field 375 provides gender identity attributes for many records.
  • VIAF records gender only as “male”, “female”, or “unknown”, so gender minorities are not represented or may be misgendered.
  • Because source systems lack shared linking identifiers, books are matched to VIAF authority records by normalized author names.The process matches listed name forms for the first author within each ISBN group.
  • Books are classified as ambiguous, male, or female when gender assertions agree, while no-data conditions are grouped as unlinked.The reported resolution categories also include missing books, authors, VIAF matches, or gender assertions.
  • Less than 2.5% of rated books have ambiguous author genders under the name-linking strategy.The strategy was selected to balance coverage with classification confidence.
  • The paper rejects demographic inference from names or faces because such approaches can be reductionistic, stereotype-reinforcing, or racially biased.It follows cataloging guidance that gender identity should not be assumed from pictures or names.

3.6 Data Set Statistics

The integrated data are sparse but sufficiently covered for analysis, and women are more represented in rating data than in the Library of Congress book corpus. GoodReads interactions approach gender parity.

  • The integrated data have sufficient coverage for meaningful analysis despite being sparse.
  • 22.7% of Library of Congress books with known author genders are written by women.
  • 30.6% of Amazon-rated books and 40.7% of BookCrossing books with known genders are written by women.
  • 37.7% of known-gender books rated on GoodReads are written by women, rising to 48.2% among shelf adds.
  • GoodReads is close to gender parity in books rated or added to shelves, while the paper cannot determine whether coverage biases affect unlinked or unknown books.

4 Experiment and Analysis Methods

The experiment combines integrated book and rating data, samples users and recommendations, evaluates collaborative-filtering accuracy, and uses hierarchical Bayesian regression to estimate gender distributions and their propagation.

  • Experimental design: 1000 users with at least 5 books of known author gender were sampled to estimate profile distributions and generate recommendations.Each sampled user received 50 recommendations from models trained on the full data set.
  • Experimental design: The study compared user-user, item-item, ALS, and BPR collaborative-filtering algorithms across implicit and explicit feedback settings.ALS was used in both feedback modes, while user-user and item-item methods used feedback-specific variants.
  • Accuracy evaluation: Accuracy was evaluated with Mean Reciprocal Rank using one held-out rating and 100-item recommendation lists for 5000 users.Hyperparameters were tuned on a separate identically constructed tuning set using Gaussian-process optimization.
  • Statistical model: A hierarchical Bayesian model estimated user author-gender tendencies while accounting for varying profile sizes, multiple comparisons, and distributional uncertainty.The model uses a logit-normal tendency distribution and negative-binomial profile-size model, with diffuse priors.
  • Statistical model: Recommendation-list gender proportions were modeled as an algorithm-specific linear function of smoothed user-profile proportions in log-odds space.Algorithm-specific slopes capture responsiveness to input profiles, intercepts capture balance at an evenly split profile, and residual variance captures independent variation.

5 Profile and Propagation Results

User profiles showed diverse author-gender tendencies, while recommendation lists were more consistent. Implicit-feedback algorithms were more responsive to profile balance, whereas explicit-feedback algorithms showed weaker responsiveness and stronger male-author skew.

  • User profiles: Estimated user tendencies were smoothed because short profiles produce coarse observed proportions and can otherwise create extreme apparent biases.Five-book profiles permit only six observed proportions, while the Bayesian tendency estimates provide a continuous distribution.
  • User profiles: Users tended to rate male authors more frequently than female authors, although female-authored books were overrepresented relative to uniform sampling from the available book set.The estimated tendencies were highly diverse, with some users strongly favoring female-authored books.
  • Recommendation lists: All recommenders produced more consistent author-gender tendencies than the underlying user profiles.Recommendation-list distributions were examined with coverage statistics, density plots, and model-based summaries.
  • Propagation: Implicit-feedback algorithms generally responded strongly to user profile balance, with slopes greater than 0.5.The regression analysis compared observed recommendation proportions with user-profile proportions and estimated algorithm-specific slopes and intercepts.
  • Propagation: Explicit-feedback algorithms were less responsive and more strongly skewed toward male-authored books, with relatively small slopes and negative intercepts.A negative intercept means an evenly balanced input profile yields recommendations containing more men than women.

6 Forced-Balance Recommendation

The study tests simple post-processing methods that constrain recommendation-list gender balance or match users’ profile balance. These adjustments generally incur little accuracy loss, with profile-matching calibration usually performing best.

  • Rationale: The forced-balance study evaluates simple rerankers because no consensus exists about the appropriate target distribution or quality-balance tradeoff.The analysis is intended to establish lower limits for distributional adjustment rather than determine a universal best tradeoff.
  • Methods: SingleEQ, GreedyEQ, and GreedyReflect are post-processors that rerank another recommender’s output.SingleEQ and GreedyEQ target approximately equal male- and female-authored representation, while GreedyReflect matches the user’s profile balance.
  • Results: Most balancing penalties are only a few percent, and equalization sometimes improves recommendation accuracy.The largest penalties occurred for algorithms that already performed poorly.
  • Results: GreedyEQ generally outperforms SingleEQ, while GreedyReflect usually performs best by matching user profiles instead of imposing a fixed 0.5 target.GreedyEQ uses multiple passes, whereas GreedyReflect applies the same multi-pass structure with profile-based calibration.
  • Conclusion: Simple approaches can adjust recommendation-output balance without substantially reducing accuracy.More sophisticated calibration and independence approaches were left for future work.

7 Summary of Findings

The study finds that rated-book gender distributions are more balanced than the known-book baseline, while recommender algorithms vary substantially in how they propagate or alter those distributions. Recommendation balance can be adjusted with minimal accuracy loss.

  • Known books are more likely to be written by men, while rated books show a more balanced gender distribution.
  • The study’s reranking results used an older book-gender file with about 2% lower coverage, which the authors do not expect to significantly affect accuracy results.
  • Users’ rating tendencies are diffuse and favor male authors overall, but less strongly than the baseline distribution.
  • Different collaborative-filtering techniques produce substantially different recommendation gender distributions.
  • Implicit-feedback algorithms reliably propagate users’ input gender balances into recommendation outputs, whereas explicit-feedback algorithms propagate them less.
  • Recommendation balance can be adjusted with minimal accuracy loss.

8 Discussion

Users’ gender tendencies differ across datasets, and collaborative-filtering algorithms respond in markedly different ways: some amplify male bias, while others reflect individual profiles. These findings are constrained by sparse data, binary gender measurement, and a small algorithm sample.

  • BookCrossing users show mild, diffuse male-author tendencies, while Amazon users show somewhat stronger but still highly diffuse tendencies.
  • Nearest-neighbor recommenders in implicit-feedback mode propagate much of each user’s profile tendency into recommendations.
  • Matrix factorization consistently produces a stronger male-author bias than its input data and is largely insensitive to individual user tendencies.
  • The relationship between recommender gender distributions is not simple and depends on the algorithm, application, and other variables.
  • The rating data is extremely sparse, and the small BookCrossing dataset limits the picture of users’ reading histories and reduces some algorithm performance.
  • High data sparsity caused matrix factorization to perform particularly poorly on offline accuracy metrics, so its findings may not represent behavior in the wild.
  • The study accounts only for binary gender identities and evaluates five algorithms, limiting claims about broader algorithmic behavior.

9 Conclusion and The Road Ahead

The paper presents an initial inquiry into how collaborative-filtering book recommenders respond to gender distributions in user preference data. It frames this work as a first step toward broader study of recommender interactions with social concerns.

  • The study finds that collaborative-filtering algorithms differ in their response to gender distributions in user preference data.
  • The paper positions itself as a first step toward understanding recommender interactions with potentially discriminatory biases and broader social issues.
  • Future work should obtain higher-quality data, including data on non-binary gender identities, and extend statistical methods accordingly.
  • Future studies should examine other content-creator features, domains, and applications, including ethnicity, movies, research literature, and social media.
  • Future research could develop algorithms that incorporate socially relevant user or item characteristics to reflect organizational, societal, or individual goals.
  • Further work should study how diversification and other recommendation refinements affect recommendation distributions.
  • The authors call for more research on how recommender systems respond to and influence their sociotechnical contexts.
Loading 1808.07586v2…