Source-linked AI summary
The Unfairness of Popularity Bias in Music Recommendation: A Reproducibility Study
Dominik Kowald, Markus Schedl, Elisabeth Lex
TL;DR
Popularity bias can underrepresent less popular items and underserve users who prefer them. The paper reproduces prior movie-domain analyses in music recommendation using Last.fm users grouped by mainstreaminess, finding popularity bias alongside worse recommendations for low-mainstream users. It also reports that GAP behaves differently in Last.fm’s much larger item space and identifies robustness and user-characteristic analysis as future work.
Problem
Recommender systems tend to underrepresent less popular items, and prior work found that users who like unpopular items can be underserved.
Method
The paper reproduces the prior study in music recommendation using Last.fm data, mainstreaminess-based user groups, and six recommendation algorithms evaluated for popularity bias and accuracy.
Results
LowMS users receive significantly worse recommendations than MedMS and HighMS users, while popularity bias favors popular artists and GAP differs from the movie-domain result.
Takeaways & Limitations
The reproduced music-domain findings indicate that users interested in unpopular artists are disadvantaged, while popularity-bias measurement may need adaptation across domains.
Takeaways & Limitations
The study’s GAP metric does not produce the same results for Last.fm as for MovieLens, possibly because Last.fm contains many more available items.
Abstract
from arXiv · showhide
Research has shown that recommender systems are typically biased towards popular items, which leads to less popular items being underrepresented in recommendations. The recent work of Abdollahpouri et al. in the context of movie recommendations has shown that this popularity bias leads to unfair treatment of both long-tail items as well as users with little interest in popular items. In this paper, we reproduce the analyses of Abdollahpouri et al. in the context of music recommendation. Specifically, we investigate three user groups from the LastFM music platform that are categorized based on how much their listening preferences deviate from the most popular music among all LastFM users in the dataset: (i) low-mainstream users, (ii) medium-mainstream users, and (iii) high-mainstream users. In line with Abdollahpouri et al., we find that state-of-the-art recommendation algorithms favor popular items also in the music domain. However, their proposed Group Average Popularity metric yields different results for LastFM than for the movie domain, presumably due to the larger number of available items (i.e., music artists) in the LastFM dataset we use. Finally, we compare the accuracy results of the recommendation algorithms for the three user groups and find that the low-mainstreaminess group significantly receives the worst recommendations.
1 Introduction
Recommender systems underrepresent less popular items, potentially underserving users who prefer them. This study reproduces prior movie-domain analyses in music recommendation using comparable research questions and Last.fm data.
- Popularity bias underrepresents less popular, long-tail items in recommendation lists and can underserve users who prefer unpopular items.
- Around 1,000 users listen to at least 20% unpopular artists, indicating substantial interest in less popular music.
- The study reproduces Abdollahpouri et al.’s analyses in music recommendation and retains their two research questions.
- Experiments use a public Last.fm dataset to analyze artist popularity in user profiles and compare six state-of-the-art algorithms’ popularity-bias propagation.
2 Popularity Bias in Music Data
The Last.fm data reveal both substantial interest in unpopular artists and relationships between profile size and artist popularity. Users with smaller profiles tend to have more popular artists, consistent with the study’s reproduced findings.
- The study selects 3,000 Last.fm users across low-, medium-, and high-mainstreaminess groups using mainstreaminess as a proxy for inclination toward popular music.
- Around one-third of users have at least 20% unpopular artists in their profiles, corresponding to the LowMS group.
- User profile size positively correlates with the number of popular artists (R = .965) but negatively correlates with average artist popularity (R = −.372).
- Users with smaller profiles tend to listen to more popular artists than users with more extensive profiles.
3 Popularity Bias in Music Recommendation
The study evaluates popularity bias in music recommendations using six algorithms, Group Average Popularity, and MAE across LowMS, MedMS, and HighMS users. Most algorithms recommend overly popular artists, while LowMS users receive significantly worse recommendations and NMF performs best among personalized algorithms.
- Except for Random and NMF, all approaches provide too-popular artist recommendations for LowMS, MedMS, and HighMS users.Figure 4 reports this pattern using the Group Average Popularity change, ∆GAP.
- Popular artists are recommended more often than unpopular artists by every algorithm except Random, with the strongest effect for MostPopular.NMF shows a weaker popularity-bias effect than the other non-random approaches.
- The ∆GAP results show no clear difference between user groups except for MostPopular, unlike the movie-domain findings.The authors attribute this difference possibly to the Last.fm dataset’s 352,805 artists versus 3,900 movies in MovieLens.
- LowMS users receive significantly worse MAE recommendations than MedMS and HighMS users for all four personalized algorithms.The difference is statistically significant according to a t-test with p < .005.
- NMF provides the best MAE results across the personalized algorithms and also produces the fairest artist-popularity results.The MAE metric is better when lower, and the best values are reported for NMF.
4 Conclusion and Future Work
The study reproduces a movie-domain analysis in music recommendation and confirms popularity bias alongside worse recommendations for users interested in unpopular artists. It also finds that GAP behaves differently on Last.fm, motivating future metric adaptation and further study of LowMS users.
- The reproduction confirms limited interest in popular items and worse recommendations for LowMS than HighMS users.
- The GAP metric does not yield the same results on Last.fm as on MovieLens, probably because Last.fm contains many more available items.
- Future work will adapt GAP for greater robustness across domains and study why LowMS users receive the worst recommendations.The authors also mention developing novel algorithms to potentially address this disparity.