Source-linked AI summary
Recommender System for Online Dating Service
Lukas Brozovsky, Vaclav Petricek
TL;DR
Online dating users face information overload and matchmaking lacks substantial dating-specific recommender research. The paper implements and evaluates collaborative-filtering and global recommenders on real dating data, finding that collaborative filtering outperforms global popularity and is preferred by users.
Problem
Online dating creates information overload, while matchmaking remains underrepresented in recommender-system research and dating-specific datasets.
Method
The paper benchmarks two collaborative-filtering and two global algorithms on a real online-dating dataset and evaluates recommendation quality with real users.
Results
User-User and Item-Item collaborative filtering outperform global popularity in prediction NMAE by 3.08% and 2.04%, respectively, and users prefer User-User recommendations.
Takeaways & Limitations
Collaborative-filtering recommenders can provide good recommendations for online dating and may benefit both service users and owners.
Takeaways & Limitations
The paper identifies incomplete evaluation of state-of-the-art model-based methods and unresolved interface bias affecting the truthfulness of ratings.
Abstract
from arXiv · showhide
Users of online dating sites are facing information overload that requires them to manually construct queries and browse huge amount of matching user profiles. This becomes even more problematic for multimedia profiles. Although matchmaking is frequently cited as a typical application for recommender systems, there is a surprising lack of work published in this area. In this paper we describe a recommender system we implemented and perform a quantitative comparison of two collaborative filtering (CF) and two global algorithms. Results show that collaborative filtering recommenders significantly outperform global algorithms that are currently used by dating sites. A blind experiment with real users also confirmed that users prefer CF based recommendations to global popularity recommendations. Recommender systems show a great potential for online dating where they could improve the value of the service to users and improve monetization of the service.
1 Introduction
Online dating users face information overload from questionnaire-based matching and large, often irrelevant result sets, especially for multimedia profiles. The paper proposes a recommender-system approach and evaluates its potential for improving user and business outcomes.
- Motivation: Online dating searches can return no matches for specific queries or huge numbers of profiles for general queries.Users must manually specify attributes and often assign weights to requirements, making ranking difficult.
- Motivation: Questionnaire-based matching is limited because users may express the same preference in different words.
- Background: Online dating sites commonly reproduce offline agencies’ questionnaires and matching processes.
- Research gap: Although matchmaking is a frequently cited recommender-system application, relatively little research has addressed it.The paper notes that prior collaborative-filtering datasets and published work did not specifically represent dating preferences.
- Contribution: The paper describes a dating recommender system, benchmarks algorithms on real-world data, and evaluates recommendation quality with real users.The stated potential benefits include higher user satisfaction and loyalty and improved monetization.
2 Related Work
Collaborative filtering is an established recommender-system approach, but dating remains underrepresented in the research literature and datasets. The paper positions its work as addressing this domain-specific gap.
- Collaborative filtering: Recommender systems have become popular through applications such as Amazon and Netflix, with collaborative filtering among the most widely used approaches.
- Collaborative filtering: Collaborative-filtering methods include memory-based user-user approaches and model-based methods such as matrix decomposition.Examples mentioned include user-user k-nearest neighbors, SVD, MMMF, item-item methods, and Bayesian networks.
- Research scope: Prior collaborative-filtering research addressed areas including prediction accuracy, scalability, cold start, and robustness.
- Research gap: Despite prior work, dating was not specifically addressed and existing datasets did not represent dating preferences.
- Research gap: The literature on collaborative-filtering recommender systems for dating remains surprisingly sparse despite dating being a frequently cited application.
3 Algorithms
The paper compares random and mean baselines with user-user and item-item collaborative-filtering algorithms. The collaborative methods estimate ratings from similarity relationships and use overlap and neighborhood parameters.
- Baselines: The random baseline assigns each user-profile pair a fixed uniformly distributed random rating within the rating scale.Keeping the prediction fixed for a pair makes results repeatable across runs.
- Baselines: The mean baseline predicts a profile’s average rating and ignores information from the active user.The random and mean methods serve as comparison baselines.
- User-user algorithm: User-user collaborative filtering finds users with similar rating vectors and predicts from ratings supplied by the most similar neighbors.Similarity is computed with Pearson correlation, and predictions use neighbor ratings adjusted by mean ratings and normalization.
- User-user algorithm: User-user similarity is computed only when users share enough ratings, and the algorithm limits the number of neighbors used.The implementation uses MinO and MaxN; User-User (5,50) uses five common ratings and at most 50 neighbors.
- Item-item algorithm: Item-item collaborative filtering computes similarities between profiles rather than rating-matrix rows.Its adjusted Pearson correlation subtracts each user’s mean to account for different rating scales.
- Item-item algorithm: Item-item predictions use the active user’s ratings for the most similar profiles to the target profile.The implementation likewise uses minimum overlap and a maximum neighborhood size, with Item-Item (5,50) as the named configuration.
4 Libimseti Dataset
The study uses a large, sparse rating matrix from the Libimseti online dating service, where people appear both as raters and rated profiles. Dataset figures describe rating activity, rating values, and similarity distributions under different overlap thresholds.
- Dataset: The dataset is a July 2005 snapshot of the real Libimseti online dating service’s rating matrix.Individuals are represented both as active providers of ratings and as profiles rated by others.
- Dataset: Libimseti contains 194,439 users, 11,767,448 ratings, and matrix sparsity of 0.03%.The paper compares these characteristics with the MovieLens and Jester collaborative-filtering datasets.
- Dataset overview: Table 1 summarizes dataset characteristics, defines an active user as someone who rated another profile at least once, and includes statistics normalized to a 0–1 scale.
- Dataset distributions: Figure 1 shows separate distributions for the number of ratings provided by a user and for rating values.
- Similarity distributions: Figure 2 presents user-user and item-item similarity distributions for minimum-overlap values of 5, 10, 15, 20, 25, and 30.
5 ColFi Recommender System
ColFi is a modular, domain-independent recommender system built around a stateless TCP/IP server, a data manager, and pluggable services. Its design supports parallel clients and the addition of collaborative filtering algorithms.
- System design: ColFi is a domain-independent, freely available recommender system designed to let developers focus on collaborative filtering algorithms.The system was tested on four datasets.
- System design: The system uses a stateless TCP/IP server with a global data manager, ColFi services, and a communication module implementing CCP.The data manager supplies data, while services expose APIs to clients.
- System design: CCP supports a limited set of remote-method invocations, making it faster than SOAP Web Services and less demanding to configure than Java RMI.The protocol is designed for simple client-server communication.
- System design: Data Manager is a plug-in interface whose implementation may cache data, access databases or file systems directly, or provide read-only access.Different implementations can therefore support different data-access behaviors.
- System design: ColFi services are stateless and independently exposed on TCP/IP ports, allowing atomic requests and parallel handling of multiple client connections.The service layer can also provide statistics in addition to collaborative filtering.
6 Benchmarks
The benchmarks evaluate recommender algorithms with three cross-validation protocols and NMAE prediction error. AllButOne hides one rating at a time and tests recovery of each hidden value.
- Benchmark setup: The study compares algorithms on the Libimseti dataset using training ratings for prediction and withheld ratings for testing accuracy.The evaluation uses three cross-validation scenarios.
- Benchmark setup: Three validation protocols are used: AllButOne, GivenRandomN, and production.Each protocol evaluates prediction quality using NMAE.
- Evaluation assumptions: Predictions that cannot be made because of insufficient data are excluded from NMAE, while their total number is reported separately.The authors report that excluding these cases does not affect algorithm ordering.
- AllButOne validation: AllButOne hides exactly one rating, asks the algorithm to predict it, and computes absolute prediction error for every rating in the test set.It is described as the simplest and most commonly used benchmark among collaborative filtering papers.
AllButOne Validation
The validation protocols examine performance under different amounts and sources of user data. GivenRandomN models user cold-start by progressively increasing each active user’s training set, while production simulates online rating insertions.
- GivenRandomN validation: GivenRandomN examines algorithm performance when less data is available from an active user, representing user cold-start.This setting concerns users who have not yet provided enough ratings.
- GivenRandomN validation: The protocol uses users with more than 100 ratings, retaining 99 random ratings for training and the remainder for testing.The training ratings generate nested sets containing progressively more observations.
- GivenRandomN validation: GivenRandomN produces an NMAE graph showing prediction error as a function of training-set size.The nested training sets represent ratings gradually provided by the active user.
- Production validation: Production validation splits the test data into initial and simulation sets of similar size, then inserts randomly selected simulation ratings through the ColFi server.The setup includes additions from new users and predicts before each insertion.
Production Validation
The production-oriented benchmark evaluates algorithms under concurrent server activity, while the reported results compare neighborhood choices and algorithm accuracy using NMAE. User-User performs best overall in the summarized comparison.
- Production setup: The benchmark uses K = 10,000 and Nmax = 100 concurrent clients.These are the stated variable parameters of the production protocol.
- Algorithm comparison: Larger neighborhoods improve User-User accuracy, but more than 50 neighbors do not significantly improve performance.Item-Item with neighborhood size 50 is used for the benchmarks.
- Algorithm comparison: User-User has the best overall performance, while Mean performs surprisingly well because of a strong global component in user preferences.The results are summarized in Table 2.
- AllButOne results: Table 2 reports AllButOne NMAE in the dataset’s original scale and the number of skipped predictions caused by insufficient data.The table contains both prediction error and unavailable-prediction counts.
- Production results: The production-validation results are presented as an NMAE graph, with Random algorithm results omitted for readability.NMAE is the reported prediction-quality measure.
GivenRandomX Validation Results
As users provide more ratings, algorithmic NMAE improves initially and then stabilizes after roughly 30–40 ratings. User-User generally achieves the lowest NMAE, while Random is omitted from the figures because of its high error.
- After about 30–40 ratings, algorithm accuracies stabilize because the algorithms have enough information to derive the active user type.Performance improves for the first few inserted ratings before reaching this stable region.
- User-User generally has the lowest NMAE values across the benchmark results.
- Random is omitted from the GivenRandomX graph because its NMAE remains relatively high at 38.27%.
- The production benchmark also omits Random results from the NMAE graphs, and its performance ordering corresponds to the AllButOne and GivenRandomN results.
7 User experiment
The user experiment compared Random, Mean, and User-User recommendation lists generated from participant ratings. Users perceived User-User as best overall, although Mean won a substantial share of pairwise comparisons and Random was generally least preferred.
- 111 participants rated 150 random profiles, producing two randomized lists of 10 unrated profiles for each comparison.The study collected 14,057 ratings for 1,000 profiles, with matrix sparsity of 11.39.
- The experiment compared Random, Mean, and User-User algorithms, selecting User-User (10,50) as the best-performing collaborative algorithm.
- User-User was perceived as the best algorithm overall, although the global Mean algorithm won a surprising fraction of recommendation duels.
- Random lost against both collaborative-filtering algorithms and Mean in the user-study comparisons.
- Random nevertheless won 12.5% and 16.87% of duels, which the authors associate with perceived profile-quality clustering from self-selection bias.
- Mean won 35.62% of its comparisons against collaborative filtering, suggesting a shared universal preference among users.
8 Discussion
Collaborative filtering outperformed global popularity recommendations, but the study identifies several boundaries and directions for improving dating recommenders. These include broader algorithm evaluation, interface bias, scalability, hybrid methods, and reciprocal matching.
- Findings: User-User and Item-Item collaborative filtering outperformed global popularity in prediction NMAE by 3.08% and 2.04%, respectively.The difference was also noticeable to users, who preferred User-User recommendations to global popularity recommendations.
- Future evaluation: A more complete evaluation should include state-of-the-art and model-based collaborative filtering methods in dating-service settings.The authors also identify scalability and performance as issues these methods might help address.
- Evaluation limitations: User interfaces may bias ratings when users guess global preferences instead of reporting personal preferences.The paper leaves open how to motivate sufficient truthful ratings.
- Future improvements: Hybrid algorithms combining collaborative filtering with content information may further improve recommendations.The paper also proposes reciprocal matching because one user's interest in another does not imply mutual interest.
- Future improvements: Reciprocal matching should account for whether recommended users are also interested in the recipient.This follows from the paper's observation that “A likes B” does not imply “B likes A”.