PaperlayerView exact revision on arXiv

Source-linked AI summary

Instagram photos reveal predictive markers of depression

Andrew G. Reece, Christopher M. Danforth

arXiv:1608.03282v2cs.SIphysics.soc-ph

TL;DR

The paper asks whether Instagram data can provide computational markers of depression, a question relevant where mental health services are limited. Using machine learning on Instagram features, it found that the best classifier correctly identified 70% of depressed cases and improved over general practitioners’ accuracy on most metrics.

  • Problem

    The paper examines whether Instagram data can provide computational markers of depression, relevant where mental health services are unavailable or underfunded.

  • Method

    Machine-learning models analyzed Instagram activity, reactions, faces, color properties, and filters extracted from participants’ photographs.

  • Results

    70% of depressed cases were correctly identified, and the best All-data classifier improved over general practitioners’ accuracy on most metrics.

  • Takeaways & Limitations

    Computational analysis of social-media histories may support low-cost mental-health screening and help identify individuals in need of treatment.

  • Takeaways & Limitations

    The Pre-diagnosis model had weaker predictive power because it was trained on considerably fewer data points than the All-data model.

Abstract

from arXiv · show

Using Instagram data from 166 individuals, we applied machine learning tools to successfully identify markers of depression. Statistical features were computationally extracted from 43,950 participant Instagram photos, using color analysis, metadata components, and algorithmic face detection. Resulting models outperformed general practitioners' average diagnostic success rate for depression. These results held even when the analysis was restricted to posts made before depressed individuals were first diagnosed. Photos posted by depressed individuals were more likely to be bluer, grayer, and darker. Human ratings of photo attributes (happy, sad, etc.) were weaker predictors of depression, and were uncorrelated with computationally-generated features. These findings suggest new avenues for early screening and detection of mental illness.

Method

The study extracted computational image features from Instagram photos, including HSV color properties, metadata, posting behavior, filters, and detected faces. All-data and pre-diagnosis models outperformed a null model, while privacy, diagnostic specificity, and conservative pre-diagnosis performance remained important limitations.

  • Limitations: Privacy was difficult to guarantee because Instagram usernames and photographs could contain identifiable features.The study also cautioned that depression was nonspecific, potentially comorbid, and possibly limited to clinically diagnosed individuals.
  • Feature extraction: HSV analysis quantified hue, saturation, and brightness to characterize image color and darkness.Higher hue indicated bluer images, while lower saturation and brightness indicated grayer and darker images.
  • Model evaluation: Both All-data and Pre-diagnosis models were decisively superior to a null model.Their confidence levels were largely identical, except for reduced confidence in Pre-diagnosis brightness and posting frequency.
  • Feature extraction: Models combined image color, metadata, posting behavior, filter use, and algorithmic face-detection features to predict depression.Predictive patterns included bluer, darker, and grayer photos; comment and like counts; posting frequency; filters; and faces.
  • Model evaluation: The best All-data classifier correctly identified 70% of depressed cases in a 100-observation example while producing 23 false alarms and 17 misses.It improved on general-practitioner accuracy for most metrics, with higher recall but lower specificity.
  • Feature validation: Human sadness and happiness ratings predicted depression, but ratings showed extremely low correlation with computational features and sadness ratings were unrelated to image color.This contrasts subjective photo assessments with computationally derived predictors.

Supplementary Information · I. Data collection procedures

The study used approved, consented survey procedures to collect depression-related information from depressed participants and demographic and health-history information from healthy participants. Depression surveys additionally assessed diagnosis timing, prior suspicion, and social-media use.

  • I. Data collection procedures: The study received approval from Harvard and University of Vermont institutional review boards before data collection.Approval numbers were #15­2529 and #CHRMS­16­135.
  • I. Data collection procedures: Participants acknowledged the study goals, expectations, procedures, and data-privacy procedures before data collection.Surveys were built using Qualtrics, and analyses used Python and R.
  • I. Data collection procedures: The depressed-participant survey collected age, first-diagnosis information, social-media use at diagnosis, and CES­D responses.Additional questions addressed depression diagnosis timing and social-media usage at that time.
  • I. Data collection procedures: Depressed participants reported their first depression-diagnosis date and whether they suspected depression before diagnosis.Participants who could not recall exact dates were instructed to approximate them.
  • I. Data collection procedures: Participants who suspected depression before diagnosis reported the number of days before diagnosis when that suspicion began.Exact dates could be approximated when necessary.
  • I. Data collection procedures: The healthy-participant survey collected age and gender and asked four health-history questions used as inclusion criteria.The questions concerned pregnancy, diagnosed depression, diagnosed Post­Traumatic Stress Disorder, and diagnosed cancer.
  • I. Data collection procedures: Healthy participants’ responses to the four health-history questions were not analyzed and only determined eligibility for the studies.This included the depression-related study reported here.

II. Face Detection

The study used a two-scale-factor face detection script to identify faces in Instagram photos. The algorithm slightly undercounted faces on average, while classification accuracy differed between photos with and without detected faces.

  • Face detection method: The face detector ran twice with scale factors 1.05 and 1.4 to improve detection of both small and large faces.The script used min_neighbors = 4 and min_size = (20px,20px).
  • Detection results: 77% accuracy was achieved for depressed participants when no face was detected, compared with 79% for healthy participants.These values were reported for the no-face-detected condition.
  • Detection results: 59% accuracy was achieved for depressed participants when 1+ faces were detected, compared with 61% for healthy participants.These values were reported for the 1+ faces-detected condition.
  • Detection error: The algorithm slightly undercounted the number of faces in photos for both depressed and healthy participants.In both groups, it undercounted by less than a single face on average.

III. Summary statistics

The study analyzed Instagram data from 166 users, including depressed and healthy participants, with 43,950 photographs collected between February 1 and April 6, 2016. After applying the CES-D cutoff, 71 depressed participants remained viable, alongside 95 healthy participants.

  • Overall sample: Data collection covered 166 Instagram users and 43,950 photographs between February 1, 2016 and April 6, 2016.Users posted a mean of 264.76 photographs (SD=396.06), with a median of 122.5 posts per user.
  • Depressed participants: Among depressed participants, 84 completed participation, while the CES-D cutoff reduced the viable sample to 71.Viable participants had a mean age of 28.8 years (SD=7.09), ranging from 19 to 55 years.
  • Healthy participants: Among healthy participants, 95 completed participation and provided Instagram data.Their mean age was 30.7 years, the age range was 19 to 53 years, and 65.3% were female; gender data were unavailable for the depressed sample.
  • Model datasets: The all-data model included 43,950 posts from 166 individuals, whereas the pre-diagnosis model included 32,311 posts using depressed participants’ pre-diagnosis posts.Aggregated observations totaled 24,713 for the all-data model and 18,513 for the pre-diagnosis model.

IV. Statistical framework

The statistical framework used Bayesian logistic regression with uninformative priors, MCMC estimation, posterior-probability reporting, and Bayes-factor model assessment. Random Forest parameters were tuned by stratified five-fold cross-validation, while classification evaluation accounted for class imbalance and limitations of naive accuracy.

  • Bayesian logistic regression: Bayesian logistic regression was used alongside frequentist results, with both methods largely agreeing.The Bayesian approach was chosen to avoid inferential challenges associated with p-values and confidence intervals.
  • Bayesian logistic regression: All regression parameters received uninformative priors because the parameters of interest were novel and lacked relevant prior literature or testing.The specified prior settings included b0 = 0 and B0 = 0.0001.
  • MCMC estimation: The Instagram model used two MCMC chains of 100,000 iterations, a burn-in of 10,000, and no thinning.The MCMClogit() function used a Metropolis algorithm for Markov Chain Monte Carlo simulations.
  • Posterior inference: Bayesian results used Highest Posterior Density Intervals to express the probability that coefficients were non-zero and Bayes factors to assess model fit.A reported 99% HPDI indicates a 99% probability that the coefficient is non-zero.
  • Convergence diagnostics: MCMC diagnostics indicated good convergence across all fitted models, with Gelman-Rubin point estimates of 1.0 for every parameter.Geweke diagnostics indicated post-burn-in convergence, and autocorrelation remained within acceptable levels.
  • Machine learning model evaluation: Random Forest parameters were optimized using stratified five-fold cross-validation, and naive accuracy was treated as potentially misleading under class imbalance.The framework noted that naive accuracy can conceal classifier strengths and weaknesses when one class predominates.

V. MCMC Diagnostics

MCMC diagnostics include trace, density, and autocorrelation plots for the All-data, Pre-diagnosis, and Ratings models. Autocorrelation figures display the first chain for conciseness, with the second chain described as nearly identical where specified.

  • Trace and density plots are provided for the All-data model MCMC simulations.
  • Autocorrelation plots are provided for the All-data model MCMC simulations, with only the first chain displayed because the second chain output is nearly identical.
  • Trace and density plots are provided for the Pre-diagnosis model MCMC simulations.
  • Autocorrelation plots are provided for the Pre-diagnosis model MCMC simulations, with only the first chain displayed because the second chain output is nearly identical.
  • Trace and density plots are provided for the Ratings model MCMC simulations.
  • An additional autocorrelation plot is labeled as being for the Pre-diagnosis model MCMC simulations.

VI. Frequentist Logistic Regression

Frequentist and Bayesian logistic regression outputs are largely in agreement.

  • Frequentist and Bayesian regression output are largely in agreement.

VII. Bayesian regression

Posterior predictive checks showed that the All-data and Pre-diagnosis models did not reproduce observed depression prevalence closely, whereas the Ratings model did. The corresponding posterior predictive p-values were 1.0, 0.039, and 0.516, respectively.

  • Posterior predictive checks: 53.5% depressed observations were replicated versus 30.9% originally in the All-data model, with a posterior predictive p-value of 1.0.The replicated observations consistently overestimated the proportion of depressed observations.
  • Posterior predictive checks: 30.02% depressed observations were replicated versus 37.97% originally in the Pre-diagnosis model, with a posterior predictive p-value of 0.039.The replicated observations slightly underestimated the observed proportion of depressed observations.
  • Posterior predictive checks: 44.2% depressed observations were replicated versus 43.9% originally in the Ratings model, with a posterior predictive p-value of 0.516.The replicated observations accurately represented the true proportion of depressed observations.
  • Posterior predictive checks: Posterior predictive p-values assess how often samples from the simulated posterior overpredict reference-class membership relative to its prevalence in the original data.Gelman et al. suggested that good replication accuracy corresponds to posterior predictive p-values between 0.05 and 0.95, although an extreme value does not necessarily mean the model is wrong.

VIII. Instagram filter examples

The examples contrast the Inkwell and Valencia filters and show different preferences among depressed and healthy participants.

  • VIII. Instagram filter examples: Inkwell converts color photos to black-and-white.The figure presents Inkwell as one of the example Instagram filters.
  • VIII. Instagram filter examples: Valencia lightens tint in Instagram photos.The figure presents Valencia as the second example filter.
  • VIII. Instagram filter examples: Depressed participants most favored Inkwell, whereas healthy participants most favored Valencia.These preferences are reported as comparisons between the two participant groups.

IX. Correlation tables

This section reports Pearson’s product-moment correlation tables for features from the All-data, Pre-diagnosis, and Ratings models. The Ratings table compares ratings features with ratings and computational features.

  • Table S5 reports Pearson’s product-moment correlation scores for All-data model features.
  • Table S6 reports Pearson’s product-moment correlation scores for Pre-diagnosis model features.
  • Table S7 reports Pearson’s product-moment correlation scores for Ratings model features, with ratings and computational features as rows.

X. Ratings inter­rater agreement · XI. Differences in filter use, Pre­diagnosis model

Ratings showed positive inter-rater correlations across all categories. Filter-use differences were assessed in the pre-diagnosis model by comparing observed and expected frequencies, with similar results reported for the all-data model.

  • X. Ratings inter­rater agreement: Rater agreement was estimated by randomly selecting two raters per photo and correlating their rating vectors with Pearson’s product-moment coefficient.The procedure was repeated across five folds and the coefficients were averaged.
  • X. Ratings inter­rater agreement: Five-fold iteration was used to mitigate sampling bias in the inter-rater agreement estimates.Each iteration randomly selected two raters from every photo before computing the correlation.
  • X. Ratings inter­rater agreement: Rater agreement was positive for happy ratings, with rhappy = .19.The reported correlation was significant at p < 1 −3.
  • X. Ratings inter­rater agreement: Rater agreement was positive for sad ratings, with rsad = .17.The passage reports positive correlations across all rating categories.
  • X. Ratings inter­rater agreement: Rater agreement was positive for interesting ratings, with rinteresting = .27.The passage reports positive correlations across all rating categories.
  • XI. Differences in filter use, Pre­diagnosis model: The pre-diagnosis filter analysis compared observed and expected usage frequencies using a Chi-squared analysis of independence.Blue bars represented disproportionate filter use by depressed participants, whereas orange bars represented the reverse.
  • XI. Differences in filter use, Pre­diagnosis model: Pre-diagnosis model filter-use results were similar to those from the all-data model.The all-data comparison was referenced in the main text Figure 3.
Loading 1608.03282v2…