PaperlayerView exact revision on arXiv

Source-linked AI summary

How to Ask for Technical Help? Evidence-based Guidelines for Writing Questions on Stack Overflow

Fabio Calefato, Filippo Lanubile, Nicole Novielli

arXiv:1710.04692v3cs.CY

TL;DR

Because Q&A sites depend on members’ willingness to help, the paper examines how question-writing relates to successful answers on Stack Overflow. It analyzes questions and user perceptions, finding that successful questions are concise, include code, avoid excessive uppercase, and use a neutral style.

  • Problem

    Because Q&A sites depend on members’ willingness to answer, the paper examines how affect, presentation quality, and timing relate to successful Stack Overflow questions.

    Source: §1 · p. 1, §2.1 · p. 2

  • Method

    The study analyzes Stack Overflow questions using regression and qualitative analyses of sentiment-bearing questions and user survey responses.

    Source: §3 · p. 5, §4.3 · p. 10, §5.2 · p. 14

  • Results

    Successful questions are concise, include code snippets, avoid excessive uppercase, and use a neutral writing style; the study also confirms guidance to avoid rudeness.

    Source: §7 · p. 15

  • Takeaways & Limitations

    The authors offer evidence-based guidelines that information seekers can follow to increase their chance of getting technical help on Stack Overflow.

    Source: §7 · p. 15

    Removing edited questions may have filtered out some questions relevant to the study, and higher-reputation members’ questions were less likely to be edited.

    Source: §6 · p. 14

Abstract

from arXiv · show

Context: The success of Stack Overflow and other community-based question-and-answer (Q&A) sites depends mainly on the will of their members to answer others' questions. In fact, when formulating requests on Q&A sites, we are not simply seeking for information. Instead, we are also asking for other people's help and feedback. Understanding the dynamics of the participation in Q&A communities is essential to improve the value of crowdsourced knowledge. Objective: In this paper, we investigate how information seekers can increase the chance of eliciting a successful answer to their questions on Stack Overflow by focusing on the following actionable factors: affect, presentation quality, and time. Method: We develop a conceptual framework of factors potentially influencing the success of questions in Stack Overflow. We quantitatively analyze a set of over 87K questions from the official Stack Overflow dump to assess the impact of actionable factors on the success of technical requests. The information seeker reputation is included as a control factor. Furthermore, to understand the role played by affective states in the success of questions, we qualitatively analyze questions containing positive and negative emotions. Finally, a survey is conducted to understand how Stack Overflow users perceive the guideline suggestions for writing questions. Results: We found that regardless of user reputation, successful questions are short, contain code snippets, and do not abuse with uppercase characters. As regards affect, successful questions adopt a neutral emotional style. Conclusion: We provide evidence-based guidelines for writing effective questions on Stack Overflow that software engineers can follow to increase the chance of getting technical help. As for the role of affect, we empirically confirmed community guidelines that suggest avoiding rudeness in question writing.

1. Introduction

Stack Overflow depends on members’ willingness to answer, making question-writing relevant both to seekers’ chance of receiving help and to community knowledge-sharing. This paper examines actionable question-writing factors and develops evidence-based guidelines.

1. Introduction

  • Q&A questions seek not only information but also other people’s help and feedback, connecting question-writing to both individual support and community knowledge-sharing.The paper notes that proper question formulation can increase support for seekers and enhance knowledge-sharing across the community.Source: §1 · p. 1
  • The study aims to validate community and research-based question-writing guidelines against evidence on questions receiving an answer accepted by the asker.The paper defines a successful answer as the answer the asker marks as the best among those received.Source: §1 · p. 2
  • The framework focuses on affect, presentation quality, and time as actionable factors, while treating asker reputation as a control factor.The paper defines success as receiving an answer accepted by the asker.Source: §1 · p. 2
  • A mixed-methods study combines analysis of 87K questions, logistic regression, qualitative analysis of positive- and negative-sentiment questions, and a user survey.The contributions include a predictive framework, an assessment of sentiment polarity’s correlation with success, and empirically validated writing guidelines.Source: §1 · p. 2

2. Conceptual framework

The paper frames question success as receiving an answer accepted by the asker and builds on prior work and Stack Overflow guidance. It organizes actionable factors and reputation control into a framework, then operationalizes them with question-level measures.

2. Conceptual framework

  • Question success is defined as receiving an answer accepted by the asker, and the framework draws on prior evidence and Stack Overflow recommendations.The paper treats “successful answer” and “best answer” interchangeably for the answer accepted as the solution.Source: §2 · p. 2

2.1. Background and related work

  • Prior Q&A research links question clarity, example code, linguistic features, expertise, reputation, and personality to success, while evidence on affect varies across communities and platforms.The paper identifies the effect of expressing emotions when seeking technical help on Stack Overflow as an open question.Source: §2.1 · p. 2

2.2. Critical factors of successful questions

  • The framework treats affect, presentation quality, and time as actionable factors and asker reputation as a control; it expects to test whether sentiment influences success.Prior evidence and community guidance motivate examining neutral style, readable presentation, and timing, alongside reputation’s reported association with success.Source: §2.2 · pp. 2–4
  • Presentation quality includes length, URLs, tags, and uppercase use, while prior work motivates testing whether readable, concise presentation predicts success.The paper also examines posting time and day of week, motivated by prior findings about successful time windows and weekend questions.Source: §2.2 · p. 3

2.3. Measuring the factors

  • Affect is measured from question-body sentiment polarity with SentiStrength, which assigns positive and negative scores before the study codes positive, negative, and neutral sentiment.Positive and negative affective loads are coded from scores in [+2, +5] and [−2, −5], respectively; neutral means +1 and −1.Source: §2.3 · p. 4
  • Presentation measures include title and body length, URL presence, single versus multiple tags, uppercase-character ratio, and whether a code snippet is present.Length categories are derived using k-means; uppercase ratio is divided into low and high categories.Source: §2.3 · p. 4
  • Time is coded by posting period and weekend versus weekday, while asker reputation is categorized as New, Low Reputation, Established, or Trusted.Reputation is assessed at the time the question was posted.Source: §2.3 · p. 4

3. Dataset

The dataset comprises questions from Stack Overflow’s July 2008–September 2014 dump, narrowed to a final sample of 87,373 questions. The authors describe the sample’s success rate, preprocessing choices, and a collinearity check on model predictors.

3. Dataset

  • The final dataset contains 87,373 questions posted from 14 August to 14 September 2014, after filtering for the study’s analysis requirements.The authors excluded self-answered, removed, closed, edited, and very recent questions; the three-day cutoff avoided questions without time to receive answers.Source: §3 · p. 5
  • 30,797 questions (35%) in the dataset had an accepted answer, and Table 2 breaks down the sample by framework factors and variables.The source dump covers Stack Overflow content from July 2008 through September 2014.Source: §3 · p. 5, Table 2 · p. 5
  • VIF values for all predictors were close to 1, and the authors conclude that predictors’ estimated variances were not affected by the values of the other explanatory variables.The authors identify VIF values in [5, 10] as denoting high collinearity in Table 3.Source: §3 · p. 5, Table 3 · p. 5

4. Results

The results combine logistic regression, qualitative sentiment analysis, and a user survey. Presentation quality stood out in the regression, while survey respondents generally favored neutral, clear questions.

4.1. Which factors are predictive of success?

  • AUC = 0.65; the logistic model differed significantly from the intercept-only null model (p < .05).Source: §4.1 · p. 6
  • 38% of sampled unsuccessful questions from new users had no accepted answer despite a solving solution being provided.Source: §4.1 · p. 7
  • OR = 2.04 for Presence of Code Snippet; OR = 0.65 for bodies over 199 words, OR = 1.16 for GMT evening, and sentiment ORs were 0.94 positive and 0.81 negative.Source: §4.1 · p. 7
  • Presentation quality was the most relevant success factor; code snippets were positive across reputation groups, while uppercase ratios were negative except for Trusted users.Negative Sentiment was negatively associated with success, particularly for Trusted users (OR = 0.55).Source: §4.1 · p. 7, Table 7 · p. 8

4.2. Qualitative analysis of emotions

  • Misclassification rose with reputation for both sentiment polarities; negative sentiment targeted self or objects, not community members, while positive sentiment toward others declined with reputation.Source: §4.2 · p. 10

4.3. User survey

  • Respondents had no consensus on posting time; 61% rejected prioritizing high-reputation askers, though comments described reputation as self-reinforcing.More than half agreed that novice users sometimes forget to mark an accepted solution.Source: §4.3 · pp. 10–12
  • 95% of respondents said code snippets improve success; respondents favored neutral style and clear, technically meaningful questions.Most also favored multiple tags (62%) and URLs (49%); 54% said abundant uppercase reduces success, while views on concise questions were unclear.Source: §4.3 · pp. 10–11
  • Survey comments described rudeness and novice-directed hostility, with some respondents avoiding accounts or treating Stack Overflow as a last resort.Source: §4.3 · pp. 12–13

5. Discussion

The discussion translates findings into question-writing advice on affect, presentation, and timing, then considers implications for modeling affect and asker reputation.

5.1. Practical implications: how to write a good question

  • Negative and positive sentiment each correlate with lower success, supporting a neutral style; negative wording usually reflects frustration or confusion, not hostility toward helpers.The discussion suggests comment moderation because prior work places hostile behavior mainly in comments.Source: §5.1 · p. 13, §5.2 · p. 14
  • Code snippets, concise questions, and appropriate capitalization align with higher success; for Trusted users, Long questions have OR = 0.31.Ninety-five percent of survey respondents strongly agreed that code snippets improve the chance of successful answers.Source: §5.1 · pp. 13–14
  • Evidence for short descriptive titles and contextual tags or URLs varies by asker reputation, despite users' favorable perceptions of tags and URLs.The findings do not confirm the related guidelines.Source: §5.1 · p. 14
  • Evening and night in GMT, corresponding to American working time, are more successful posting periods; weekend effects are negligible (OR = 1.10).Most surveyed users were unaware of these time slices, and their impact may change as community demographics shift.Source: §5.1 · p. 14

5.2. Theoretical implications

  • The success-factor model differs significantly from a null model but has low AUC = 0.65, possibly because it omits factors such as expert availability and topic popularity.Technology maturity may also affect success probability.Source: §5.2 · p. 14
  • In a qualitative sample of 240 questions, negative polarity included self-directed frustration and sadness but no hostility toward other people, supporting finer-grained affect measures.Polarity alone cannot reveal the direction of the emotion.Source: §5.2 · p. 14
  • The question-focused analysis does not capture follow-up discussions, where prior studies report negative attitudes; future replication could examine this question-nurturing phase.The authors note that comments lack the reputation mechanisms applied to questions and answers.Source: §5.2 · p. 14
  • Asker reputation was the main predictor of question success, but the study found no evidence that higher-reputation users receive more help.The authors suggest controlled experiments to investigate reputation's impact.Source: §5.2 · p. 14

6. Limitations

The study’s findings are bounded by its Stack Overflow setting, operational choices, and sentiment-analysis constraints. The authors also note that the dataset excludes edited questions and non-actionable predictors.

6. Limitations

  • Results are specific to Stack Overflow; applying the methodology to other technical Q&A sites could produce different findings.The authors identify this as an external-validity limitation for generalizing beyond the study setting.Source: §6 · p. 14
  • Success is measured by an accepted answer, although helpful answers may go unmarked when askers are unaware of or indifferent to acceptance rules.Thus, an unaccepted question does not necessarily indicate that the asker was dissatisfied.Source: §6 · p. 14
  • The dataset excludes edited questions because the dump lacks edit histories, and comparable reputation-category distributions mitigate—but do not remove—concerns about that filtering.Without edit histories, the authors could not recover what a question looked like when its successful answer was posted.Source: §6 · pp. 14–15
  • Ordinal-variable results may depend on category thresholds, although the study used k-means clustering to avoid choosing ranges arbitrarily.Asker-reputation categories instead followed Stack Overflow’s own thresholds.Source: §6 · p. 15
  • Positive/negative sentiment polarity does not capture the range of affective states expressed in technical discussions.The authors call for further research on emotion detection in software engineering.Source: §6 · p. 15
  • Sentiment tools can misclassify technical jargon and error messages as negative; software-engineering-specific resources were unavailable for this analysis.The authors note that domain dependence and false positives for negative sentiment may affect interpretation.Source: §6 · p. 15
  • The model omits non-actionable predictors such as expert availability, topic, and technology maturity, limiting its scope as a general model of Q&A success.This exclusion was consistent with the study’s goal of producing practical writing guidelines.Source: §6 · p. 15

7. Conclusions

The conclusions recommend concise, well-presented questions and a neutral emotional style, while noting that sentiment analysis in software engineering remains challenging. The authors also identify possible uses for detecting emotion direction in technical discussions.

7. Conclusions

  • Question writers are advised to include code snippets, be concise, avoid unnecessary uppercase, and account for low- and high-efficiency hours.These recommendations address presentation quality and timing; the study also advises against rudeness in question writing.Source: §7 · p. 15
  • A neutral writing style is associated with a higher probability of success, and the authors confirm community guidance to avoid rudeness.They report that expression of sentiment regardless of polarity might be detrimental to success.Source: §7 · p. 15
  • Emotion direction could help distinguish negative attitudes toward interlocutors from negative attitudes toward oneself or problem sources in help requests.The authors suggest these distinctions could inform moderation or prioritization and help recommend controversial threads to experts and project leaders.Source: §7 · p. 15
  • The authors call for sentiment tools tuned to software engineering to distinguish neutral sentences from emotionally loaded ones in technical discussions.They identify domain-dependent lexicon use as a limitation of current polarity-classification resources.Source: §7 · p. 15

Appendix B. – Breakdown of survey respondents

The appendix lists survey respondents by demographic, occupational, Stack Overflow experience, reputation, and typical site use.

Appendix B. – Breakdown of survey respondents

  • Respondent entries include age, gender, country, and occupation.The appendix identifies respondents individually as P1 through P43.Source: Appendix B · pp. 20–21
  • The breakdown also records respondents’ years of Stack Overflow experience and reputation where listed.Source: Appendix B · pp. 20–21
  • Typical site use is categorized as mostly asking, mostly answering, or doing both.Source: Appendix B · pp. 20–21
Loading 1710.04692v3…