PaperlayerView exact revision on arXiv

Source-linked AI summary

One-sample aggregate data meta-analysis of medians

Sean McGrath, XiaoFei Zhao, Zhi Zhen Qin, Russell Steele, Andrea Benedetti

arXiv:1709.03016v2stat.ME

TL;DR

Meta-analysis methods often transform reported medians into estimated means and standard errors, although the appropriate way to pool medians is uncertain. This paper proposes direct median-based approaches and compares them with transformation-based methods through simulations and an application to tuberculosis diagnosis delay. The median-based approaches perform better especially for skewed data or high inter-study variance, and can outperform or match the transformation approach’s best-case scenario.

  • Problem

    The appropriate method for pooling study-reported medians is unknown, while existing approaches transform medians and spread into estimated means and standard errors.

  • Method

    The paper proposes two direct median-based approaches, compares them with transformation-based methods in simulations, and applies them to patient delay in tuberculosis diagnosis.

  • Results

    Median-based approaches performed better than transformation-based approaches in nearly all investigated scenarios, especially with skewed data or high inter-study variance.

  • Takeaways & Limitations

    For median data, median-based approaches performed better than or comparably to using each study’s actual sample mean and standard error, the transformation methods’ best-case scenario.

  • Takeaways & Limitations

    Considering only the log-normal distribution limits generalizability because performance may change for data generated from other distributions.

Abstract

from arXiv · show

An aggregate data meta-analysis is a statistical method that pools the summary statistics of several selected studies to estimate the outcome of interest. When considering a continuous outcome, typically each study must report the same measure of the outcome variable and its spread (e.g., the sample mean and its standard error). However, some studies may instead report the median along with various measures of spread. Recently, the task of incorporating medians in meta-analysis has been achieved by estimating the sample mean and its standard error from each study that reports a median in order to meta-analyze the means. In this paper, we propose two alternative approaches to meta-analyze data that instead rely on medians. We systematically compare these approaches via simulation study to each other and to methods that transform the study-specific medians and spread into sample means and their standard errors. We demonstrate that the proposed median-based approaches perform better than the transformation-based approaches, especially when applied to skewed data and data with high inter-study variance. In addition, when meta-analyzing data that consists of medians, we show that the median-based approaches perform considerably better than or comparably to the best-case scenario for a transformation approach: conducting a meta-analysis using the actual sample mean and standard error of the mean of each study. Finally, we illustrate these approaches in a meta-analysis of patient delay in tuberculosis diagnosis.

1. Introduction

Meta-analysis commonly pools study-level means and standard errors, but some studies report medians with different spread measures, especially for time-based outcomes. The best way to meta-analyze medians remains unresolved, motivating approaches that work directly with medians.

  • Median reporting is common for time-based outcomes such as patient delay, which may reflect non-normal underlying data.
  • Continuous-outcome meta-analyses typically require a common measure of center and spread, often the sample mean and its standard error.
  • Existing methods estimate each median study’s sample mean and standard error before pooling estimated means.
  • This work proposes and compares direct median-based approaches with transformation methods, including settings mixing means and medians, and applies them to tuberculosis diagnosis delay.

2. Approaches to a One-Sample Meta-Analysis of Medians

The section reviews inverse-variance pooling and explains why medians are difficult to meta-analyze directly. It then contrasts transformation-based mean estimation with two proposed median-based alternatives.

  • Inverse-variance framework: Inverse-variance meta-analysis pools study-specific effects using weights determined by their precision.Fixed-effect weights use sampling variance, whereas random-effects weights also incorporate estimated between-study heterogeneity.
  • Challenge of pooling medians: The sampling variance of a reported median is typically unavailable and difficult to estimate from sample quantiles because it depends on the outcome distribution.
  • Transformation-based approaches: Existing approaches estimate each study’s mean and standard error from reported medians and spread before applying inverse-variance pooling.The considered summaries include medians with quartiles or extrema, and means with standard errors.
  • Median-based approaches: The proposed alternatives pool the actual study medians using methods analogous to fixed-effect and random-effects analyses.They do not require a reported measure of spread, are non-parametric, and make no distributional assumption about underlying distributions.
  • Transformation-based approaches: Transformation-based methods estimate means and standard errors using reported quartiles or minimum and maximum values, with normality assumptions for the quartile-based method.The resulting estimates are used to construct fixed-effect and random-effects pooled means and confidence intervals.

Median of Medians (MM)

The median-of-medians approach uses the median of study-specific medians as the pooled estimate and constructs an approximate 95% confidence interval around it.

  • Point estimate: The pooled median estimate is the median of the study-specific medians.
  • Confidence interval: The method constructs an approximate 95% confidence interval using quantiles of the study-specific medians.
  • Interpretation: Equal study weighting makes this method akin to a random-effects analysis with high estimated heterogeneity.

Weighted Median of Medians (WM)

The weighted-median approach targets a fixed-effect analysis by pooling study-specific medians with sample-size-proportional weights and weighted quantiles.

  • Point estimate: The pooled median estimate is the weighted median of the study-specific medians.
  • Confidence interval: The approximate 95% confidence interval uses weighted quantiles of the study-specific medians as its lower and upper limits.
  • Weighting: Study weights are proportional to sample size and normalized to sum to 1 as an estimate of precision.
  • Weighted quantiles: A weighted sample quantile is obtained from the ordered observations at the largest cumulative weight not exceeding the target quantile, with interpolation between consecutive values.

3. Simulation Study

The simulation study evaluated median-based and transformation-based approaches across median-only, mean-only, and mixed-reporting scenarios, varying skewness and inter-study variance.

  • 3.1. Data generation: The study simulated one-sample aggregate-data meta-analyses with 15 or 50 studies and study sizes centered at 50 or 100 subjects.Outcomes followed log-normal distributions with varying skewness and random study effects controlling inter-study variance.
  • 3.1. Data generation: The simulation generated study-level means, standard errors, medians, quartiles, minima, and maxima across 104 parameter combinations.Each combination was replicated with 1000 generated data sets.
  • 3.2. Study-reporting scenarios: Primary scenarios included studies reporting medians with quartiles, medians with minima and maxima, or sample means with standard errors.Sensitivity analyses mixed means and medians according to Shapiro-Wilk normality classifications.
  • 3.3. Meta-analysis approaches: Median-based methods estimated pooled medians, whereas transformation-based methods estimated study means and standard errors before pooling fixed-effect or random-effects means.When means were analyzed with median-based methods, they were treated as medians, assuming approximate symmetry.
  • 3.4. Performance measures: Performance was assessed using percent error, absolute percent error, mean squared error, and 95% confidence-interval coverage.The truth was defined as the generating mean for mean-based approaches and the generating median for median-based approaches.

4. Results

Median-based approaches generally outperformed transformation-based approaches when studies reported medians, particularly under high skewness or inter-study variance, while treating means as medians performed poorly.

  • Overall results: 50 studies and a median study size of 100 were retained because study count and study size did not considerably differentiate performance.This conclusion was based on absolute percent error, percent error, mean squared error, and 95% confidence-interval coverage.
  • Overall results: When all studies reported medians, median-based approaches had lower median absolute percent error and approximately 0% overall median percent error than transformation-based approaches.Their target was the true median, whereas transformation methods targeted the true mean.
  • Skewness: With highly skewed data, transformation-based approaches had median absolute percent error of 66% or higher, whereas median-based approaches remained below 8%.These results applied when only medians were presented and inter-study variance was fixed at τ^2 = 1/4.
  • Inter-study variance: At inter-study variance τ^2 = 4, median-based approaches had median absolute percent error under 30%, compared with over 80% for transformation-based approaches.This comparison used high mean skew and targeted the true median versus the true mean, respectively.
  • Mean squared error: Median-based methods had nearly 0 median mean squared error with median-only or mixed reporting, whereas treating means as medians increased mean squared error.The overall median mean squared error for MM and WM was nearly 0 with medians and 1 with means.
  • Confidence-interval coverage: MM achieved nominal or near-nominal 95% confidence-interval coverage across skew levels, while WM coverage ranged from 88% to 89%.Random-effects transformation approaches had lower coverage for the true mean in the reported comparison.
  • Sensitivity analysis: Approximately 92% of studies reported medians in the Shapiro-Wilk sensitivity analysis, with median reporting increasing as skewness and inter-study variance increased.Median reporting reached approximately 100% at very high skewness and nearly 100% when τ^2 = 4.
  • Worst-case performance: Under the worst-case simulation setting, the median of study-specific medians produced median absolute percent error of 41%.The worst case combined low study counts and study sizes with high inter-study variance and skewness.

5. Example: Estimating Patient Delay in Pulmonary Tuberculosis Diagnosis

The methods were applied to pooled patient delay in tuberculosis diagnosis in China using 50 studies reporting median delay, producing substantially different pooled estimates across approaches.

  • Data: The data set contained 50 Chinese studies of patient delay, all reporting median delay; 49 also reported the number of subjects.Patient delay was defined as time from symptom onset to first contact with a health care provider.
  • Methods: MM was applied to all 50 studies, while WM used the 49 studies reporting numbers of subjects with median delay.Median-based analyses excluded an outlier study in the presented results.
  • Methods: Transformation methods used reported quartiles or minima and maxima to estimate study means and standard errors before pooling.When both spread measures were available, only the first and third quartiles were retained.
  • Results: MM estimated patient delay at approximately 16 days, whereas WM estimated 38 days because two studies with the largest delays contained over half the total subjects.The two median-based estimates therefore differed substantially in this application.
  • Interpretation: The simulation indicated that the minimum–maximum transformation method was highly inaccurate and performed worse than the quartile-based transformation method.The application also showed considerable variation in estimated study means.

6. Discussion

The median-based approaches outperformed transformation-based approaches, especially with skewed data or high inter-study variance, while retaining useful coverage. The discussion also identifies scope limitations and recommends reporting both means and medians when possible.

  • Median-based approaches had lower absolute percent error than transformation-based approaches, with nominal or near-nominal coverage and percent error near 0%.This pattern held when meta-analyzing medians or mixtures of means and medians.
  • Median-based methods require neither reported spread nor distributional assumptions and are insensitive to correlations between effect estimates and variances.Inverse-variance weighting can be substantially biased when those quantities are correlated.
  • Median-based methods performed better than analyses using actual study means and standard errors when data were highly skewed or had high inter-study variance.With approximately symmetric data and low inter-study variance, their performance was comparable.
  • The median-based approaches do not provide estimates of inter-study heterogeneity, although transformation-based approaches estimated heterogeneity poorly in the simulation.Heterogeneity estimates may still be useful for deciding whether studies should be summarized together.
  • Simulation results were limited in generalizability because outcomes were generated only from log-normal distributions.Performance may change for data generated from other distributions.
  • Authors are encouraged to report means or medians according to skewness, preferably reporting both because additional information improves pooled-estimate accuracy.Analysts are advised to select an approach based on skewness, using median-based methods when mean Bowley’s coefficient exceeds 0.1.
  • In the tuberculosis example, mean Bowley’s skewness was 0.38 and pooled estimates varied substantially between unweighted and weighted medians when an outlier study was included.Including the outlier produced estimates of 18 days and 60 days, respectively.
  • Future work will estimate sampling variance for medians from commonly reported summary measures, enabling inverse-variance pooling.

Appendix A

Appendix A presents absolute-percent-error results for median-based and transformation-based approaches across simulation conditions. The tables restrict attention to 50 studies and a median study size of 100 subjects.

  • The Appendix A tables report median absolute percent error with first and third quartiles across inter-study variance and mean-skew levels.
  • Table A1 contains results for median-based approaches.
  • Table A2 contains transformation-based results and compares them with MEANSFE or MEANSRE absolute percent error in the “Means Given” column.

Appendix B

Appendix B presents percent-error results for median-based and transformation-based approaches across inter-study variance and mean-skew levels. The tables use simulations with 50 studies and a median study size of 100 subjects.

  • The Appendix B tables report percent error summarized by its median, first quartile, and third quartile.
  • Table B1 reports results for median-based approaches.
  • Table B2 reports transformation-based results and gives percent error for MEANSFE or MEANSRE in the “Means Given” column.

Appendix C

Appendix C reports mean-squared-error results across inter-study variance and mean-skew levels, with interaction plots for primary and sensitivity analyses. Results are restricted to simulations with 50 studies and a median study size of 100 subjects.

  • The Appendix C tables summarize MSE using the median, first quartile, and third quartile across inter-study variance and mean-skew combinations.
  • Table C1 contains results for median-based approaches.
  • Table C2 contains transformation-based results and reports MSE for MEANSFE or MEANSRE in the “Means Given” column.
  • Figure C1 shows interaction plots by mean skew level, with performance measured by MSE.
  • Figure C2 shows interaction plots by inter-study variance, with performance measured by MSE.

Appendix D

Appendix D presents coverage results for median-based and transformation-based approaches, including comparisons with approaches using reported means. The tables restrict simulations to 50 studies and a median of 100 subjects per study, excluding absolute percent errors above 500%.

  • Coverage results: The appendix tables report 95% CI coverage across combinations of inter-study variance and mean skew levels.The displayed results distinguish median-based approaches from transformation-based approaches, with a separate table describing coverage for approaches using reported means.
  • Data handling: Studies with absolute percent error greater than 500% were removed before calculating the tabulated coverage results.Entries marked NA correspond to scenarios not observed in the simulation study.
  • Coverage results: The reported coverage values vary across low and very high mean-skew levels and across inter-study variance settings.The appendix includes rows for variance settings such as 1 and 4, with corresponding low and very high skew categories.
Loading 1709.03016v2…