PaperlayerView exact revision on arXiv

Source-linked AI summary

Using Genetic Distance to Infer the Accuracy of Genomic Prediction

Marco Scutari, Ian Mackay, David Balding

arXiv:1509.00415v4stat.MEq-bio.GNq-bio.QM

TL;DR

Genomic prediction is often evaluated under same-population assumptions, though applications may require predictions for genetically distinct individuals. The paper estimates predictive-accuracy decay across increasing genetic distances, finding mostly approximately linear decay in FST and accurate curves across varied data settings. These curves can help compare training populations and decide when to retrain, subject to a shared genetic basis across populations.

  • Problem

    Random-split cross-validation assumes test individuals come from the training population, while practical predictions often target genetically distinct individuals and generalization to distant populations has been little investigated.

    Source: Introduction · p. 2

  • Method

    The approach predicts across increasingly differentiated training-population subsets and relates mean Pearson correlation to FST, while also considering mean kinship.

    Source: Introduction · p. 2

  • Results

    Most decay curves were approximately linear in FST, and estimates were accurate across different settings and human, plant, and animal data sets.

    Source: Discussion · p. 14

  • Takeaways & Limitations

    Decay curves can compare training populations and help decide when prediction accuracy is no longer sufficient for selection and the model should be retrained.

    Source: Discussion · p. 16

    The trait must share a genetic basis across training and target populations, and training-population homogeneity can limit the estimated curve’s FST range.

    Source: Discussion · p. 15

Abstract

from arXiv · show

The prediction of phenotypic traits using high-density genomic data has many applications such as the selection of plants and animals of commercial interest; and it is expected to play an increasing role in medical diagnostics. Statistical models used for this task are usually tested using cross-validation, which implicitly assumes that new individuals (whose phenotypes we would like to predict) originate from the same population the genomic prediction model is trained on. In this paper we propose an approach based on clustering and resampling to investigate the effect of increasing genetic distance between training and target populations when predicting quantitative traits. This is important for plant and animal genetics, where genomic selection programs rely on the precision of predictions in future rounds of breeding. Therefore, estimating how quickly predictive accuracy decays is important in deciding which training population to use and how often the model has to be recalibrated. We find that the correlation between true and predicted values decays approximately linearly with respect to either $\F$ or mean kinship between the training and the target populations. We illustrate this relationship using simulations and a collection of data sets from mice, wheat and human genetics.

Author Summary

The section concerns how studies generalize to different populations.

Introduction

The introduction frames the challenge of predicting quantitative traits in genetically distinct populations and outlines a distance-based approach for examining how predictive accuracy changes.

Introduction

  • Generalization to distantly related target populations has been little investigated, although many applications require predictions beyond the training population.Source: Introduction · p. 2
  • Random-split cross-validation assumes training and test individuals come from the same population, unlike many genomic-selection prediction settings.Source: Introduction · p. 2
  • Greater distance between training and target populations is associated with lower average predictive accuracy, as shown in dairy cattle studies.Source: Introduction · p. 2
  • The approach splits training samples into increasingly differentiated pairs, predicts across each pair, and relates mean Pearson correlation to genetic distance.The paper considers mean kinship and FST, while focusing on FST.Source: Introduction · p. 2
  • The decay curve is proposed to compare training populations, guide model retraining timing, and inform which distantly related individuals to introduce.Source: Introduction · p. 2

Materials and Methods

The methods estimate predictive-accuracy decay across genetic distance by forming training and target subsets. They examine decay curves in simulations and wheat and mouse data, comparing them with population-level predictions or hold-out cross-validation.

Genomic Prediction Models

  • Genomic prediction uses GBLUP or marker-effect models, with the study focusing on elastic net and predictive correlation, which random-split cross-validation may overestimate for more distant targets.The paper notes that random-split cross-validation can remain optimistic when target individuals differ more from training individuals than the test samples do.Source: Genomic Prediction Models · pp. 3–4

Kinship Coefficients and FST

  • Mean kinship and estimated FST are inversely related, and their high correlation supports reporting FST as the genetic-distance measure for decay curves.The adopted FST estimator is obtained by maximizing a Beta-Binomial likelihood of allele frequencies.Source: Kinship Coefficients and FST · p. 4

Real-World Data Sets

  • The evaluation uses wheat, mice, and human data, spanning continuous traits and ancestry groups.Wheat includes 376 varieties, 2,712 markers, and four traits; mice include 1,940 individuals with growth-rate and weight records; human data include 1,043 individuals and 90,487 SNPs from chromosomes 1 and 2. Wheat genotype-environment interactions were accounted for in an incomplete-block design.Source: Real-World Data Sets · pp. 4–5

Decay Curves for Predictive Accuracy

  • The decay procedure clusters individuals into maximally divergent training and target subsets, swaps individuals between them, refits models, and uses LOESS to estimate predictive correlation across FST.The procedure repeats each swap 40 times; its linear-regression alternative can extrapolate beyond the initial FST but becomes unreliable as predicted correlation approaches zero.Source: Decay Curves for Predictive Accuracy · pp. 5–6

Simulation Studies

  • The genomic-selection simulation uses wheat varieties registered in the last five years of WHEAT as founders, runs forward simulations with 10, 50, 200, or 1,000 causal variants, and produces decay curves for each.An augmented scenario raises the training population to 800 varieties, compared with the original population of 200. Separately, cross-population prediction uses HUMAN data and simulated phenotypes, with 5, 20, 100, 2,000, 10,000, or 50,000 causal variants selected among variants with minor allele frequency >5%.Source: Simulation Studies · pp. 6–7
  • Asian individuals served as the simulation training population, while individuals from other continents were targets used to assess whether decay curves matched their F_ST and predictive-correlation points or crossed their 95% confidence intervals.Effect sizes were held constant across populations.Source: Simulation Studies · p. 7

Real-World Data Analyses

  • For WHEAT and MICE, 40 hold-out cross-validation estimates averaged ρ̂_CV used training and target subsamples sized to match those from clustering, with elastic net predictions on target subsamples.The decay curve was expected to cross the region where the (F̂_ST, ρ̂_CV) points clustered.Source: Real-World Data Analyses · pp. 7–8
  • WHEAT decay curves covered yield, height, flowering time, and grain protein content using French varieties for training and UK and German varieties for targets; a second yield curve used pre-1990 varieties for training, as in.For the second curve, varieties registered in 1990–1999 and after 2000 were targets.Source: Real-World Data Analyses · p. 8
  • MICE decay curves for growth rate and weight used each F005, F008, F010, and F016 inbred family in turn as training, with the remaining families as targets.Source: Real-World Data Analyses · p. 8

Results

Across simulations and real-world data, decay-curve estimates generally tracked linear interpolation closely. In the WHEAT and MICE data, cross-validation pairs were near the low-distance end of the curves.

General Considerations

  • 87% of decay-curve estimates differed from linear interpolation by ≪0.02; estimates were within ≪0.05 of reference correlations in 39 of 94 comparisons.The decay-curve and linear-interpolation difference was ≪0.02 in 41 of 47 cases.Source: General Considerations · p. 8
  • Within the decay-curve span, 23 of 44 target estimates differed by ≪0.05 and 38 by ≪0.10; extending the curve with linear interpolation reduces precision.Regression did not produce a stronger linear relationship than linear interpolation (p = 0.784).Source: General Considerations · pp. 8–9

Real-World Data Analyses

  • In WHEAT and MICE cross-validation, average F_ST was 0.006 and 0.001, respectively, and average correlations were within ≪0.02 of decay estimates in 10 of 12 cases.The spread of cross-validation correlations was also similar to the spread around the decay curves.Source: Real-World Data Analyses · p. 9
  • In MICE, decay curves or their linear approximations crossed target-population 95% confidence intervals in 18 of 24 cases, though those intervals were wide.In WHEAT, all curves except flowering time crossed the reported Germany and UK intervals.Source: Real-World Data Analyses · p. 9
  • For flowering time, balancing Ppd-D1a alleles across WHEAT halves yielded ρ_D=0.77 and ρ_P=0.79, unlike the overestimates for Germany and the UK.For yield, ρ_D=0.44 versus ρ_P=0.40 for 1990–1999 varieties, and ρ_D=0.44 versus ρ_P=0.42 for post-2000 varieties.Source: Real-World Data Analyses · p. 9

Simulation Studies

  • With 200 or 1000 causal variants, decay curves tracked generation means within 0.06 for the first three generations, apart from a 0.09 difference in one case.With 50 or 10 variants, curves increasingly overestimated the means, consistent with large-effect alleles fixing under selection and reducing accuracy without a matching F_ST increase.Source: Simulation Studies · p. 10
  • With an augmented training population of 800 varieties, decay curves fit the first four generations within 0.04.For 200 causal variants, linear approximations differed from generation means by about 0.08 in generations 5–7 and 0.10 in generations 8–9; for 1000 variants, differences were ≪0.05 in generations 4–9.Source: Simulation Studies · pp. 10–11
  • In the HUMAN cross-population simulation, linear approximations crossed distant-population 95% confidence intervals in 14 of 18 cases.Decay behavior varied with causal-variant count, and curve quality did not appear to degrade as that count decreased.Source: Simulation Studies · p. 12

Discussion

The discussion presents decay curves as a model-agnostic way to estimate how predictive accuracy changes with genetic distance, while emphasizing that linearity and extrapolation have important limits. It connects these estimates to breeding decisions and predictions for related populations without direct training data.

Discussion

  • The approach avoids GBLUP’s parametric assumptions and can be used with competitive models that estimate predictive correlation.The authors demonstrate decay-curve accuracy across human, plant, and animal real-world data sets.Source: Discussion · p. 14
  • Predictive-correlation decay is approximately linear in FST for most curves, making regression a cheaper alternative to LOESS.Regression needs fewer FST–decay points and therefore fewer genomic prediction models to fit.Source: Discussion · p. 14
  • Linear approximations are most appropriate when predictive correlation remains well above zero; low-heritability traits and other factors can produce nonlinear decay.Departures from linearity can also reflect causal-variant count, small training populations, and exogenous effects.Source: Discussion · pp. 14–15
  • Unlike, the analyses find no statistically significant difference between linear relationships for FST and its square.The cited simulation differed in its use of simulated dairy-cattle genotypes and GBLUP.Source: Discussion · p. 14
  • Decay curves are less reliable for less-polygenic traits: selection can shift key causal alleles and sharply reduce accuracy while genome-wide FST changes little.In wheat flowering time, the Ppd-D1a allele differs across countries, and the curve overestimates predictive accuracy despite small FST values.Source: Discussion · pp. 14–15
  • In the HUMAN simulation, shared causal variants can yield agreement between decay estimates and predictive accuracy, but equal effect sizes may make estimates optimistic.Strong effects that carry across populations can flatten the curve, causing linear interpolation to underestimate predictive accuracy.Source: Discussion · p. 15
  • Extrapolation loses precision as FST increases, and predictions may be poor for very genetically distinct targets.The target and training populations must share a genetic basis; denser genomic data and larger samples may improve accuracy and precision.Source: Discussion · p. 15
  • Breeding programs can compare training populations and plan retraining; decay curves may also estimate accuracy for related non-descendant targets.The authors identify cross-country wheat, cross-family mice, and human populations as examples; close ancestors in training suggest reliability for planning.Source: Discussion · p. 16

Supporting Information

The supporting information lists supplementary Methods and Results, simulation decay-curve figures, and analyses of relationships involving F_ST, mean kinship, and predictive correlation.

S1 Text.

  • The supplement includes additional Methods and Results.Source: S1 Text · p. 16
  • It includes decay curves from the simulation studies.Source: S1 Text · p. 16
  • It lists analyses of the relationship between F_ST and mean kinship, and comparisons involving ρ² and F_ST² versus ρ and F_ST.Source: S1 Text · p. 16

A.1 Breeding Program Simulation using the WHEAT data

The supplement provides predictive-correlation tables for two wheat breeding-program simulations, with different training populations.

A.1 Breeding Program Simulation using the WHEAT data

  • Table A.1 reports predictive correlations for simulations using 200 varieties from the 2002–2007 WHEAT data as the training population.It defines averages by generation, training-population size, and number of causal variants, and describes decay-curve and linear-extrapolation estimates.Source: Table A.1 · p. 22
  • Table A.2 reports predictive correlations for simulations using the 800 varieties available after the second selection round as the training population.Its notation matches Table A.1.Source: Table A.2 · p. 23

A.2 Cross-Population Simulation using the HUMAN data

Table A.3 reports predictive correlations for simulations using HUMAN data, comparing target-population correlations with decay-curve and linear-extrapolation estimates.

A.2 Cross-Population Simulation using the HUMAN data

  • Table A.3 defines ρP as the target-population correlation from the full training population and gives ρD and ρL estimates when available.ρD is available when the target population falls within the decay curve's span; ρL is the linear-extrapolation estimate.Source: Table A.3 · p. 24

B.1 WHEAT Data

The WHEAT section presents decay curves for four traits using French varieties and identifies the plotted trend, estimates, and target-population comparisons.

B.1 WHEAT Data

  • Figure B.1 presents decay curves for grain yield, height, flowering time, and grain protein content estimated from French wheat varieties.Source: Figure B.1 · p. 25
  • The plot distinguishes the mean decay trend and its 95% confidence interval from the linear interpolation, cross-validation estimates, and British and German target-population correlations.Red brackets show the target-correlation 95% confidence intervals.Source: §B.1 · p. 25

B.2 MICE Data

The passages describe MICE decay curves for weight and growth rate, cross-validation comparisons, and genetic-distance summaries spanning wheat, mice, and human data.

B.2 MICE Data

  • Decay curves for weight and growth rate are estimated from the four largest MICE families; red squares mark held-out population correlations, with 95% confidence intervals.The families are labelled F005, F008, F010, and F016.Source: Figure B.2 · p. 26, Figure B.3 · p. 27

B.3 Cross-Validation and Decay Curve in the WHEAT and MICE data

  • Table B.4 reports decay-curve and cross-validation predictive correlations, including mean genetic distance and the decay-curve estimate at that distance.The table covers analyses shown in Figures B.1, B.2, and B.3.Source: Table B.4 · p. 28

C Kinship and FST

D Relationship between Squared Predictive Correlation and F 2

The analysis compares linear and quadratic relationships between predictive correlation and genetic distance, building on a prior proposal to relate squared correlation to squared mean kinship. The difference from that proposed relationship is not statistically significant in the reported test.

  • The analysis treats FST and mean kinship as interchangeable because they are described as almost perfectly linearly correlated, and compares linear with quadratic fits.The linear fit regresses estimated decay-curve predictive correlation on estimated FST and measures R2LINEAR; both fit measures are reported for real and simulated phenotypes.Source: D Relationship between Squared… · p. 32
  • The permutation test finds no significant difference between the paper’s relationship and the one suggested in (p = 0.784).The test uses 10,000 permutations.Source: D Relationship between Squared… · p. 32
  • The paper reports R2LINEAR and R2QUADRATIC separately for real-phenotype analyses and simulation-study data.Tables D.6 and D.7 correspond to those two analysis groups.Source: Table D.6 · p. 33, Table D.7 · p. 33
Loading 1509.00415v4…