Multiple Comparisons/Testing Issues

  • Qingrun ZhangEmail author
  • Jurg Ott


The statistical testing of multiple genetic markers in genetic linkage and association studies is discussed and shown to lead to a multiple-testing problem. Various solutions are discussed and demonstrated on published data. The false discovery rate (FDR) and several approaches of estimating it, are mentioned. Randomization (permutation) testing is highly recommended.


False Discovery Rate Genomewide Association Study Control Association Study Permutation Sample Weinberg Disequilibrium 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Clerget-Darpoux F, Babron MC, Bonaiti-Pellie C (1987) Power and robustness of the linkage homogeneity test in genetic analysis of common disorders. J Psychiatr Res 21:625–630CrossRefPubMedGoogle Scholar
  2. 2.
    Sherrington R, Brynjolfsson J, Petursson H, Potter M, Dudleston K, Barraclough B, Wasmuth J, Dobbs M, Gurling H (1988) Localization of a susceptibility locus for schizophrenia on chromosome 5. Nature 336:164–167CrossRefPubMedGoogle Scholar
  3. 3.
    Bonferroni CE (1936) Teoria statistica delle classi e calcolo delle probabilità. Pubblicazioni del Istituto Superiore di Scienze Economiche e Commerciali di Firenze 8:3–62Google Scholar
  4. 4.
    Cheverud JM (2001) A simple correction for multiple comparisons in interval mapping genome scans. Heredity 87:52–58CrossRefPubMedGoogle Scholar
  5. 5.
    Devlin B, Roeder K, Wasserman L (2003) Analysis of multilocus models of association. Genet Epidemiol 25:36–47CrossRefPubMedGoogle Scholar
  6. 6.
    Storey JD, Tibshirani R (2003) Statistical significance for genomewide studies. Proc Natl Acad Sci U S A 100:9440–9445CrossRefPubMedGoogle Scholar
  7. 7.
    Efron B (2004) Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis. Journal of the American Statistical Association 99:96–104CrossRefGoogle Scholar
  8. 8.
    Benjamini Y, Drai D, Elmer G, Kafkafi N, Golani I (2001) Controlling the false discovery rate in behavior genetics research. Behav Brain Res 125:279–284CrossRefPubMedGoogle Scholar
  9. 9.
    Klein RJ, Zeiss C, Chew EY, Tsai JY, Sackler RS, Haynes C, Henning AK, SanGiovanni JP, Mane SM, Mayne ST, Bracken MB, Ferris FL, Ott J, Barnstable C, Hoh J (2005) Complement factor H polymorphism in age-related macular degeneration. Science 308:385–389. Epub 2005 Mar 2010Google Scholar
  10. 10.
    Fisher RA (1935) The design of experiments. Oliver & Boyd, EdinburghGoogle Scholar
  11. 11.
    Manly BFJ (2007) Randomization, bootstrap, and Monte Carlo methods in biology. Chapman & Hall/CRC, Boca Raton, FLGoogle Scholar
  12. 12.
    Nijenhuis A, Wilf HS (1978) Combinatorial algorithms for computers and calculators. Academic, New YorkGoogle Scholar
  13. 13.
    Sham PC, Curtis D (1995) Monte Carlo tests for associations between disease and alleles at highly polymorphic loci. Ann Hum Genet 59:97–105CrossRefPubMedGoogle Scholar
  14. 14.
    Kimmel G, Shamir R (2006) A fast method for computing high-significance disease association in large population-based studies. Am J Hum Genet 79:481–492CrossRefPubMedGoogle Scholar
  15. 15.
    Skol AD, Scott LJ, Abecasis GR, Boehnke M (2006) Joint analysis is more efficient than replication-based analysis for two-stage genome-wide association studies. Nat Genet 38:209–213. Epub 2006 Jan 2015Google Scholar
  16. 16.
    Lazzeroni LC, Lange K (1998) A conditional inference framework for extending the transmission/disequilibrium test. Hum Hered 48:67–81CrossRefPubMedGoogle Scholar
  17. 17.
    Hoh J, Wille A, Ott J (2001) Trimming, weighting, and grouping SNPs in human case-control association studies. Genome Res 11:2115–2119CrossRefPubMedGoogle Scholar
  18. 18.
    McIntyre LM, Martin ER, Simonsen KL, Kaplan NL (2000) Circumventing multiple testing: a multilocus Monte Carlo approach to testing for association. Genet Epidemiol 19:18–29CrossRefPubMedGoogle Scholar
  19. 19.
    Becker T, Cichon S, Jonson E, Knapp M (2005) Multiple testing in the context of haplotype analysis revisited: application to case-control data. Ann Hum Genet 69:747–756CrossRefPubMedGoogle Scholar
  20. 20.
    Hoh J, Ott J (2000) Scan statistics to scan markers for susceptibility genes. Proc Natl Acad Sci U S A 97:9615–9617CrossRefPubMedGoogle Scholar
  21. 21.
    Zheng G, Freidlin B, Gastwirth JL (2006) Comparison of robust tests for genetic association using case-control studies. IMS Lecture Notes-Monograph Series 49:253–265CrossRefGoogle Scholar
  22. 22.
    Matthews AG, Haynes C, Liu C, Ott J (2008) Collapsing SNP genotypes in case-control genome-wide association studies increases the type I error rate and power. Statistical Applications in Genetics and Molecular Biology 7:Art. 23Google Scholar
  23. 23.
    Zhang Q, Wang S, Ott J (2008) Combining identity by descent and association in genetic case-control studies. BMC Genet 9:42CrossRefPubMedGoogle Scholar
  24. 24.
    Fung HC, Scholz S, Matarin M, Simon-Sanchez J, Hernandez D, Britton A, Gibbs JR, Langefeld C, Stiegert ML, Schymick J, Okun MS, Mandel RJ, Fernandez HH, Foote KD, Rodriguez RL, Peckham E, De Vrieze FW, Gwinn-Hardy K, Hardy JA, Singleton A (2006) Genome-wide genotyping in Parkinson’s disease and neurologically normal controls: first stage analysis and public release of data. Lancet Neurol 5:911–916CrossRefPubMedGoogle Scholar
  25. 25.
    Balding DJ (2006) A tutorial on statistical methods for population association studies. Nat Rev Genet 7:781–791CrossRefPubMedGoogle Scholar
  26. 26.
    Lander E, Kruglyak L (1995) Genetic dissection of complex traits: guidelines for interpreting and reporting linkage results. Nat Genet 11:241–247CrossRefPubMedGoogle Scholar
  27. 27.
    Pe’er I, Yelensky R, Altshuler D, Daly MJ (2008) Estimation of the multiple testing burden for genomewide association studies of nearly all common variants. Genet Epidemiol 32:381–385CrossRefPubMedGoogle Scholar
  28. 28.
    Ott J (2004) Association of genetic loci: Replication or not, that is the question. Neurology 63:955–958PubMedGoogle Scholar
  29. 29.
    Gotzsche PC (2006) Believability of relative risks and odds ratios in abstracts: cross sectional study. Bmj 333:231–234CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  1. 1.Beijing Institute of GenomicsChinese Academy of SciencesBeijingChina

Personalised recommendations