#jsDisabledContent { display:none; } My Account |  Register |  Help

# Model selection

Article Id: WHEBN0003664073
Reproduction Date:

 Title: Model selection Author: World Heritage Encyclopedia Language: English Subject: Collection: Publisher: World Heritage Encyclopedia Publication Date:

### Model selection

Model selection is the task of selecting a statistical model from a set of candidate models, given data. In the simplest cases, a pre-existing set of data is considered. However, the task can also involve the design of experiments such that the data collected is well-suited to the problem of model selection. Given candidate models of similar predictive or explanatory power, the simplest model is most likely to be the best choice.[1]

Konishi & Kitagawa (2008, p. 75) state, "The majority of the problems in statistical inference can be considered to be problems related to statistical modeling". Relatedly, Sir David Cox (2006, p. 197) has said, "How [the] translation from subject-matter problem to statistical model is done is often the most critical part of an analysis".

## Contents

• Introduction 1
• Methods for choosing the set of candidate models 2
• Criteria for model selection 3
• Notes 5
• References 6

## Introduction

The scientific observation cycle.

In its most basic forms, model selection is one of the fundamental tasks of scientific inquiry. Determining the principle that explains a series of observations is often linked directly to a mathematical model predicting those observations. For example, when Galileo performed his inclined plane experiments, he demonstrated that the motion of the balls fitted the parabola predicted by his model.

Of the countless number of possible mechanisms and processes that could have produced the data, how can one even begin to choose the best model? The mathematical approach commonly taken decides among a set of candidate models; this set must be chosen by the researcher. Often simple models such as polynomials are used, at least initially. Burnham & Anderson (2002) emphasize throughout their book the importance of choosing models based on sound scientific principles, such as understanding of the phenomenological processes or mechanisms (e.g., chemical reactions) underlying the data.

Once the set of candidate models has been chosen, the statistical analysis allows us to select the best of these models. What is meant by best is controversial. A good model selection technique will balance goodness of fit with simplicity. More complex models will be better able to adapt their shape to fit the data (for example, a fifth-order polynomial can exactly fit six points), but the additional parameters may not represent anything useful. (Perhaps those six points are really just randomly distributed about a straight line.) Goodness of fit is generally determined using a likelihood ratio approach, or an approximation of this, leading to a chi-squared test. The complexity is generally measured by counting the number of parameters in the model.

Model selection techniques can be considered as estimators of some physical quantity, such as the probability of the model producing the given data. The bias and variance are both important measures of the quality of this estimator; efficiency is also often considered.

A standard example of model selection is that of curve fitting, where, given a set of points and other background knowledge (e.g. points are a result of i.i.d. samples), we must select a curve that describes the function that generated the points.

## Criteria for model selection

The most commonly-used criteria are (i) the Akaike information criterion and (ii) the Bayes factor and/or the Bayesian information criterion (which to some extent approximates the Bayes factor).

## Notes

1. ^ This follows directly from formal expressions of Occam's Razor such as Minimum Message Length and others.

## References

• Aho, K.; Derryberry, D.; Peterson, T. (2014), "Model selection for ecologists: the worldviews of AIC and BIC", .
• Anderson, D.R. (2008), Model Based Inference in the Life Sciences, Springer.
• Ando, T. (2010), Bayesian Model Selection and Statistical Modeling, CRC Press.
• Burnham, K.P.; Anderson, D.R. (2002), Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach, 2nd edition, Springer-Verlag. ISBN 0-387-95364-7 [This has over 25000 citations on Google Scholar.]
• Chamberlin, T.C. (1890), "The method of multiple working hypotheses", Science 15: 93. (Reprinted 1965, Science 148: 754–759 [2].)
• Claeskens, G.; Hjort, N.L. (2008), Model Selection and Model Averaging, Cambridge University Press.
• Cox D.R. (2006), Principles of Statistical Inference, Cambridge University Press.
• Konishi, S.; Kitagawa, G. (2008), Information Criteria and Statistical Modeling, Springer.
• Lahiri, P. (2001), Model Selection, Institute of Mathematical Statistics.
• Leeb H., Pötscher B. M. (2009), "Model selection", Handbook of Financial Time Series (editor—T. G. Anderson), p. 889-925, Springer; doi:10.1007/978-3-540-71297-8_39.
• Lukacs, P. M.; Thompson, W. L.; Kendall, W. L.; Gould, W. R.; Doherty, P. F. Jr.; Burnham, K. P.; Anderson, D. R. (2007), "Concerns regarding a call for pluralism of information theory and hypothesis testing", .
• Massart, P. (2007), Concentration Inequalities and Model Selection, Springer.
• Wit, E.; van den Heuvel, E.; Romeijn, J.-W. (2012), "‘All models are wrong...’: an introduction to model uncertainty" (PDF), Statistica Neerlandica 66: 217–236, .
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.

Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.