World Library  
Flag as Inappropriate
Email this Article

Identifiability

Article Id: WHEBN0022884398
Reproduction Date:

Title: Identifiability  
Author: World Heritage Encyclopedia
Language: English
Subject: Completeness (statistics), Identification, Two-way analysis of variance, Regression discontinuity design, Maximum likelihood
Collection:
Publisher: World Heritage Encyclopedia
Publication
Date:
 

Identifiability

In statistics, identifiability is a property which a model must satisfy in order for precise inference to be possible. We say that the model is identifiable if it is theoretically possible to learn the true value of this model’s underlying parameter after obtaining an infinite number of observations from it. Mathematically, this is equivalent to saying that different values of the parameter must generate different probability distributions of the observable variables. Usually the model is identifiable only under certain technical restrictions, in which case the set of these requirements is called the identification conditions.

A model that fails to be identifiable is said to be non-identifiable or unidentifiable; two or more parametrizations are observationally equivalent. In some cases, even though a model is non-identifiable, it is still possible to learn the true values of a certain subset of the model parameters. In this case we say that the model is partially identifiable. In other cases it may be possible to learn the location of the true parameter up to a certain finite region of the parameter space, in which case the model is set identifiable.

Definition

Let ℘ = {Pθ: θ∈Θ} be a statistical model where the parameter space Θ is either finite- or infinite-dimensional. We say that ℘ is identifiable if the mapping θPθ is one-to-one:[1]

P_{\theta_1}=P_{\theta_2} \quad\Rightarrow\quad \theta_1=\theta_2 \quad\ \text{for all } \theta_1,\theta_2\in\Theta.

This definition means that distinct values of θ should correspond to distinct probability distributions: if θ1θ2, then also Pθ1Pθ2.[2] If the distributions are defined in terms of the probability density functions, then two pdfs should be considered distinct only if they differ on a set of non-zero measure (for example two functions ƒ1(x)=10≤x<1 and ƒ2(x)=10≤x≤1 differ only at a single point x=1 — a set of measure zero — and thus cannot be considered as distinct pdfs).

Identifiability of the model in the sense of invertibility of the map θPθ is equivalent to being able to learn the model’s true parameter if the model can be observed indefinitely long. Indeed, if {Xt}⊆S is the sequence of observations from the model, then by the strong law of large numbers,

\frac{1}{T} \sum_{t=1}^T \mathbf{1}_{\{X_t\in A\}} \ \xrightarrow{a.s.}\ \operatorname{Pr}[X_t\in A],

for every measurable set AS (here 1{…} is the indicator function). Thus with an infinite number of observations we will be able to find the true probability distribution P0 in the model, and since the identifiability condition above requires that the map θPθ be invertible, we will also be able to find the true value of the parameter which generated given distribution P0.

Examples

Example 1

Let ℘ be the normal location-scale family:

\mathcal{P} = \Big\{\ f_\theta(x) = \tfrac{1}{\sqrt{2\pi}\sigma} e^{ -\frac{1}{2\sigma^2}(x-\mu)^2 }\ \Big|\ \theta=(\mu,\sigma): \mu\in\mathbb{R}, \,\sigma\!>0 \ \Big\}.

Then

\begin{align} f_{\theta_1}=f_{\theta_2}\\ &\Leftrightarrow\ \tfrac{1}{\sqrt{2\pi}\sigma_1}e^{ -\frac{1}{2\sigma_1^2}(x-\mu_1)^2 } = \tfrac{1}{\sqrt{2\pi}\sigma_2}e^{ -\frac{1}{2\sigma_2^2}(x-\mu_2)^2 } \\ &\Leftrightarrow\ \tfrac{1}{\sigma_1^2}(x-\mu_1)^2 + \ln \sigma_1 = \tfrac{1}{\sigma_2^2}(x-\mu_2)^2 + \ln \sigma_2 \\ &\Leftrightarrow\ x^2\big(\tfrac{1}{\sigma_1^2}-\tfrac{1}{\sigma_2^2}\big) - 2x\big(\tfrac{\mu_1}{\sigma_1^2}-\tfrac{\mu_2}{\sigma_2^2}\big) + \big(\tfrac{\mu_1^2}{\sigma_1^2}-\tfrac{\mu_2^2}{\sigma_2^2}+\ln\sigma_1-\ln\sigma_2\big) = 0 \\ \end{align}

This expression is equal to zero for almost all x only when all its coefficients are equal to zero, which is only possible when |σ1| = |σ2| and μ1 = μ2. Since in the scale parameter σ is restricted to be greater than zero, we conclude that the model is identifiable: ƒθ1θ2θ1=θ2.

Example 2

Let ℘ be the standard linear regression model:

y = \beta'x + \varepsilon, \quad \operatorname{E}[\,\varepsilon|x\,]=0

(where ′ denotes matrix transpose). Then the parameter β is identifiable if and only if the matrix E[xx′] is invertible. Thus, this is the identification condition in the model.

Example 3

Suppose ℘ is the classical errors-in-variables linear model:

\begin{cases} y = \beta x^* + \varepsilon, \\ x = x^* + \eta, \end{cases}

where (ε,η,x*) are jointly normal independent random variables with zero expected value and unknown variances, and only the variables (x,y) are observed. Then this model is not identifiable,[3] only the product βσ² is (where σ² is the variance of the latent regressor x*). This is also an example of set identifiable model: although the exact value of β cannot be learned, we can guarantee that it must lie somewhere in the interval (βyx, 1÷βxy), where βyx is the coefficient in OLS regression of y on x, and βxy is the coefficient in OLS regression of x on y.[4]

If we abandon the normality assumption and require that x* were not normally distributed, retaining only the independence condition εηx*, then the model becomes identifiable.[3]

Software

In the case of parameter estimation in partially observed dynamical systems, the profile likelihood can be also used for structural and practical identifiability analysis.[5] An implementation of the Profile Likelihood Approach is available in the MATLAB Toolbox PottersWheel.

See also

Notes

  1. ^ Lehmann & Casella 1998, Definition 1.5.2
  2. ^ van der Vaart 1998, p. 62
  3. ^ a b Reiersøl 1950
  4. ^ Casella & Berger 2001, p. 583
  5. ^ Raue, A; Kreutz, C; Maiwald, T; Bachmann, J; Schilling, M; Klingmüller, U; Timmer, J (2009), "Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood", Bioinformatics 25 (15): 1923–9,  

References

This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
 
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
 
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.
 


Copyright © World Library Foundation. All rights reserved. eBooks from Project Gutenberg are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.