World Library  
Flag as Inappropriate
Email this Article

Kolmogorov-Sinai entropy

Article Id: WHEBN0002614219
Reproduction Date:

Title: Kolmogorov-Sinai entropy  
Author: World Heritage Encyclopedia
Language: English
Subject: Entropy (information theory), Predictability, Entropy (disambiguation), Subshift of finite type
Publisher: World Heritage Encyclopedia

Kolmogorov-Sinai entropy

In mathematics, a measure-preserving dynamical system is an object of study in the abstract formulation of dynamical systems, and ergodic theory in particular.


A measure-preserving dynamical system is defined as a probability space and a measure-preserving transformation on it. In more detail, it is a system

(X, \mathcal{B}, \mu, T)

with the following structure:

  • X is a set,
  • \mathcal B is a σ-algebra over X,
  • \mu:\mathcal{B}\rightarrow[0,1] is a probability measure, so that μ(X) = 1, and μ(∅) = 0,
  • T : XX is a measurable transformation which preserves the measure μ, i.e., each A\in \mathcal{B} satisfies μ(T−1A) = μ(A).

This definition can be generalized to the case in which T is not a single transformation that is iterated to give the dynamics of the system, but instead is a monoid (or even a group) of transformations Ts : XX parametrized by sZ (or R, or N ∪ {0}, or [0, +∞)), where each transformation Ts satisfies the same requirements as T above. In particular, the transformations obey the rules

  • T0 = idX : XX, the identity function on X;
  • T_{s} \circ T_{t} = T_{t + s}, whenever all the terms are well-defined;
  • T_{s}^{-1} = T_{-s}, whenever all the terms are well-defined.

The earlier, simpler case fits into this framework by definingTs = Ts for sN.

The existence of invariant measures for certain maps and Markov processes is established by the Krylov–Bogolyubov theorem.


Examples include:


The concept of a homomorphism and an isomorphism may be defined.

Consider two dynamical systems (X, \mathcal{A}, \mu, T) and (Y, \mathcal{B}, \nu, S). Then a mapping

\phi:X \to Y

is a homomorphism of dynamical systems if it satisfies the following three properties:

  1. The map φ is measurable,
  2. For each B \in \mathcal{B}, one has \mu (\phi^{-1}B) = \nu(B),
  3. For μ-almost all xX, one has φ(Tx) = Sx).

The system (Y, \mathcal{B}, \nu, S) is then called a factor of (X, \mathcal{A}, \mu, T).

The map φ is an isomorphism of dynamical systems if, in addition, there exists another mapping

\psi:Y \to X

that is also a homomorphism, which satisfies

  1. For μ-almost all xX, one has x = \psi(\phi x)
  2. For ν-almost all yY, one has y = \phi(\psi y).

Generic points

A point xX is called a generic point if the orbit of the point is distributed uniformly according to the measure.

Symbolic names and generators

Consider a dynamical system (X, \mathcal{B}, T, \mu), and let Q = {Q1, ..., Qk} be a partition of X into k measurable pair-wise disjoint pieces. Given a point xX, clearly x belongs to only one of the Qi. Similarly, the iterated point Tnx can belong to only one of the parts as well. The symbolic name of x, with regards to the partition Q, is the sequence of integers {an} such that

T^nx \in Q_{a_n}.\,

The set of symbolic names with respect to a partition is called the symbolic dynamics of the dynamical system. A partition Q is called a generator or generating partition if μ-almost every point x has a unique symbolic name.

Operations on partitions

Given a partition Q = {Q1, ..., Qk} and a dynamical system (X, \mathcal{B}, T, \mu), we define T-pullback of Q as

T^{-1}Q = \{T^{-1}Q_1,\ldots,T^{-1}Q_k\}.\,

Further, given two partitions Q = {Q1, ..., Qk} and R = {R1, ..., Rm}, we define their refinement as

Q \vee R = \{Q_i \cap R_j \mid i=1,\ldots,k,\ j=1,\ldots,m,\ \mu(Q_i \cap R_j) > 0 \}.\,

With these two constructs we may define refinement of an iterated pullback

\bigvee_{n=0}^N T^{-n}Q = \left \{Q_{i_0} \cap T^{-1}Q_{i_1} \cap \cdots \cap T^{-N}Q_{i_N} \text{ where } i_\ell = 1,\ldots,k ,\ \ell=0,\ldots,N,\ \mu \left (Q_{i_0} \cap T^{-1}Q_{i_1} \cap \cdots \cap T^{-N}Q_{i_N} \right )>0 \right \}

which plays crucial role in the construction of the measure-theoretic entropy of a dynamical system.

Measure-theoretic entropy

The entropy of a partition Q is defined as[1][2]

H(Q)=-\sum_{m=1}^k \mu (Q_m) \log \mu(Q_m).

The measure-theoretic entropy of a dynamical system (X, \mathcal{B}, T, \mu) with respect to a partition Q = {Q1, ..., Qk} is then defined as

h_\mu(T,Q) = \lim_{N \rightarrow \infty} \frac{1}{N} H\left(\bigvee_{n=0}^N T^{-n}Q\right).\,

Finally, the Kolmogorov–Sinai or metric or measure-theoretic entropy of a dynamical system (X, \mathcal{B},T,\mu) is defined as

h_\mu(T) = \sup_Q h_\mu(T,Q).\,

where the supremum is taken over all finite measurable partitions. A theorem of Yakov G. Sinai in 1959 shows that the supremum is actually obtained on partitions that are generators. Thus, for example, the entropy of the Bernoulli process is log 2, since almost every real number has a unique binary expansion. That is, one may partition the unit interval into the intervals [0, 1/2) and [1/2, 1]. Every real number x is either less than 1/2 or not; and likewise so is the fractional part of 2nx.

If the space X is compact and endowed with a topology, or is a metric space, then the topological entropy may also be defined.

See also


  • Michael S. Keane, "Ergodic theory and subshifts of finite type", (1991), appearing as Chapter 2 in Ergodic Theory, Symbolic Dynamics and Hyperbolic Spaces, Tim Bedford, Michael Keane and Caroline Series, Eds. Oxford University Press, Oxford (1991). ISBN 0-19-853390-X (Provides expository introduction, with exercises, and extensive references.)
  • Lai-Sang Young, "Entropy in Dynamical Systems" (ISBN 0-691-11338-6


  • T. Schürmann and I. Hoffmann, The entropy of strange billiards inside n-simplexes. J. Phys. A28, page 5033ff, 1995. PDF-Dokument
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.

Copyright © World Library Foundation. All rights reserved. eBooks from Project Gutenberg are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.