Boltzmann, ergodic theory, and chaos by Donald S. Ornstein

100



D. S. Ornstein



1. A Newtonian system. The ball moves at constant speed and bounces off the sides

and the obstacles, angle of incidence equaling the angle of reflection.



Figure 1



The phase space, M , is the unit tangent bundle to the table (position and direction)

and the invariant measure is the three-dimension Lebesque measure. (We will denote

x x N x

this model by .F t ; X ; u; P /.)

2. A random model

Markov flow. For each N we define a Markov flow. Start with N points, Npi , on the

table. Our process stays at Npi for time N ti and then flips a fair coin to decide which

of a pair of points (determined by Npi ) to jump to.

O O O O

We will denote this model by .F t ; X t ; u;N P /.

O

Theorem. There is one B t , X , u, P ,1 and a sequence NP , such that

x x N x

.B t ; X; u; P / D .F t ; X ; u; P / D Sinai billiards,

O O O O

.B t ; X; u; NP / D .F t ; X t ; u; NP / D Markov flow,

NP



! P in the L1 norm.



NP can be defined by a partition of X (or M ) into a finite number of atoms and a

point, Npi , in each atom. We can think of NP as a course graining of M . We get the

model for our Markov flow by viewing billiards through this course graining.

We would like to be able to use any sequence of course graining, but this is clearly

impossible. However, we can use any sequence of course grainings with an arbitrary

small modification. That is, we can start with any sequence of partitions, NQ that,

1



F t stands for a generic flow, whereas B t stands for a specific F t .



Boltzmann, ergodic theory, and chaos



101



in the limit, separate points. (We could assume that the sup of the diameters, d , the

! 0.) We can choose any "N ! 0 and insist that jNQi Npi j < "N.

Our model for billiards does not take into account our inability to make measurements of infinite accuracy. By adding a sequence of course grainings, we get a model

where we can see the ball with arbitrarily good, but finite, accuracy. In this model, the

distinction between a system evolving according to Newton’s laws and one evolving

by coin flipping disappears.

Another justification for modifying our Newtonian model by course graining is that

Boltzmann’s introduction of u does not completely explain the arrow of time. We need

to take into account our ability to start our system in a set, E, of very small probability

(all the gas in one half of the box) and our inability to start it in F t .E/ (because this set

is spread out uniformly in the phase space).

These special starting sets correspond to some sort of course graining.

Another justification comes from chaos “theory”. This “theory” is a collection of

results that center on the observation that some Newtonian systems, even very simple

ones, “appear” to be random.

The usual version is that some Newtonian systems are sensitive to initial conditions

and, because we can’t observe the initial conditions accurately (we only see the system

through a course graining), the system is very unpredictable (“appears” to be random).

Our result says that, not only does Sinai billiards “appear” random, but, in a certain

sense, it is random. When we look at billiards through a course graining, we see exactly

a Markov flow.



NQi



Part B

Our discussion seems to have wandered away from Boltzmann’s legacy, but, in fact, it is

based on Boltzmann’s entropy, which was adapted by Shannon for stationary processes

and, then, introduced into ergodic theory by Kolmogorov.

We will start our discussion with discrete time.

We call a measure preserving transformation on an abstract measure space.F; X; u/

a Bernoulli shift (B-shift) if, for some P , .F; X; u; P / is the model for an independent

process.

One of the central, and oldest, questions in ergodic theory was: are all B-shifts the

same? Kolmogorov answered this question by introducing entropy as an invariant for

measure preserving transformations, and showed that if .F; X; u; P / is an independent

P

process with probabilities pi , then the entropy of .F; X; u/ is i pi log pi (thus, not

all B-shifts are the same).

The next step was proving the converse: i.e., B-shifts with the same entropy are the

same.

The Kolmogorov–Sinai introduction of entropy also organized the .F; X; u/ into

different levels of randomness.

At the bottom, there are the .F; X; u/ of zero entropy. This is equivalent to: for

any finite-valued P , the sample path from 0 to 1 determines the output at all future



102



D. S. Ornstein



times. We think of these .F; X; u/ as not really random.

On the other end, we have the Kolmogorov transformations (K-automorphisms)

where there is no finite-valued P such that F , X , u, P has zero entropy (is predictable).

The K-automorphisms include the B-shifts.

It was once believed that the only K-automorphisms were the B-shifts, but this is

not true and the K-automorphisms are a large class that includes uncountably many of

the same entropy; some have no square root; some are not the same as their inverse, etc.

It was also believed (the Pinsker conjecture) that every transformation was the direct

product of a zero entropy transformation and a K-automorphism. This is, again, false.

Returning to continuous time, we have the following isomorphism theorem, whose

proof depends heavily on entropy.

(1) Every B-shift can be embedded in a flow (there is a flow, B t , such that, for some

t0 , the discrete time transformation, B t0 , is the B-shift.

(2) There are only two such flows, one of finite and one of infinite entropy. (The

finite entropy flow is unique, up to a constant rescaling of time, i.e., change the unit of

time). We denote these by B t and B t1 .

(3) All of the discrete time transformations, B t0 (and B t1 ) are B-shifts.

0

(4) All of the B-shifts are realized by fixing some t0 in B t or B t1 (so they are really

not so different after all).

(5) The only factors of B t are B t (possibly rescaled). (The factors of B t1 are B t

or B t1 .)

(6) B t (or B t1 ) is a full entropy factor of any F t of non-zero entropy.

(5) and (6) need a little explaining. For .F t ; X; u; P / it could happen that more than

one x in X gives the same sample path P .F t .x//. This is the model for a stationary

process only after we have lumped these x together. This lumping gives an invariant

sub- -algebra and F t restricted to an invariant sub- -algebra is called a factor.

An important implication of the isomorphism theorem is that, at the level of abstraction of measure preserving flows on abstract measure spaces .F t ; X; u/ there is a

unique flow that is the most random possible (really two flows, B t and B t1 .)

This is justified by the feeling that independent processes are the most random.

Furthermore, if we think of P as a measurement on .F t ; X; u/, then (6) implies that if

F t has non-zero entropy, then any measurement on F t will also occur as a measurement

on B t (or B t1 ). Furthermore, if F is not B t (or B t1 ), then it will have measurements

that do not appear as measurements on B t (or B t1 ).

Furthermore, if .F t , X , u, P /, is a stationary process, where F t is not B t (or B t1 ),

N

then this process cannot be approximated arbitrarily well (even in the d series) by a

Markov flow.

Part of the isomorphism theorem that we did not describe are criteria for proving a

system to be Bernoulli.

Applying one of these criteria, and using some hard results of Sinai, it was proved

in that Sinai billiards are modeled by B t .

Another criterion shows that the Markov flows are modeled by B t (in particular, the

Markov flow where we flip back and forth between just two points.)



Boltzmann, ergodic theory, and chaos



103



There is a long list of systems that are modeled by B t or B t1 , but we will give just

one more example2 . Any SRB measure is modeled by B t .

We built our discussion around Sinai billiards, but we could have used any flow on

a Riemannian manifold that preserves a smooth measure and which can be proved to

be Bernoulli (modeled by .B t ; X; u; P //.



References

[1] C. E. Shannon, A mathematical theory of communication, Bell System Tech. J. 27 (July

1948), 379–423.

[2] —, A mathematical theory of communication, Bell System Tech. J. 27 (October 1948),

623–656.

[3] A. N. Kolmogorov, A new metric invariant of transitive dynamical systems and Lebesgue

space automorphisms, Dokl. Akad. Nauk SSR 119 (5) (1958), 861–864 (in Russian).

[4] —, Entropy per unit time as a metric invariant of automorphisms, Dokl. Akad. Nauk SSR

124 (1959), 754–755 (in Russian).

[5] R. Adler and B. Weiss. Entropy a complete invariant for automorphisms of the tours, Proc.

National Acad. Sci. USA 57 (1967), 1573–1576.

[6] Ya. G. Sinai, On the notion of entropy of a dynamical system, Dokl. Akad. Nauk SSR 124

(1959), 768–771.

[7] —, A weak isomorphism of transformations with invariant measure, Dokl. Akad. Nauk SSR

147 (1962), 797–800.

[8] — (ed.), Dynamical Systems II, Encyclopedia Math. Sci., Volume 2, Springer-Verlag, Berlin

1988.

[9] G. Gallavotti and D. S. Ornstein. Billiards and Bernoulli schemes, Comm. Mathem. Phys.

38 (1974), 83–101.

[10] —, The Billiard Flow with a Convex Scatterer is Bernoullian, to appear.

[11] D. S. Ornstein, Bernoulli shifts with the same entropy are isomorphic, Adv. Math. 4 (1970),

337–352.

[12] —, Two Bernoulli shifts with infinite entropy are isomorphic, Adv. Math. 5 (1970), 339–348.

[13] —, Factors of Bernoulli shifts are Bernoulli shifts, Adv. Math. 5 (1970), 349–364.

[14] —, Imbedding Bernoulli shifts in flows, Contributions to Ergodic Theory & Probability.

Springer-Verlag, Berlin 1970 178–218.

[15] —, The isomorphism theorem for Bernoulli flows, Adv. Math. 10 (1973), 124–142.

[16] —, An example of Kolmogorov automorphism that is not a Bernoulli shift, Adv. Math. 10

(1973), 49–62.

[17] —, A K-automorphism with no square root and Pinsker’s conjecture, Adv. Math. 10 (1973),

89–102.

[18] —, A mixing transformation for which Pinsker’s conjecture fails, Adv. Math. 10 (1973),

103–123.

2



Because I think it the most relevant to the kinds of things that might have interested Boltzmann.



104



D. S. Ornstein



[19] D. S. Ornstein and P. C. Shields. An uncountable family of K-automorphisms, Adv. Math.

10 (1973), 63–88.

[20] D. S. Ornstein and B. Weiss. Geodesic flows are Bernoullian, Israel J. Math. 14 (1973),

184–198.

[21] —, Statistical properties of chaotic systems, Bull. Amer. Math. Soc. 24 (1991), 11–116.

[22] M. Ratner, Anosov flows with Gibbs measures are also Bernoullian, Israel J. Math. 17

(1974), 380–391.

[23] M. Keane and M. Smorodinsky. Bernoulli shifts on the same entropy are finitarily isomorphic, Ann. of Math. 109 (1979), 397–406.

[24] F. Ledrappier, Propriétés ergodiques des mesures de Sinai, Inst. Hautes Études Sci. Publ.

Math. 59 (1984), 163–188.

[25] Mañe, Ergodic theory and differentiable dynamics, Ergeb. Math. Grenzgeb. 8, SpringerVerlag, Berlin 1987.



The Boltzmann family in Graz 1886

Children (left to right): Henriette, Ida Katherina, Ludwig Hugo, Arthur

(Courtesy of Ilse M. Fasol-Boltzmann and Gerhard Ludwig Fasol)



Ludwig Boltzmann in Graz 1887

Standing (left to right): Walther Nernst, Franz Streinzt, Svante Arhennius,

Richard Hiecke; sitting (from left): Eduard Aulinger, Albert von Ettingshausen,

Ludwig Boltzmann, Ignaz Klemencic, Viktor Hausmanninger

(Courtesy of the Österreichische Zentralbibliothek für Physik)



134 years of Boltzmann equation

Carlo Cercignani



1 The first 100 years

In 1872 [1] Boltzmann obtained the equation that bears his name and proved what later

was called the H -theorem. Before writing his great paper, Boltzmann had learned to

master Maxwell’s techniques [2]. In fact, already in 1868 he had extended Maxwell’s

distribution to the case when the molecules are in equilibrium in a force field with potential [3], including the case of polyatomic molecules as well [4]. The energy equipartition

theorem was also extended by him to the case of polyatomic molecules [5]. Boltzmann

interprets the distribution function in two ways, which he seems to consider as a priori

equivalent: the first one is to think of it as the fraction of a time interval sufficiently long,

during which the velocity of a specific molecule has values within a certain volume

element in velocity space, whereas the second (quoted in a footnote to paper [3]) is

based on the fraction of molecules, which, at a given time instant, have a velocity in

the said volume element. It seems clear, as remarked by Klein [6], that Boltzmann did

not feel, at that time, any need to analyse the equivalence, implicitly assumed, between

these two meanings, which are so different. He soon realized, however, (in a footnote

to paper [5]) that it was necessary to make an assumption, “not improbable” for real

bodies made of molecules that are moving because they possess “the motion that we

call heat”. This assumption, according to which the coordinates and the velocities of the

molecules take on, in an equilibrium state, all values compatible with the assigned total

energy of the gas, became later familiar with the name of ergodic hypothesis, given to

it by Paul and Tatiana Ehrenfest [7] .

Before discussing the basic paper of 1872 and the subsequent evolution of our

understanding of the Boltzmann equation, we remark that in 1871 Boltzmann felt ready

for a new attempt to understand the Second Law [8], starting from the equilibrium law

that he had obtained in his previous papers and illustrating the difference between heat

and work. He equated, as he had done previously and was common after the work of

Clausius and Maxwell, something, that he denoted by T and called temperature, to

the average of kinetic energy per atom (hence without using the so-called Boltzmann

constant, which was introduced by Planck much later). We stress the fact that this

identification, apart from a factor, is easily justified only if there is a proportionality

between thermal energy and temperature; this is the case for perfect gases and solids

at room temperature. The concept of temperature is indeed rather subtle, because it

does not have a direct dynamic meaning. In a more modern perspective, the concept

of entropy, introduced by Boltzmann in kinetic theory (together with thermal energy)

appears more basic (though admittedly less intuitive) and temperature appears as a

restricted concept, strictly meaningful for equilibrium states only.

The problems we have alluded to do not enter in the case considered by Boltzmann

and we can accept his identification without objections. It is then clear that the total



108



C. Cercignani



energy E, sum of the kinetic and potential energies, will have the following average

value:

hEi D N T C h i

(1)

where is the potential energy and hqi denotes the average of a quantity q whereas N

is the total number of molecules. It is then clear that one can change the value of hEi

in two ways, i.e. by changing either the temperature or the potential, so slowly as to go

through equilibrium states, to obtain

ıhEi D N ıT C ıh i



(2)



where ı (rather than the more typical d ) denotes an infinitesimal change. If we denote

the heat supplied to the system in the process by ı Q and compute it as the difference

between the increase of average total energy and the average work done on the system,

we have:

ı Q D ıhEi hı i D N ıT C ıh i hı i:

(3)

We remark that ıh i (the change of the average value of ) and hı i (the average of

the change of the value of ) are different because the averages will depend on certain

macroscopic parameters, typically temperature, which are allowed to change in the

process under consideration.

The expression in equation (3) is not the differential of some state function Q, a

circumstance underlined here by the presence of a star superscript affecting the symbol ı. Boltzmann showed, however, that, if we divide the expression under consideration

by T , one obtains the exact differential of a function, which he, of course, identified

with entropy. He also proceeded to computing it explicitly for a perfect gas and for

a simple model of a solid body, thus finding, in the first case, a result well known in

thermodynamics, in the second an expression, from which he easily succeeded in obtaining the Dulong–Petit formula, empirically known for specific heats of most solids

near room temperature.

Even if somebody acquainted with the usual thermodynamic calculations may find

it a bit strange that the work performed on the system is due to the change in the potential

rather than to the motion of a piston, the derivation by Boltzmann is impeccable, if one

grants that the equilibrium distribution is what is called nowadays Maxwell–Boltzmann,

and is now more or less standard. It was, however, a treatment that excluded irreversible

phenomena and it could not have been otherwise since the said distribution holds only

for equilibrium states.

But Boltzmann was by then ready for the last step, i.e. the extension of the statistical

treatment to irreversible phenomena, on the basis of a new integrodifferential equation,

which bears his name. As soon as he was sure of this result, he wanted to publish a short

paper on Poggendorff’s Annalen in order to ensure his priority in this discovery and to

subsequently elaborate the results in a complete form for the Academy of Vienna. Since

Stefan was against publishing twice the same material, we are left with just the memoir

of almost 100 pages presented to the Academy [1]. This may explain the strange title,

“Further researches on the thermal equilibrium of gas molecules”, chosen to present a

wealth of new results.



134 years of Boltzmann equation



109



The paper started with a critique to the derivation of velocity distribution in a gas

in an equilibrium state, given by Maxwell [2], with an emphasis on the fact that the

said deduction had only shown that the Maxwellian distribution, once achieved, is not

altered by collisions. However, said Boltzmann, “it has still not yet been proved that,

whatever the initial state of the gas may be, it must always approach the limit found

by Maxwell” [1]. When writing this statement Boltzmann had obviously in mind the

spatially homogeneous case, to which the first part of the memoir is actually devoted.

On the basis of an “exact treatment of the collision processes”, he obtained an

equation for the distribution function, usually denoted by f , i.e. the probability density

of finding a molecule at a certain position x with a certain velocity ξ at some time

instant t .

In the first part of the memoir he restricted himself to the case when f depends

just on time and kinetic energy. This equation may appear a bit strange to the eyes of

those who have in mind the version of the same equation which can be found in more

recent treatments, not only because of the use of the letter x to denote kinetic energy.

In fact the circumstance that he adopts this variable as an independent variable instead

of velocity introduces several square roots in the equation; these are due to the fact that

the volume element whose measure does not change with time during the evolution

of the system, thanks to Liouville’s theorem, contains the volume element in velocity

space d 1 d 2 d 3 . When transforming the variables in polar coordinates one obtains,

in addition to the solid angle element, the element jξj2 d jξj or, in terms of the kinetic

energy Ekin and apart from constant factors, .Ekin /1=2 dEkin .

By means of his equation, Boltzmann showed not only that the Maxwell distribution

is a steady solution of the equation, but that no other such solution can be found.

This goal is achieved by introducing a quantity, that turns out to be, apart from a

constant factor, the opposite of the entropy; the possibility of expressing the entropy

in terms of the distribution function, though in a certain sense not unexpected, does

not cease to stand as a remarkable fact, that must have produced a deep impression on

Boltzmann’s contemporaries. In fact, as remarked by the author himself, it implied an

entirely different approach to the proof of the Second Law, that showed not only the

existence of an entropy function for the equilibrium states, but also permitted to study

its increase in irreversible processes.

The paper goes on with an alternative derivation based on a model with discrete

energies, in such a way that the integrodifferential equation for the distribution function

becomes a system of ordinary nonlinear differential equations. The use of discrete energies has always appeared “much clearer and intuitive” [1] to Boltzmann. This statement

may sound like a naïvety, but might also indicate a surprising intuition about the difficulties of a rigorous proof of the trend to equilibrium, which disappear if one has to deal

with a discrete, finite system of equations, since the unknown f is, at any time instant,

a finite set of numbers, instead of a function (we are dealing with a finite-dimensional

space, rather than with a function space); this simplification permits to make use of

a property, already known in Boltzmann’s days (the so-called Bolzano–Weierstrass

theorem) in order to deduce the trend under consideration without particularly refined

mathematical arguments. Many historians of science have underlined the circumstance



110



C. Cercignani



that these discrete models used by Boltzmann led Planck to the discovery of his energy

quanta, as Planck himself acknowledged [9].

Just a few pages of the voluminous memoir by Boltzmann concern the calculation

of the transport properties in a gas. It is in these pages, however, that Boltzmann laid

down his equation in the most familiar form for us, where the distribution function

depends upon time t, upon velocity ξ and upon position x (the vector notation is, of

course, anachronistic). His calculations show that the viscosity, heat conduction and

diffusion coefficients can be computed by means of his equation with results identical

with those of Maxwell, for the so called Maxwellian molecules. Boltzmann, however,

warned his readers against the illusion of an easy extension of his calculations to the

case of more complicated interaction laws.

In order to explain Boltzmann’s contributions in this exceptionally important paper,

we shall start from this last part. We shall use an approach anachronistic not only because

of the notation but also because of the argument we shall use. In fact, otherwise, it does

not appears to be possible to treat the subject in an understandable and short form.

We shall imagine, unless we say otherwise, the molecules as hard, elastic and perfectly smooth spheres. Not only this choice will simplify our presentation, but it is also

in a reasonable agreement with experience. Using more refined models would quantitatively improve this agreement, but would introduce several technical complications,

without changing anything from a conceptual standpoint.

In order to discuss the behavior of a system of N (identical) hard spheres, it is

very convenient to introduce the 6N -dimensional phase space, where the Cartesian

coordinates are the 3N components of the N position vectors of the sphere centers x i

and the 3N components of the N velocities ξi . Thus, if we have one molecule, we need

a six-dimensional space, if we have two molecules we need twelve dimensions, etc. In

this space, the state of the system, if known with absolute accuracy, is represented by a

point having given values of the aforementioned coordinates. If the state is not known

with absolute accuracy, we must introduce a probability density f .x; ξ; t / which gives

the distribution of probability in phase space. Given f0 , the value of f at t D 0, we

can compute f for t > 0, provided we have an equation giving its time evolution.

Assuming that body forces (such as gravity) are omitted (for the sake of simplicity),

Boltzmann wrote this equation in the form

df

DG

dt



L:



(4)



Here df =dt is the partial derivative with respect to time in the space homogeneous

case, but is, in general, the time derivative along the molecule’s trajectory:

df

@f

@f

D

Cξ

:

dt

@t

@x



(5)



Ld xd ξdt gives the expected number of particles with position between x and x C d x

and velocity between ξ and ξ Cd ξ which disappear from these ranges of values because

of a collision in the time interval between t and t Cdt and Gd xd ξdt gives the analogous

number of particles entering the same range in the same time interval. The count of