The Puzzle of Entropy

I’ve been chiseling away at Cycles of Time (2010), by Roger Penrose. I say “chiseling away,” because Penrose’s books are dense and not for the fainthearted. It took me three years to fully absorb his The Emperor’s New Mind (1986). Penrose isn’t afraid to throw tensors or Weyl curvatures at readers.

This is a library book, so I’m a little time constrained. I won’t get into Penrose’s main thesis, something he calls conformal cyclic cosmology (CCC). As the name suggests, it’s a theory about a repeating universe.

What caught my attention was his exploration of entropy and the perception our universe must have started with extremely low entropy.

As it turns out, this creates a bit of a puzzle, and there’s an aspect to it I’d never considered before. The puzzle comes from our understanding that, because entropy always increases, it must have been lower in the past.

We have similar logic regarding the big bang: The universe is expanding now, and appears to have been expanding in the past. Therefore it must have been smaller long ago.

We see that entropy always increases — so consistently so that we recognized it as law. Therefore it must have been less long ago.

But there is a conundrum posed by the cosmic microwave background (CMB). It is seen as having extremely high entropy in virtue of its smooth distribution. That high-entropy distribution is a key argument in favor of cosmic inflation (for which CCC provides an alternate explanation).

What’s more, the primal big bang fireball, “infinite” in density and energy, is very much what we think of as a high entropy situation. (It’s been something that’s long bugged me. How is the big bang low entropy?)

Penrose’s CCC is an attempt to resolve the conundrum. He posits a very distant future when all that remains are low-energy photons and gravity waves. The universe would be timeless (photons, unlike particles with mass, have no clock). This apparently has a conformal map to something that looks like a big bang in a rebooted universe.

Further, the highly accelerated cosmic expansion (due to dark energy) in the distant future of the previous universe (and eventually ours) is what looks like cosmic inflation in this universe (and the next).

It’s similar to big bounce theories in being cyclic, but there’s no big crunch, just a continual expansion until nothingness and conformal map (which I totally don’t understand) to a new big bang.

§ §

Whatever. Speculative metaphysics is interesting, but it’s ultimately a form of diamond-hard science fiction (which, admittedly, I love).

I can say that I haven’t found anything particularly objectionable about the idea (such as I have with the BUH, the MWI, or the MUH). All cosmologies require basic axioms, things that just are, and I find a physical, single, evolving, even repeating, universe more plausible than those three.

Penrose is clear about how speculative this is, but he does provide some avenues that might falsify or support his theory. He’s pursued some of them with what he considers interesting results.

It would take me a lot more readings, and probably learning more about tensors, to say much more about his conformal cyclic cosmology. Instead, since the part of the book that really grabbed me was about entropy, I thought I’d write about that.

§

If you search my blog for “entropy” you’ll get quite a few hits. I’ve mentioned it now and then, because it’s so fundamental to everything.

If you click the entropy tag, you get four posts that discuss it in detail (not including this one). The earliest of those is one of the first posts I published here: Barrel of Wine; Barrel of Sewage.

It gives, what I still think is, a pretty good informal overview of entropy. You should read it if you haven’t, since I’ll refer to the CD example. Today I’m going to give you a formal definition.

§ §

Entropy defines differently depending on the domain. In communications (information theory), it focuses on errors in data. In computer science, it can refer to randomness for cryptography.

Those are specialized views. The most general English-based definition is that entropy is a measure of disorder. This definition is so general it drives most physicists a little crazy (because: define “disorder”… and “measure”).

A more precise English-based definition is more opaque: Entropy is the number of indistinguishable micro-states consistent with the observed macro-states. Clear as a brick wall, yeah?

I’m going to try to knock down that wall. If I’m successful, you’ll have a basic understanding of the formal mathematical definition of entropy:

Which, by the way, is what is sometimes known as Boltzmann entropy to distinguish it from other versions. For now just know that S is the entropy (and B is for Boltzmann).

§

Let’s go back to that second definition: Entropy is the number of indistinguishable micro-states consistent with the observed macro-states.

A macro-state is a property we can measure in a system. A macro-state is an overall property of the whole system, such as its temperature, pressure, or color. In the example of the CD collection, how well it’s sorted is a property of the whole collection.

A micro-state is a property of a piece of the system — usually, as the name implies, a small, even tiny, piece. The molecules of a gas or liquid all have individual properties, such as location and energy. Each CD in the collection has individual properties, such as its index in the set. (There’s a first, second, third, and so on.)

We say that micro-states are indistinguishable when different arrangements of micro-states have no effect on a given macro-state.

Consider air molecules spread-out evenly throughout a closed room. The air has a pressure that’s the same for an unimaginable number of different arrangements of those spread-out molecules. So long as the molecules are evenly distributed, the pressure is the same.

So vast numbers of molecule arrangements are indistinguishable in terms of the air pressure. That’s high entropy.

Now suppose a magic spell suddenly compresses all the air in the room into a tiny one-inch cube in the corner. There are still many ways to arrange the molecules in the tiny cube, but far fewer than in the whole room. What’s more, if the room is empty, then even one molecule outside the cube is distinguishable.

The result is a much smaller set of indistinguishable micro-states and thus low entropy.

§ §

In an average room, the number of air molecules is roughly 10²⁵. The number of ways that many molecules can be arranged in a one-inch cube is a number beyond comprehension. It’s basically the factorial of that 25-digit number of particles!

Since these numbers are beyond understanding, let’s take it way (way) down. Let’s assume there are only 125 air molecules (they are very large) and they fill the one-inch cube. We’re saying there are five molecules per inch, and the cube fits 5×5×5=125 molecules. Think of it as a little box of ball bearings.

It has 125! (factorial) = 1.88×10²⁰⁹ possible arrangements.

Suppose we expand the cube to two inches on each side. This increases the volume by eight (two cubed). Now there are 10×10×10=1000 locations for 125 molecules and 875 empty spaces.

This has 1000! = 4.02×10²⁵⁶⁷ possible arrangements.

Quite a bit more. Going from a one-inch cube of 125 molecules to a two-inch cube increases the number of micro-states by over two-thousand orders of magnitude.

My calculator isn’t capable of doing 3375!, which is the number of possible arrangements for a three-inch cube. If the exponent increases similarly, it’s over a 30-thousand digit number.

This is with only 125 molecules in a three-inch cube. Imagine how huge the numbers get with realistic numbers of air molecules and the full size of the room.

(The problem is we can’t imagine numbers like these. As we’ll see below, we’ll get a slight assist in dealing with such huge numbers.)

§

This vast increase in possible arrangements is why entropy always increases.

Given the freedom to roam the entire room, the odds of all the molecules finding one of the low-entropy arrangements, while possible, has a probability that is effectively zero. Generally speaking, the odds of finding any lower entropy configuration are extremely prohibitive.

Here’s the entropy equation again:

Now we can make sense of the other parts.

The K_B is a constant, a fixed number required to make the equation work.

(Mathematicians often use K for “konstant” and the B refers to Boltzmann, who came up with this formula. The value is known as Boltzmann’s constant.)

The Omega (Ω) is the number of micro-states of the system in question. As just discussed, it’s generally an extremely large number. (Remember that it’s the number of configurations of particles, not the number of particles.)

Which is why the log operator is there. Vastly over-simplifying, the log of a number is how many zeros it has. (For instance, the log of 1000 is exactly 3. The log of 1 is exactly 0.)

Using the log of the micro-states number makes comparing the entropy of systems more reasonable. Instead of the unimaginably large numbers involved, we can compare their logs.

In the 125-molecule example, the one-inch cube had Ω = 10²⁰⁹, so its log is 209. The two-inch cube had Ω = 10²⁵⁶⁷, so its log is 2567. Therefore the entropy difference is 2567 – 209 = 2358.

If the three-inch cube does have Ω = 10³⁰⁰⁰⁰⁺, then its entropy would be 30,000+. Even in our toy model, the entropy numbers get large. In a realistic model, the entropy numbers get astronomical (the configuration space numbers, as I said, are beyond comprehension).

(Note that I’ve seriously papered over certain aspects of logs.)

§ §

Entropy, and the second law of thermodynamics, work because of the behavior of micro-states plus time.

That entropy always increases is a statistical law, but a powerful one. It applies most strongly to large systems with lots of tiny pieces, but it’s a lurking presence in all systems.

When I pick this up again, it’ll be to explain (according to Penrose) the puzzle of the necessity of a low-entropy past versus the apparent high-entropy reality of the big bang.

Stay entropic, my friends!

∇