The Physics of Randomness

Entropy is a fundamental law, but why does entropy even exist?

If we open this valve right now, without thinking, you know that these particles will distribute themselves equally between the two chambers. I think everyone knows that.

However, the “why” is interesting, as it is both simple and complex at the same time.

Suppose you want to toss a coin once, the probability of it being a head is 1/2 = 0.5.

So, out of the two possibilities (heads or tails), you can confidently say that the outcome is random. If you toss it one hundred times, the probability of getting all heads is about 10-30; in other words, it’s nearly impossible. One can also show that the probability of getting half the number of tosses as heads and the other half as tails after tossing the coin one million times is nearly 1(0.999999). In other words, in this case, it is simply impossible to not get it. This can be summarized by one beautiful sentence: The summation of many random events can give a seemingly nonrandom one.

Now, consider a closed flask that has one billion atoms and is connected to a closed and empty flask through a valve. Initially, the atoms are all gathered in one flask. When we open the valve and give it enough time, we’ll see that there’s approximately an equal number of atoms in both flasks. And this brings us back to the initial question: Why?

To answer this question, one must introduce the notion of “multiplicity”. Multiplicity, in simple terms, is the number of ways you can arrange the states that you have amongst the available platforms. For example, you have two baskets (platforms) and four apples (states), and the apples are labeled 1, 2, 3, 4. You can place three apples in one basket, say 1, 3, 4 (order doesn’t matter), and one apple in the other, or you can place all four apples in one basket and no apples in the other. However, the arrangement that yields the highest multiplicity (as in the number of ways you could distribute the apples amongst the baskets) is if you place two apples in one basket and two in the other. If you place apples 1, 2, or 1, 3, or 1, 4, or 2, 3, or 2, 4, or 3, 4 in one basket, you’ll get six total ways to arrange the apples in that basket.

If you place three apples in one basket and one apple in the other, you’ll get four total ways to arrange the apples in that basket. The equal distribution of apples amongst the baskets gives us more ways to arrange these apples.

Using this concept in the flask of two chambers, and considering only 4 particles in the first chamber. According to the above case, the 4 molecules will most probably separate equally between the two chambers since this has the maximum multiplicity. But why will the system move towards the state with the highest multiplicity?

The key behind the answer to this question is a notion we previously explained: Randomness. The molecules move completely randomly inside the system and thus, there’s no reason why they would prefer a chamber over the other. So, every molecule has a 50% chance to be in the left chamber and a 50% chance to be in the right one.

This is the essence of the law of maximum multiplicity. The below are the 1 by 3 (meaning 1 particle in first chamber and 3 in the next) and 3 by 1 where each color is a distinct particle.

Wait, won’t the state with 3 in one chamber and 1 in the other have more possibilities? It doesn’t of course. We can’t combine 3 by 1 and 1 by 3 configurations when giving multiplicity since the two are distinguishable. The numbers will look as follow:

Now the notion of randomness tells us that no single possibility is favored over the other, meaning that they all acquire the same weight of occurrence. So, the state of maximum number of possibilities (multiplicity) will naturally occur. If this is not clear, you will certainly imagine it in this next physical model.

According to the above numbers, there are 16 possible arrangements of the system (6 + 8 + 2). Let’s say that we opened the valve at time t = 0 minutes and left it for 60 minutes. If we have 16 possibilities with equal probabilities of existing (by the notion of randomness), each state will acquire an average of 60/16 = 3.75 minutes. So, the system of a 2 by 2 distribution will exist for 3.75 × 6 = 22.5 minutes. On the other hand, the systems of 3 by 1 and 1 by 3 distributions will exist for 3.75 × 4 = 15 minutes each. And finally, the systems of a 4 by 0 and 0 by 4 distributions will exist for 3.75 × 1 = 3.75 minutes. You can see that the system will spend most of its time in the state of equal distribution (which has the most multiplicity).

However, if our system was in the order of 10^23 molecules, we can confidently say that the whole 60 minutes would correspond to an equal distribution.

I haven’t mentioned entropy yet, but as you’ve probably noticed, this article focuses on multiplicity. This is to show you that the real meaning behind entropy lies in multiplicity. Entropy is just proportional to the natural logarithm of multiplicity given by the following equation:

Entropy = Boltzmann constant × ln(multiplicity)

Of course, this applies to every possible physical event, but with one condition. The system should be isolated, meaning no energy should enter or leave it. This is because, if the system is supplied with energy, we can in fact break the law of maximum multiplicity, but this is for another post.

At first glance, “randomness” and “disorder” might seem like oversimplified terms to describe entropy. But you can look at it in this way:

A disordered state refers to the high number of ways you can randomly arrange the system which has high entropy. That way: Entropy gives us information about the system.

Thursday is were we’ll take a weird turn. All this is really cool, but it looks oddly similar to a biological phenomena which is at least as fundamental… Darwinian natural selection! It’s the Biology of randomness in some sense, and at the same time it’s an algorithm written by nature forces by logic. The full version of Thursday as always is for paid subs, consider subscribing to the paid tier if you’re enjoying and you want to go further in understanding the universe.

Have a nice weekend!

PS: Reply back with any question you might have, I enjoy them a lot and I respond to every one!