Zurek's improvement of the definition of entropy

"A potentially important application of algorithmic complexity to physics was proposed by Wojtek Żurek of the Los Alamos National Laboratory in New Mexico. In order to rid Boltzmann's definition of entropy of its troublesome element of subjectivity, Żurek suggested an almost imperceptible modification of it. Recall that entropy is a measure of missing information about a system. It therefore depends on what an observer happens to know: a smarter being has more information, is missing less, and thus assigns a lower entropy to a system than a more limited creature. To render entropy more objective, Żurek recommended adding a measure of recorded information to that of missing information. The sum of the two remains constant --if you remove data from one column, it reappears in the other. The observant creature thus becomes redundant; only the entries in its notebook or computer memory matter.

But how to access the amount of recorded information? Żurek chose algorithmic complexity as the most natural measure. Accordingly, his new, improved entropy consists of two portions: the conventional entropy as measured by the formula on Boltzmann's tomb, plus a piece that is normally inconceivably tiny, and accounts for the algorithmic complexity of the listing of recorded knowledge about the system. A mathematical description of the size and shape of a vessel containing a gas might be a typical item in the list, while missing information includes the coordinates of a vast number of atoms. Notice that in the hypothetical case that every position and every velocity of every atom is known, the Boltzmann entropy of the system is zero, but the added term --the length of the description of what's known, in binary code-- will be huge, bringing the total entropy back to its previous value. After a hundred years the reek of subjectivity has finally been lifted from the Second Law of Thermodynamics.

In spite of its cogency, Żurek's improved entropy has not gained much support.

Hans Christian von Baeyer, Information, the new language of science, Chapter 12

Simple. It reminds me of the kinetic and potential energy in adiabatic mechanical system, in which the sum of the two stays constant at all times. I also remember vaguely that the history of the total energy followed the same trajectory than that of entropy; only one portion of the energy was first defined and only when the second portion was defined, the energy became a constant of the system and starting to be accepted with its components, as valid quantities.

Some comments on entropy

Definition

Since I reviewed the basic teachings concerning entropy, I see entropy everywhere. The most helpful definition of entropy is the one regarding the state of order. The highest the order, the lowest the entropy. Thus, I declare myself an enemy of entropy, in the sense that I, we, always try to create some order, to put things in order: putting the plates away in the kitchen, entangling an electric wire, etc. Why is it actually easier to put things in disorder and to create entropy than the reverse? One useful explanation to comprehend this difference is to see that a state in order is an improbable state, while a state in disorder is a probable one: the electric wire is more likely to be tangled after so many years than to stay untangled, and there are many ways to put a mess in a room, but only one to put things at their places.

Entropy and time

There is some controversy concerning the relationship between entropy and time. The problem is that the fundamental laws of classical physics (from Newton) are reversible in time; that is whatever happens in one direction (toward to the future) can very well happen in the other direction (toward the past). Thus, as we see a drop of milk in a tea cup spreading and diffusing throughout the volume, we should see all the milk particles to come back and form the initial drop of milk. This is indeed possible according to Poincaré's recurrence theorem, although, because the state of the drop is very unlikely compared to all the states where the milk is spread, the probability that this happens is tiny (but in theory, it could happen!).

On the other hand, the second law of thermodynamics says that for a closed system, the entropy has to increase: the spreading of the drop of milk within the cup is a perfect example of entropy increase. Some, such as Prigogine, argues that entropy carries with itself the so-called arrow of time: because entropy increases, we can make the difference between past and future. But the question then remained, is the second law compatible to the reversible laws of classical physics? Roger Penrose, in his book The emperor's new mind, argues that the second law is not only compatible, but also, contrary to Prigogine's view, that the entropy does not carry the arrow of time with it. Whatever the direction, toward the past or the future, the entropy has to increase within a closed system, in particular within a system where there is no constraint on the entropy. In the case of the drop of milk, although toward the future there is no constraint and the entropy increases indeed, toward the past, there is the constraint of the initial conditions saying that the entropy is low at the beginning: thus, if you run the experiment backwards, there is the constraint that at the end, the entropy is lower than at the beginning. Because of this constraint, the second law does not apply as such and in consequence, the entropy does not itself carry the arrow of time. The arrow of time exists because our system started with a state of low entropy. The remaining question is thus, why and how did we start with a state of low entropy?

Source of low entropy

Both L. Botlzmann and R. Penrose describe the struggle for life as a struggle for low entropy, with the ultimate source of low entropy being the sun. L. Botlzmann writes

"The general struggle for existence of animate beings is therefore not a struggle for raw materials [...] nor energy [...], but a struggle for entropy, which becomes available through the transition of energy from the hot sun to the cold earth."

and R. Penrose says

"We do not need to gain energy from our environment because energy is conserved. But we are continually fighting against the second law of thermodynamics. Entropy is not conserved; it is increasing all the time. To keep ourselves alive, we need to keep lowering the entropy that is within ourselves."

Thus, how do we get this low entropy? The ultimate source of entropy is the sun, and plants are the organisms which are using directly this entropy source, transforming it into molecular structures, themselves ready to be eaten. We, humans, via the food web, are eating plants or animals who themselves eat plants, to get low entropy for our body.

In this aspect, I am then wondering if we can class the food web in terms of entropy content. The plants would get a source of low entropy S_i, some of it would be used such that the entropy content gained in the eaten plant would be actually larger, S_i < S_plant . This process would repeat in that the higher in the food web, the higher the entropy content. In that respect, I would conclude that 1) humans would be organisms with some of the highest entropy (the most disorder) and 2) we should all be vegetarians in order to efficiently get low entropy in our diet. Do you agree with these conclusions?

Why the sun?

R. Penrose also explains why the sun is a source of low entropy and I was very surprised to learn that the reason is nearly a geometrical one. The sun is a hot spot, a small disk of light compared to the entire sky. Because of this geometrical configuration, the energy we receive from the sun has a much lower entropy that the energy sent back to space by earth because this energy is sent into all directions. Thus, if I understand correctly, if the earth was surrounded by many suns so much so that they would cover the entire sky, there would not be any source of low entropy and life would be unable to exist? Of course, one still needs to explain why the sun, itself a compact star and thus a source of low entropy, exists but the explanation goes on with cosmological arguments that I understand much less.

References
Ludwig Boltzmann, The second law of thermodynamics, in Theoretical physics and philosophical problems
Jean Bricmont, Science of chaos or chaos in science?
Roger Penrose, The emperor's new mind

Boltzmann on reason and passion

Read what have to say great scientists, historical figures or artists on contemporary problems, popular culture or on the little things in life can be either worrisome, such as the recent comment by the co-discoverer of DNA saying that black people are less intelligent, or charming such as L. Boltzmann, the great scientist from the late 19th century, commenting on our everyday struggle between reason and passion.

Far from defending a pure rational behavior, L. Boltzmann not only accepts the human weakness of reacting instinctively but also recognizes that at times instinct can very much enlight people's life. L. Boltzmann writes:

"How far removed we are from pure rational grounds being the motives of all our actions! The innermost impulses to action mostly still arise from innate drives and passions, that is from instincts germinating within us without our concurrence, which do indeed become harmful and reprehensible if dominating the intellect, but nevertheless are necessary to lend our actions liveliness and our character its peculiar colouring. The machinery of the world maintains itself, as Schiller says, «today, as ever, by hunger and love, and the time is as yet far off when philosophy will hold the universal circuit together»."

Ludwig Boltzmann, On the principles of mechanics, in Theoretical physics and philosophical problems

Such observation has been made over and over during History. For instance, during the Thirty Years War, Sweden chancellor expressed it as followed:

"Nesci, mi fili, quantilla ratione mundus regatur"
"you don't know, my dear boy, with what little reason the world is governed"

cited by Ludwig von Bertalanffy in General system theory.

The importance of small steps in science (and in life in general)

One day, when finishing up my dinner in a Chinese restaurant, I cracked open my fortuneteller cookie and what I read stroke me and reminded me a long-time forgotten lesson that my mother used to teach my sisters and me. My mother's saying was like "the little rivers make the great ones". The Chinese one was something like "great things are achieved by small steps". I was quite amazed how I failed to remember this lesson which sounds so modest and powerful in the same time. Life around you goes so fast that at times it pushes you to burn the essential steps; but then, you burn yourself before reaching your goal.

Thus, it is with pleasure that I would like to refresh our Confucius-like philosophy today and reminded us of the importance of such simple lesson. What motivated me is this lesson appeared also in one of Ludwig Boltzmann's essay, a very important physicist of the late 19th century. This concerns science only but it applies, as he shows, to everything in life:

"Nowhere less than in natural science does the proposition that the straight path is the shortest turn out to be true. If a general intends to conquer a hostile city, he will not consult his map for the shortest road leading there; rather he will be forced to make the most various detours, every hamlet, even if quite off the path, will become a valuable point of leverage for him, if only he can take it; impregnable places he will isolate. Likewise, the scientist asks not what are the currently most important questions, but «which are at present solvable?» or sometimes merely «in which can we make some small but genuine advance?». As long as the alchemist merely sought the philosopher's stone and aimed at finding the art of making gold, all their endeavours were fruitless; it was only when people restricted themselves to seemingly less valuable quesitons that they created chemistry. Thus natural science appears completely to lose from sight the large and general questions; but all the more splendide is the success when, groping in the thicket of special questions, we suddenly find a small opening that allows a hitherto undreamt of outlook on the whole"

Ludwig Boltzmann, The second law of thermodynamics, in Theoretical physics and philosophical problems