Zurek's improvement of the definition of entropy

"A potentially important application of algorithmic complexity to physics was proposed by Wojtek Żurek of the Los Alamos National Laboratory in New Mexico. In order to rid Boltzmann's definition of entropy of its troublesome element of subjectivity, Żurek suggested an almost imperceptible modification of it. Recall that entropy is a measure of missing information about a system. It therefore depends on what an observer happens to know: a smarter being has more information, is missing less, and thus assigns a lower entropy to a system than a more limited creature. To render entropy more objective, Żurek recommended adding a measure of recorded information to that of missing information. The sum of the two remains constant --if you remove data from one column, it reappears in the other. The observant creature thus becomes redundant; only the entries in its notebook or computer memory matter.

But how to access the amount of recorded information? Żurek chose algorithmic complexity as the most natural measure. Accordingly, his new, improved entropy consists of two portions: the conventional entropy as measured by the formula on Boltzmann's tomb, plus a piece that is normally inconceivably tiny, and accounts for the algorithmic complexity of the listing of recorded knowledge about the system. A mathematical description of the size and shape of a vessel containing a gas might be a typical item in the list, while missing information includes the coordinates of a vast number of atoms. Notice that in the hypothetical case that every position and every velocity of every atom is known, the Boltzmann entropy of the system is zero, but the added term --the length of the description of what's known, in binary code-- will be huge, bringing the total entropy back to its previous value. After a hundred years the reek of subjectivity has finally been lifted from the Second Law of Thermodynamics.

In spite of its cogency, Żurek's improved entropy has not gained much support.

Hans Christian von Baeyer, Information, the new language of science, Chapter 12

Simple. It reminds me of the kinetic and potential energy in adiabatic mechanical system, in which the sum of the two stays constant at all times. I also remember vaguely that the history of the total energy followed the same trajectory than that of entropy; only one portion of the energy was first defined and only when the second portion was defined, the energy became a constant of the system and starting to be accepted with its components, as valid quantities.

Has the Universe finite or infinite complexity?

"[L]et's now finally discuss whether the physical universe is like π=3.1415926... which only has a finite complexity, namely the size of the smallest program to generate π, or like Ω, which has unadulterared infinite complexity.
Well, if you believe in quantum physics, then Nature plays dice, and that generates complexity, an infinite amount of it, for example, as frozen accidents, mutations that are preserved in our DNA. So at this time most scientists would bet that the universe has infinite complexity, like Ω does. But then the world is incomprehensible, or at least a large part of it will always remain so, the accidental part, all those frozen accidents, the contingent part.
But some people still hope that the world has finite complexity like π it just looks like it has high complexity. If so, then we might eventually be able to comprehend everything, and there is an ultimate TOE [Theory of Everything]! But then you have to believe that quantum mechanics is wrong, as currently practiced, and that all quantum randomness is really only pseudo-randomness, like what you find in the digits of π. You have to believe that the world is actually deterministic, even though our current scientific theories say that it isn't!
[...]Wolfram believes that very simple deterministic algorithms ultimately account for all the apparent complexity we see around us, just like they do in π. He believes that the world looks very complicated, but is actually very simple. There's no randomness, there's only pseudo-randomness. Then nothing is contingent, everything is necessary, everything happens for a reason. [Leibniz!]
[...]
Or perhaps from inside this world we will never be able to tell the difference, only an outside observer could do that."

Gregory Chaitin, Metamath!, Appendix II.

Notice that the last argument has also been mentioned by Karl Popper in his Open Universe (see my review of his book).

Randomness cannot be defined

"Borel's conclusion is that there can be no one definitive definition of randomness. You can't define an all-inclusive notion of randomness. Randomness is a slippery concept, there's something paradoxical about it, it's hard to grasp. It's all a matter of deciding how much we want to demand. You have to decide on a cut-off, you have to say «enough,» let's take that to be random."

Gregory Chaitin, Meta Math!, Complexity, randomness and incompleteness.

"Biology Is Just a Dance" by Brian Goodwin

Here are some interesting extracts from an on-line article by Brian Goodwin published in Edge who past away recently:

"Will biology join up with physics, take on its flavor, have this notion of rules, organization, regularity, order? The new movement is transforming biology from a historical science, which is what it is at the moment, the objective of Darwinism being to reconstruct the history of life on Earth. Well, that's not the style of physics. Physics is about laws, the principles of organization of matter. We're doing the same thing in biology; we're looking for the principles of organization, the dynamics of the living process. Once that's understood, you're in a position to say, "Ah! History followed such and such a course in expressing and revealing the subtle order in this particular type of organization of matter we call the living state." Thus, the first thing is to understand the living state."

"The small-scale variation and the detailed adaptation of organisms to their habitats are very well explained by neo- Darwinism, but the global problem, the large-scale evolutionary problem, is unsolved. How do you get evolutionary novelty? Emergent order? The difference between squids and fishes and penguins. That's what the science of complexity is beginning to address — to demonstrate how emergent qualities can develop out of complexity, so that you get the emergence of order. The difficulty is making the theoretical work connect with the biological evidence. Most of the modeling currently done on computers is still very abstract, and there's not a lot of detailed evidence as to how that translates into what actually goes on in organisms."

So, what are the main tangible results so far?

You can read the entire article on the Edge website.

Humble models

David Orrell got his PhD from the University of Oxford on the modelling of nonlinear systems. Although he got it only in 2000, he earned some authority and describes us in his book, The future of everything, his point of view of the failure of present models to predict correctly anything, from the weather to the economy.

His main argument is composed of two points. First, he notices that natural systems are like some theoretical systems that are called automata systems: they are systems based on a set of local interacting rules. Among the three classes of automata systems, one is composed of uncomputable systems: there is no way to speed up the calculation and the only way to know the future of the systems is to run the model.

The problem, which is his second point, is that we are not and may never use the right set of rules. All known models have some kind of parameterization of the processes that are not modelled -because we do not model from the atom to a society. The additional difficulty is that models of natural systems are like natural systems, full of feedbacks, which make them highly sensitive to any parameterization. They don't even have to be chaotic to be completely wrong:

"By varying a handful of parameters within apparently reasonable bounds, we can get a single climate model to give radically different answers"

David Orrell, The future of everything, Chapter 8.

And thus, we might never be able to predict the future as Laplace dreamed of:

"Lack of predictability is a deep property of life. Any organism that is too predictable in its behaviour will die. And in an unpredictable environment, the ability to act creatively, while maintaining a kind of dynamic internal order, is a prerequisite. The balance of positive and negative feedback loops, when combined with the computational irreducibility of life processes, makes the behaviour of complex life forms impossible to accurately model. The problem is not that such organisms are erratic, but that they combine creativity with control. House plants are quite stable (they tend to stay in their pots and don't suddenly walk off to join the forest), but it would still be impossible to predict the exact effect of moving a single plant from a shaded spot to a warm greenhouse, based only on a detailed understanding of its biochemistry. If we can't do it for a plant, we can't do it for a planet. Life, it seems, evolves toward rich, complex structures, which defy simplistic analysis."

David Orrell, The future of everything, Chapter 8.

Thus, should we even bother to try to predict? The answer is yes because although the models are wrong, they are one way to try to predict the future. What the authors try to put a term is on the confidence, and at times arrogance, of modellers. They should be the first to recognize that their models are not perfect and, on top of it, are not that objective at all -the models are full of assumptions that are, after careful look, just a set of subjective views of the world hidden behind technical terms. Thus, the author would like some kind of balance: between the objective ways to predict the future and the subjective ones:

"Objectivity and subjectivity must be in balance, and inform each other, just like the positive and negative feedbacks loops that characterize living systems. We will choose to protect nature only if we value it -and not just as an object, but because it is alive. The only way we will respect it is if we understand that we cannot control it.
In non-linear, complex systems, change often happens abruptly, like water turning to ice. Extreme change is normal. This makes prediction difficult, but it also holds out tremendous hope, because it means that a sudden change in course can be expected. Such change often comes from the bottom up, rather than the top down [...]. Unlike deterministic mechanical systems, we have a choice; we can determine our own destiny. We are not slaves to the initial condition, our genes, or the efficient market. We are unpredictable, and that's not a bad thing.
The science of complexity will not build a better GCM [General Circulation Model], and neither Gaia theory or earth system science. Their stories are more of humility than of human ingenuity. But if we as a species are standing at a precipice, it is better that we see the world feelingly than be completely blinded by our mental models; that we know what we do not know. Creativity often emerges from a state of uncertainty. Grasping for illusory knowledge by over-modelling our environment is therefore part of the problem.
[...]
Mathematical models will always be indispensable. Like language, they are a way to understand the world, and organize and communicate our thoughts. They help us perform hypothetical experiments, explore possible scenarios, and expose fragilities. Most of all, they help us comprehend what is happening now."

David Orrell, The future of everything, Chapter 8, italics are mine.

Thus, modellers, keep doing the good work. But please, drop the certainty and try to be more humble.

Notes from The Ages of Gaia by James Lovelock

First, let's start with a word of wisdom:

"The young usually find the constraints of convention too heavy to escape, except as part of a cult. The middle-aged have no time to spare from the conservative business of living. Only the old can happily make fools of themselves."

James Lovelock, The Ages of Gaia, Introductory.

The second point concerns entropy and the fact that has become more and more obvious to me recently, that for any achievement, from the construction of a building to that of an idea, waste is unavoidable and this waste represents the entropy that needs to be rejected for the achievement to be meaningful, to be out of the ordinary and randomness:

"You, as you read these words, are creating entropy by consuming oxygen and the fats and sugars stored in your body. As you breathe, you excrete waste products high in entropy into the air, such as carbon dioxide, and your warm body emits to your surroundings infrared radiation high in entropy. If your excretion of entropy is as large or larger than your internal generation of entropy, you will continue to live and remain a miraculous, improbable, but still legal avoidance of the second law of the Universe. «Excretion of entropy» is just a fancy way of expressing the dirty words excrements and pollution. [...] We animals pollute the air with carbon dioxide, and the vegetation pollutes it with oxygen. The pollution of one is the meat of another. Gaia [Planet Earth] is more subtle and, at least until humans appeared, polluted the region of the Solar System with no more than the gentle warmth of infrared radiation."

James Lovelock, The Ages of Gaia, What is Gaia?

I will write, tomorrow, "Entropy" on the side of my garbage cans.

We keep going with an issue concerning the stability of a system and its level of complexity. Is the more complex a system, the more stable? This is, from what I have heard still a controversial issue. In his book, James Lovelock agrees with the theoretical ecologist, Robert May: the more complex a system, the more fragile and unstable and inversely. This goes maybe against the naive assumption that if a system has a greater diversity, it has a greater chance to handle external perturbation. But May's mathematics prove the contrary: "increasing complexity makes for dynamical fragility rather than robustness". Thus, "the complex natural ecosystems currently under siege in the tropics and subtropics are less able to withstand our battering than are the relatively simple temperate and boreal systems." (R. May, in Theoretical Ecology, cited in The Ages of Gaia, Exploring Daisyworld)

In the chapter Middle Ages, James Lovelock mentions an extraordinary theory that Earth biosphere would be responsible in part to...the plate tectonics:

"The geologist Don Anderson has speculated that the deposition of limestone on the ocean floor [via, for instance, the dying and sinking of Coccolithophores and the burying of their calcium shells] is a key factor in the motion of the Earth's crust. He proposed that sometime far back in the Earth's history, sufficient limestone was deposited to alter the chemical composition of the crustal rocks of the ocean floor near the continental margins. As a result an event, called the basalt-eclogite phase transition by geologists, took place. This transition so altered the physical properties of the crustal rocks that it became possible for the great machinery of plate movement to begin turning."

James Lovelock, The Ages of Gaia, The Middle Ages.

This theory would explain why plate tectonics are not a universal properties of the planets. I have no idea, however, if this idea has been dropped or is still alive. More reading will be needed. But fascinating idea nonetheless.

I will finish on an improved definition of Gaia's theory, that also gives some explanation on the origin of the interaction between the biosphere and its environment. James Lovelock, himself, has corrected a previous definition of Gaia's theory and has re-defined it in his book as follows:

"Living organisms and their material environment are tightly coupled. The coupled system is a superorganism, and as it evolves there emerges a new property, the ability to self-regulate climate and chemistry."

James Lovelock, The Ages of Gaia, Gaia since 1988.

In this definition, the co-evolution of the biosphere and its environment is the key to explain why the self-regulation of the system is a likely property. I know that there is a lot of criticisms against Gaia, even with this improved definition. Although I am also a bit skeptical, I am wondering why there is no such criticism relative to the thermal regulation of mammals. In this case as well, it should be hard to believe that cells can organize at such a higher level that the whole system succeeds in regulating its temperature. If such feast is possible for an organism, given the geological time over which evolution is working, why not for the Earth's system as well?

Across the scientific fields

Philip Anderson argues that although reductionism is the effective way to do science, it does not necessarily lead to an understanding of the whole:

"The ability to reduce everything to simple fundamental laws does not imply the ability to start from those laws and reconstruct the universe."

P. Anderson, More is different, in Science (1972), vol. 177.

For him, science is fundamental not only at the level of particles but also at every higher and more complex levels: an object, a fluid, an ocean, a body, a society, Earth, the Universe:

"The behavior of large and complex aggregates of elementary particles, it turns out, is not to be understood in terms of a simple extrapolation of the properties of a few particles. Instead, at each level of complexity entirely new properties appear, and the understanding of the new behaviors requires research which I think is as fundamental in its nature as any other."

P. Anderson, More is different, in Science (1972), vol. 177.

This point of view of science(s) has also been recently expressed by Edward O. Wilson in his book Consilience. and Ian Stewart in his book Does God play dice?.

The structure of science

Ian Stewart, in his book Does God Play Dice?, describes how science is structured. The explanations and theories provided by science are hierarchic; they start from the theories of fundamental particles and atoms, follow through theories of fluid dynamics, ecosystem, etc and finish with theories of sociology and art. Each explanation is constructed on top of the theories that are at a lowest level but in the same time, it does not care of the detail of these lower-level theories: the equations of fluid dynamics are constructed for a small water parcel, typically several moles of water, but it does not care about the individual atoms, nor about the fact that gravity has yet to be explained by the physics of particles. This important view of science is also shared by Edward O. Wilson in his book Consilience. Here is Ian Stewart's quote:

"Current science possesses no truly fundamental theories - not in the sense that they describe what nature actually does. They are all approximations, valid within some reasonably well-defined domain. Quantum mechanics work well at the submicroscopic level. General relativity is great for describing entire universes [...]. Science is a patchwork of models, each of which has been enormously refined within its own domain. The models habitually disagree when those patches overlap. Some disagreements are relatively harmless: atomic theory and continuum fluid mechanics disagree on the fine structure of water, holding it to be respectively to be discrete and infinitely divisible, but on macroscopic scales continuity and discreteness effectively approximate each other. Others are fatal: for example, as I write, the best current theory of astrophysics and the best current theory of cosmology compel us to accept stars older than the universe that contains them. Today's science is a pluralist patchwork of locally valid models, not a global monolith. Indeed it succeeds because it is a pluralist pacthwork of locally valid models.
Our concept of explanation is also a patchwork. A philosophical model that fits it well is what Richard Dawkins calls 'hierarchical reductionism', which sees scientific theories as a hierarchical structure, with some on different levels from others, corresponding to different levels of description of phenomena. (The hierarchy is not rigid and the levels need not be like layers of bricks in a wall.) For example, the complexities of ecosystems are explained by referring them back to those of organisms; organisms are explained by the growth of spatially organized proteins and other macromolecules; the complex organization of organisms is referred back to the linear complexity of their DNA code; the complexity of DNA is referred back to combinations of simpler atoms - and so on, right back to the Theory of Everything.
As Dawkins rightly remarks, it is not necessary to trace every phenomenon right back down this chain of reductions in order to understand it. Chemistry can be considered as 'given' for the purposes of understanding DNA; DNA can be taken as 'given' for the purpose of understanding protein manufacture in organisms, and so on.
[...]
What we tend to forget, when told a story with this structure, is that it could have had many different beginnings. Anything that lets us start from the molecular level would have done just as well. A totally different subatomic theory would be an equally valid starting-point for the story, provided it led to the same general feature of a replicable molecule. [...] It has to be or else we would never be able to keep a goat [within a wooden fence] without first doing a Ph.D. in subatomic physics."

Ian Stewart, Does God Play Dice?, Farewell, Deep Thought.

A pragmatic, positive and integrated use of the ideas of nonlinear science and complex systems

Manuel De Landa closes his book, A thousand years of nonlinear history, with a reflection on the pragmatic use of the ideas borrowed from nonlinear science and the science of complex systems. The two main forces that De Landa has been recognized throughout his book are homogenization and heterogenization. The first force assures a pyramidal or stratified construction with a hierarchy and a strong commandment, the second a flat or destratified organization or meshwork with no central control.

De Landa recognizes that linear science, adapted to describe hierarchic and stratified systems, have dominated the western thought for the last three centuries and thus limited our view of the world. On the other hand, the actual homogenization of the world, in terms of economies or ecology and occuring over many scales, have rendered the world more linear:

"[A]s our industrial, medical and educational systems became routinized, as they grew and began to profit from economies of scale, linear equations accumulated in physical sciences and equilibrium theories flourished in the social sciences. In a sense, even though the world is inherently nonlinear and far from equilibrium , its homogenization meant that those areas that have been made uniform began behaving objectively as linear equilibrium structures, with predictable and controllable properties." Manuel De Landa, A thousand years of nonlinear history, Conclusion and speculations.

Still, the nonlinear contribution is important and indeed necessary to avoid the world and Earth to become a dead, predictable and uncreative body. So what should we do? First, De Landa cautions not to adopt the extremist view of destroying the homogeneous part of the system. As Deleuze and Guattari wrote:

"If you free [the system] with too violent an action, if you blow apart the strata without taking precautions, then instead of drawing the plane you will be killed, plunged into a black hole, or even dragged into catastrophe. Staying stratified -organized, signified, subjected-, is not the worst that can happen; the worst that can happen is if you throw the strata into demented or suicidal collapse, which brings them back down on us heavier than ever." Deleuze and Guattari, A thousand plateaus, pp 160-161, cited in A thousand years of nonlinear history, Conclusion and speculations.

What a balanced and moderate view from post-modernists such as Deleuze and Guattari that are supposed to be, according to what I heard about them, more nihilist that positivist! It reminds me of Camus who wrote in, L'homme rebel, that revolutions were not necessary. Revolutions are actually counter-productive. They may do more harm than good and only by changing the world small steps at a time, can we achieve the ideal dreamed. De Landa, Deleuze and Guattari expand that idea acknowledging the complexity and nonlinearity of the system:

"This is how it should be done: lodge yourself on a stratum, experiment with the opportunities it offers, find an advantageous place on it, find potential movements of deterritorialization, possible lines of flight, experience them, produce flow conjunctions here and there, try out continuum of intensities segment by segment, have a small plot of new land at all times." Deleuze and Guattari, A thousand plateaus, pp 160-161, cited in A thousand years of nonlinear history, Conclusion and speculations.

De Landa explains that

[a]ll these precautions are necessary in a world that does not possess a ladder of progress, or a drive toward increased perfection, or a promised land, or even a socialist pot of gold at the end of the rainbow. Moreover, these warnings derive from a recognition that our world is governed not only by nonlinear dynamics, which makes detailed prediction and control impossible, but also by nonlinear combinatorics, which implies that the number of possible mixtures of meshwork and hierarchy, of command and market, of centralization and decentralization, are immense and that we simply cannot predict the emergent properties of these myriad combinations will be." in A thousand years of nonlinear history, Conclusion and speculations.

De Landa pursues:

"Thus the call for a more experimental attitude toward reality and for an increased awareness of the potential for self-organization inherent in even the humblest forms of matter-energy." in A thousand years of nonlinear history, Conclusion and speculations.

He finishes by emphasizing that this approach does not necessarily mean a hopeless and boring view of life. Far from it:

"It is important, however, not to confuse the need for caution in our exploration of the nonlinear possibilities of (economic, linguistic, biological) reality, and the concomitant abandonment of utopian euphoria, with despair, resentment and nihilism. There is, indeed, a new kind of hope implicit in these new views. After all, many of the most beautiful and inspiring things on our planet may have been created through [partial] destratification. A good example of this may be the emergence of birdsongs: the mouth became destratified when it ceased to be a strictly alimentary organ, caught up in the day-to-day eating of flesh, and began to generate other flows (memes) and structures (songs) where the meshwork element dominated the hierarchical. The emergence of organic life itself, while not representing a more perfect stage of development than rocks, did involve a greater capacity to generate self-consistent aggregates, a surplus of consistency. The human hand may also have involved a destratification, a complete detachment from locomotive functions and a new coupling with the external environment, itself further destratified when the hand began converting pieces of it (rocks, bones, branches) into tools. Thus, despite all the cautionary tales about simplistic calls for anarchic liberation, there is in these theories a positive, even joyful conception of reality. And while these views do indeed invoke the «death of man», it is only the death of «man» of the old «manifest destinies», not the death of humanity and its potential for destratification." in A thousand years of nonlinear history, Conclusion and speculations.

A summary of "A thousand years of nonlinear history": the geological component and why the West dominates the World

Manuel De Landa ambitiously borrows the vocabulary and concepts from the science of nonlinearity and complex systems and applies them to the history of Europe from the year 1000 to 2000. The main goal of the book is to draft a possible schematic trajectory explaining the domination of Europe during that period over the rest of the world, especially the empire of Islam and China. The idea is that this domination came up because of the lucky congruence of multitude of complex and interacting economic, climatic, geographic and social processes that triggered auto-catalytic processes, that it is processes fuelling themselves via positive feedbacks, and not because of either the fate of History as Marx suggested, the sole power of some top-down political and economical concept as capitalism and its (really) "invisible hand", or a fundamental dominating aspect of the European people, such as a psyche or a religion shaped for "success". In this sense, De Landa follows the traces of many (western) authors who have been trying to re-equilibrate the idea that we have of the creative power of the different cultures in the History and put back the spoiled child 'Europa' to its right place, as Guns, Germs and Steel by Jared Diamond (one of the best book and theory ever) successfully did.

The originality of De Landa is to avoid at every instant the use of any subjective concious or unconscious explanations by constantly using the scientific language, metaphors but also actual processes from the science of complex systems.

The first part deals on the geological aspect of the European society, its cities, institutions and economical system:

"From this point of view cities arise from the flow of matter-energy, but once a town's mineral infrastructure has emerged, it reacts to those flows, creating a new set of constraints that either intensifies or inhibits them. Needless to say, the walls, monumental buildings, streets, and houses of a town would make a rather weak set of constraints if they operated on their own. Of course, they do not. Our historical exploration of urban dynamics must therefore include an analysis of the institutions that inhabit cities, whether the bureaucracies that run them or the markets that animate them. Although these institutions are the product of collective human decision making, once in place they also react back on their human components to limit them and control them, or, on the contrary, to set them in motion or accelerate their mutation."

Manuel De Landa, A thousand years of nonlinear history, Geological history: 1000-1700 AD


Notice the vocabulary of dynamics employed and the numerous feedback loops considered in that explanation. One of the virtuous cycle that exists, according to De Landa, is the combination of markets and anti-markets. The first are actual markets, composed of small-scale and truly capitalistic companies where competition rules, while the second are composed of large-scale companies that are anti-capitalistic by preventing competition to exercise its force:

"Markets and bureaucracies, as well as planned and unplanned cities, are concrete instances of a more general distinction: self-organized meshworks of diverse elements, versus hierarchies of uniform elements. But again, meshworks and hierarchies not only coexist and intermingle, they constantly give rise to one another.
[...]Thus, once markets grew past the size of local, weekly gatherings, they were ranked and organized form the top, giving rise to a hybrid form:a hierarchy of meshworks. The opposite hybrid, a meshwork of hierarchies, may be illustrated by the system of power in the Middle Ages."

Manuel De Landa, A thousand years of nonlinear history, Geological history: 1000-1700 AD

The chance of Europe, that Islam or China did not have, was that it was never too anarchic nor too controlled. Its chance has been the presence of that multitude of people, competing each other over that relatively small piece of land, where the virtuous cycle of markets and anti-markets survived until invading the institutions and the psyche of Europe. In some sense, even without mentioning it, De Landa describes here the concept of the edge of chaos, the limit between too much order and too much randomness; only at the edge, life and creativity survive. Islam and China have both a too strong central commandment that prevented, according to De Landa, the local market-like processes to develop and bring the innovations to the high level necessary for a civilisation to win the evolutionary game:

"The emergence of powerful nation-states, and the concomitant decrease in the autonomy of the cities they absorbed (and even of the city-states that remained independent), could have brought the different forms of self-stimulating dynamics we have described to a halt. That this did not happen was due yet to one more form of autocatalysis unique to the West: continued arm races. [T]his type of self-stimulation depended in turn on the fact that the nations of Europe, unlike China and Islam, were never able to form a single, homogeneous empire, and have remained until today a meshwork of hierarchies. It was within this meshwork that advances in offensive weaponry stimulated innovations in defense technology, leading to an ever-growing armament spiral.
[...]
Many of the inventions that Europeans used to colonize the world (the compass, gunpowder, paper money, the printing press) were of Chinese origin, while Europe's accounting techniques and instrument of credits (which are often cited as examples of her unique «rationality») came from Islam. Thus, nothing intrinsic to Europe determined the outcome, but rather a dynamics bearing no inherent relationship to any culture. [A]n excess of centralized decision making in the East kept turbulent dynamics under control, while they raged unobstructed in the West. To be sure, at several points in her history Europe could have become a unified hierarchy, and this would have ground these dynamics to a halt. This happened in the sixteenth century with the Hapsburg Empire, and later on with the rise of Napoleon and Hitler. Yet all these efforts proved abortive, and European nations remained a meshwork."

Manuel De Landa, A thousand years of nonlinear history, Geological history: 1000-1700 AD

As again, those theories and trajectories are hard to proved. They are only suggestive but they have the merit to bring a more moderate albeit complex view of the world and to liberate ourself from the extremist, at times naive and too simplistic, visions that we have of our world. The Truth might lay somewhere in between.

A brief history and critic of complexity theory

This is a brief history synthesized by E. O. Wilson in Consilience of the accomplishments and promises of the science of complexity as well as its present drawbacks and its objectives needed to be fulfilled to convince a larger portion of the science community.

"Complexity theory was born in the 1970s, gathered momentum in the early 1980s, and was envelopped in controversy by the mid-1990s. The issues of contention are almost as tangled as the systems the theorists hoped to unravel. I think it possible to cut to the heart of the matter, as follows. The great majority of scientists, their minds focused narrowly on well-defined phenomena, do not care about complexity theory. Many have not yet heard of it. [...] Those who care can be divided into three camps. The first comprises a heterogeneous scattering of skeptics. They believe that brains and rain forests are too complicated ever to be reduced to elementary processes, let alone reconstituted in a manner that predicts the whole. Some of the skeptics doubt the existence of deep laws of complexity, at least any that can be grasped by the human mind.

In the second camp are the fervent advocates, a band of audacious complexity theorists, exemplified by Stuart Kauffman (author of the The origins of order) and Christopher Langton, who work at the Santa Fe Institute in New Mexico, unofficial headquarters of the complexity movement. They believe not only that deep laws exist but that their discovery is on the near horizon. Some of the essential elements of the laws, they say, are already emerging from mathematical theories that use exotic conceptions such as chaos, self-criticality, and adaptative landscapes.[...] Their grail is a set of hoped-for master algorithms that will speed passage from atom to brain and ecosystem, consistent with reality but requiring far less factual knowledge than would be needed without the algorithms.

The third group of scientists, of which I am a reluctant member, has settled along positions strung between the two extremes of rejection and unbridled support. I say reluctant, because I would like to be a true believer: I really am impressed by the sophistication and élan of the complexity theorists, and my heart is with them. But my mind is not, at least not yet. I believe with many other centrists that they are on the right track -but only more or less, maybe, and still far short of success. [...] The basic difficulty, to put the matter plainly, is an insufficiency of facts. [...] The postulates they start with clearly need more detail. Their conclusions thus far too vague and general to be more than rallying metaphors, and their abstract conclusions tell us very little that is really new.
[...]
None of the elements of complexity theory has anything like the generality and the fidelity to factual detail we wish from theory. None has triggered an equivalent cascade of theoretical innovations and practical applications. What does complexity theory need to be successful [...]?

Complexity theory needs more empirical information."

Edward O. Wilson, Consilience (Chapter 5)

Obviously, this discussion needs to be continued and I will keep posting critics and successes of complexity theory.

From complexity to reductionism and back to complexity

Edward O. Wilson in his Consilience describes the two fundamental steps of science to understand Nature. First, you need to break down the processes into simple blocks. That is reductionism. Then you need to build back the complexity of the system. "To dissect a phenomenon into its elements, [...] is consilience by reduction. To reconstitute it, and especially to predict with knowledge gained by reduction how nature assembled it in the first place, is consilience by synthesis. That is the two-step procedure by which natural scientists generally work: top down across two or three levels of organization at a time by analysis, then bottom up across the same levels by synthesis" (Chapter 5).

"The greatest challenge today, not just in cell biology and ecology but in all of science, is the accurate and complete description of complex systems. Scientists have broken down many kinds of systems. They think they know most of the elements and forces. The next task is to reassemble them, at least in mathematical models that capture the key properties of the entire ensembles. Success in this enterprise will be measured by the power researchers acquire to predict emergent phenomena when passing from general to more specific levels of organization. That in simplest terms is the great challenge of scientific holism" (Chapter 5).

There is also an order of complexity. From "simple" to more complex, we find: physics, followed by biology, followed by sociology, with the arts closing the chain. "[T]he opposite journey from physics to end points, is extremely problematic. As the distance away from physics increases, the options allowed by the antecedent disciplines increase exponentially. Biology is almost unimaginably more complex than physics, and the arts equivalently more complex than biology. To stay on course all the way seems impossible. And worse, we cannot know before departure whether the complete journey we have imagined even exits" (Chapter 5).

"The profane word now having been spoken on hallowed ground, a quick disclaimer is in order. While it is true that science advances by reducing phenomena to their working elements -by dissecting brains into neurons, for example, and neurons into molecules- it does not aim to diminish the integrity of the whole. On the contrary, synthesis of the elements to re-create their original assembly is the other half of scientific procedure. In fact, it is the ultimate goal of science" (Chapter 10).

Such disclaimer should lessen the critics of reductionism. Reductionism is a fundamental stage that we need to go through to understand Nature. We have barely started to reconstruct the parts. Patience.

Advantages and disadvantages of modelling

Ludwig von Bertalanffy writes about any attempt to model Nature and its constituents:

"Conceptual models which, in simplified and therefore comprehensible form, try to represent certain aspects of reality, are basic in any attempt at theory; whether we apply the Newtonian model in mechanics, the model of corpuscle or wave in atomic physics, use simplified models to describe the growth of a population, or the model of a game to describe political decisions. The advantages and dangers of models are well known. The advantage is in the fact that this is the way to create a theory -i.e. the model permits deductions from premises, explanation and prediction, with often unexpected results. The danger is oversimplification: to make it conceptually controllable we have to reduce reality to a conceptual skeleton- the question remaining whether, in doing so, we have not cut out vital parts of the anatomy. The danger of oversimplification is the greater the more multifarious and complex the phenomenon is. This applies not only to «grand theories» of culture and history but to models we find in any psychological or sociological journal.

Ludwig von Bertalanffy, General system theory, Chapter 8

An unlikely place for a confrontation between universalism and complexity

I found a very sensitive illustration of the confrontation between universalism/reductionism and complexity/relativism in a quite unlikely place. André Gorz writes in Lettre à D., Histoire d'un amour about the different way his wife, an englishwoman, and himself, influenced by French universalism, used to think:

"J'avais besoin de théorie pour structurer ma pensée et t'objectais qu'une pensée non structurée menace toujours de sombrer dans l'empirisme et l'insignifiance. Tu répondais que la théorie menace toujours de devenir un carcan qui interdit de percevoir la complexité mouvante du réel."

How lucid this statement is!

See also this post about the same confrontation.

A definition of complexity

Murray Gell-Mann gives a definition based on information theory of complexity in his book The quark and the jaguar. In this context, a measure of complexity can be associated with the actual length (in bits for instance) of the description of the regularities in a message.

If the message is perfectly regular, that is composed of only 1's, then the length of this regularity is short, "only 1's", and the complexity is small. On the other hand, if the message is perfectly random, so much so there is no regularity whatsoever, then the complexity is also low. Only when there is many different regularities together, the complexity will be large.

Although this definition might not satisfy a lot of people, at least it is the first time that I read any attempt to quantitatively define complexity.

universality versus relativism, reductionism versus complexity

This is an ambitious post and the topics are larger than life but I would like to share here some thoughts from The moment of complexity by Mark. C. Taylor. At the beginning of his essay, Taylor gives us an original account of the history of the debate on, let's say it, how to see the world: are there universal laws or is everything relative - culturally rather than physically speaking? what is the correct method to comprehend Nature, reduce the problem to a series of simple problem or see it as a whole? These are of course very general and deep questions, which may have as many answers as sand grain on a beach. But this does not prevent us to, at least, discuss about it.

Taylor details the philosophy of several intellectuals, among whom that of Lévi-Strauss. Lévi-Strauss is an advocate of universal laws and a reductionist view of the human culture. He is quoted saying:

"By drawing up an inventory of all the customs that had been observed, all those imagined in myths, those evoked in the games of children and adults, and the dreams of healthy and sick individuals and psychopathological behaviors, one would be able to draw up a period table like that of the chemical elements, in which all real or merely possible customs would appear grouped in families, and in which we would simply need to recognize those which societies have in fact adopted.

(Lévi-Strauss, quoted in The moment of complexity by Mark. C. Taylor, chapter 2; italics are mine)

There are many advocates of the other point of view, one of them, and maybe not the most clearer one according to Taylor, is Foucault. Foucault does not deny there are some kind of universal laws but he insists that these laws have been constructed within our own cultural frame, so that these laws are not, I will dare to say it, "universal enough":

"[W]hile admitting that there is an «order of things», he insists that this order is neither natural nor essential and thus cannot be preordained or unchanging. Whatever order is at work the world is historically contingent and therefore to a certain extent arbitrary."

(Mark C. Taylor, The moment of complexity, chapter 2)

This debate is typical between the scientist who applies reductionism in a professional way, and the humanist who restrains himself from doing so. Taylor writes "[w]hile certain scientists tend to reduce culture to nature, many humanists defiantly reduce nature to culture" (Mark C. Taylor,The moment of complexity, chapter 7). For the former, the understanding of human culture and its psychology is attainable from the study of the building blocks that are the genes: you have or have not the gene of painting, singing, writing, seeing the world this way and not this way etc. For the later, nature itself is a psychological construction different between individuals so that little can be said in general.

For Edward O. Wilson, reductionism, far from denying the complexity of nature, should rather be seen as a tool, not a philosophy or a statement about the world. For him, reductionism is "the search strategy employed to find points of entry into otherwise impenetrably complex systems. Complexity is what interests scientists in the end, not simplicity" (Edward O. Wilson, Consilience: the unity of knowledge, cited in The moment of complexity, chapter 2). Edward O. Wilson adds the following lucid statement:

"The love of complexity without reductionism makes art; the love of complexity with reductionism makes science."

A sensible illustration of such confrontation is also amazingly found in Lettre à D., Histoire d'un amour by André Gorz writing about the different way his wife, an englishwoman, and himself, influenced by French universalism, think:

"J'avais besoin de théorie pour structurer ma pensée et t'objectais qu'une pensée non structurée menace toujours de sombrer dans l'empirisme et l'insignifiance. Tu répondais que la théorie menace toujours de devenir un carcan qui interdit de percevoir la complexité mouvante du réel."

Although I have a lot of respect about critics of science, because as Descartes said we should always doubt, I have to admit that Edward O. Wilson's view seems correct. As again and again, the conclusion from this debate is not so much that one side is necessarily correct and the other wrong, but that each point of view is adapted to different situations or goals.

To conclude this little post, I would like to bring to your attention that according to Marc C. Taylor, it is possible to go beyond this classic debate. One of the mind-blowing example is the feedback that an idea can have on the brain itself. For instance, in linguistic, there is an academic school which thinks that the brain developed also from the use of the words themselves. Terrence Deacon writes that "the major structural and functional innovations that make human brains capable of unprecedented mental feats evolved in response to the use of something as abstract and virtual as the power of words" (T. Deacon, The coevolution of language and the brain, cited in The moment of complexity, chapter 7). James Gardner says also that "information can and does flow upstream into the genome from the particular extended phenotype we know as human civilization" (cited in The moment of complexity, chapter 7).

Applied to the debate surrounding reductionism as I understand it, the simple schema the scientific community chooses to describe Nature would have a certain effect on the brain over time, would physiologically improve it so that the brains of the scientists would come up with a new schema, more adapted to the world. In some way, it is true that the view of science has evolved and still evolves dramatically ; the idea here is that the improvement in the description of Nature has not been completely independent of the previous descriptions; in other words, the description of the world at a certain time has helped to shape the new description at a later time via a physiological change of the brain itself. In this case, reductionism still is the basic tool to comprehend Nature, but the reductionist schema has some internal subjectivity associated with the physiological limits of the brain. I do not know how far this is accepted but it is true that, if correct, it enlarges the debate and may bring possibilities to accommodate both the scientist and the humanist.

An example of convergence of physics and spirituality

I was recently struck when reading the extracts of a discussion hold by Osho, a spiritual Indian leader (see Osho, Aimer vivre). In the same way as the science of complexity is discovering that most of the organized structures on Earth, living or not, seems to be located at an edge between order and chaos, Osho tries to teach his "students" to be in the middle between a rigid discipline and a flexible spontaneity, which he himself calls chaos, and to always be able to shift from one end to another.

Wisdom or madness of crowds?

Here are my present conclusions concerning my readings of the "Wisdom of Crowds" by James Surowiecki and some other articles.

First and foremost -in order to kill any controversy-, accepting the wisdom of crowds is not accepting an unproven concept coming from above like the idea of God. Accepting the wisdom of crowds is opening the mind to some cases where, when specific conditions are fulfilled, the crowd will be wiser or as wise as the wisest individual in the crowd. Accepting the wisdom of crowds is rather asking the old concept that only experts can resolve problems to step down its throne, to accept its failures once in a while and that there are other ways to find a solution.

If we do accept that an expert or a group of experts can perform the job in some circumstances and that a group of laymen can perform the job in other circumstances, then there is no controversy. By accepting this, not only the experts still appear useful for a society, and more than before, but new ways to tackle an issue are found. These new ways to resolve a problem are due to our recent technological prowess such as the Internet; bear in mind that these are just the tools, and that they have invaded our society before we even formulated their purpose and function. That is why we, as a society, have so much difficulty in accepting their capabilities.

So, yes an expert can be the perfect solution for a job. And yes, a crowd can also be the perfect solution for another job. So, what types of job each is capable of performing? In the former case, a job which needs very specific knowledge, so specific that only a few people are familiar with. This happens in research for instance -and I am not writing this because I am a researcher myself. We are not going tomorrow to make a group of laymen to work on the Equatorial Deep Jets (a very obscure set of currents found around the equatorial belt of some oceans -my subject) or on the latest string theory. On the other hand, crowds will perform well if the job consists in repeated predictions on a complex problem, the issues and outcomes of which are not too far from the general cultural baggage of the members of the crowds. In this case, the experts not only might be quickly dry of new ideas to resolve the new problems but they may actually be wrong, basing their prediction on out-of-date or too simplistic models.

We do know many examples when crowds are wrong (or mad). This is such a cliché that it does not need a list of examples. On the other hand, although we do complain once in a while how experts can be wrong, we always have the feeling that we are unfair and speak more with our heart than with reason. But the facts are, and there are many, so-called experts do not perform well and can actually over a long period of time perform very badly. This is the case, to take one example, of the CEOs and other financial gourous who on average performed poorly during the speculative financial bubble of the 90's. The share of their companies have actually decreased over this time.

Now, we know, or guess, the necessary conditions to be an expert: many years of study, scrutinized work by peers, etc. But what about the crowds? How can we assure that a group a priori ready to resolve a problem will succeed? Three main points: 1) diversity, 2) independence and 3) a dose of centralization. Diversity will assure you a wide range of solutions, one of them being maybe the best one. Independence will avoid one member to influence the thinking of the other members and avoid the group to actually act as an individual. The last point is that you need information to flow between all the members and in all directions: too much centralization and the system is dead, too tied, not enough and the system is anarchist, too dispersive, incoherent.

Good and bad experts exist. Mad and wise crowds exist also. It is the latter that we just discovered and goes so much against our individual-oriented society, where each brain is important and has its value, that it is hard to accept.

Now, I have to admit that there is a trick. How can you define the "best" solution as I have vaguely used? In some cases, such as the market, the "best" solution varies from one person to another and it is then a question of values rather than of performance. If you ask me if the market performs well, I will say yes if what you want is the shares of each company to increase with time in some orderly way. I will say no if what you want is a better society where its actual living members are doing well. So, next time you discuss the madness or wisdom of crowds, be sure first that everybody agrees on what is the best solution. Depending on the solution, either experts, or crowds or a mix of the two will be necessary.

Beyond Newton's law

Since Lorenz (1963), physicists came to realize that the predictability of the future state of a deterministic system, that is a system whose dynamical equations are known, was not a trivial matter of computation. For over two centuries, and still a strong belief in the present Western societies, one thought that given the initial and boundary conditions, every deterministic system should be perfectly predictable.
During the 20th century, one started to realize instead that for most systems, except if one is armed with infinite computational power, there will be a time in the future where our prediction will be completely false. Poincaré already noticed that although the two-body problem was a perfectly resolved exercise, in other words a dead problem, the three-body one, however, was not. Predicting the future state of the system Moon, Earth and Sun is still a relatively hot topic.
During the last 50 years, some scientists have started to attempt to overcome this obstacle of unpredictability. This is the science of complexity where one still tries to extract universal patterns from systems which appear chaotic or random. So far, it seems that this science has mostly produced qualitative results, so much so that some have started to doubt its real scientific benefits and have accused it of being at times too philosophical, even mystical. But this science first and foremost does not deny the complexity of natural systems, which is itself a great step forward. By doing so, it attempts to go beyond the predictability problem and to fulfill Newton's dream to comprehend Nature. So much has yet to be learnt, and the science of complexity has so far successfully shaken our stern belief in equilibrium and in linear causal chains of events.

Lorenz, E. N., 1963, "Deterministic nonperiodic flow", Journal of the Atmosphere Sciences, 20, 130-141.