Complexity: A Guided Tour by Melanie Mitchell

• Reductionism is the most natural thing in the world to grasp. It’s simply the belief that “a whole can be understood completely if you understand its parts, and the nature of their ‘sum’.”
• Reductionism has been the dominant approach to science since the 1600s.
• Since the time of Descartes, Newton, and other founders of the modern scientific method until the beginning of the twentieth century, a chief goal of science has been a reductionist explanation of all phenomena in terms of fundamental physics.
• In complex systems, many simple parts are irreducibly entwined, and the field of complexity is itself an entwining of many different fields.
• Common properties of complex systems
• Complex collective behavior
• Signaling and information processing
• Adaptation
• Now I can propose a definition of the term complex system: a system in which large networks of components with no central control and simple rules of operation give rise to complex collective behavior, sophisticated information processing, and adaptation via learning or evolution.
• Systems in which organized behavior arises without an internal or external controller or leaders are sometimes called self-organization.
• Since simple rules produce complex behavior in hard-to-predict ways, the macroscopic behavior of such systems is sometimes called emergent.
• Here is an alternative definition of a complex system: a system that exhibits nontrivial emergent and self-organizing behavior.
• The central question of the sciences of complexity is how this emergent self-organized behavior comes about.
• It was the understanding of chaos that eventually laid to rest the hope of perfect prediction of all complex systems, quantum or otherwise. The defining idea of chaos is that there are some systems--chaotic systems---in which even minuscule uncertainties in measurements of initial position and momentum can result in huge errors in long-term predictions of these quantities. This is known as “sensitive dependence on initial conditions”.
• A linear system is one you can understand by understanding its parts individually and then putting them together.
• A nonlinear system is one in which the whole is different from the sum of the parts.
• Seemingly random behavior can emerge from deterministic systems, with no external source of randomness.
• The behavior of some simple, deterministic systems can be impossible, even in principle, to predict in the long term, due to sensitive dependence on initial conditions.
• In summary, changing, hard-to-predict macroscopic behavior is a hallmark of complex systems.
• Dynamical systems there provide a mathematical vocabulary for characterizing such behavior in terms of bifurcations, attractors, and universal properties of the ways systems can change.
• In short, classical mechanics attempts to say something about every single microscopic entity by using Newton’s laws. Thermodynamics gives laws of macroscopic entities--heat, energy, and entropy--without acknowledging that any microscopic molecules are the source of these macroscopic entities. Statistical mechanics is a bridge between these two extremes, in that it explains how the behavior of the macroscopic entities arise from statistics of large ensembles of microscopic entities.
• Shannon’s definition of information involves a source that sends a message to a receiver.
• The second law of thermodynamics states that the total entropy of an isolated system will always increase until it reaches its maximum value.
• To summarize the major ideas of Darwin’s theory:
• Evolution has occurred; that is, all species descended from a common ancestor. The history of life is a branching tree of species.
• Natural selection occurs when the number of births is greater than existing resources can support so that individuals undergo competition for resources.
• Traits of organisms are inherited with variation. The variation is in some sense random--that is, there is no force or bias leading to variations that increase fitness. Variations that turn out to be adaptive in the current environment are likely to be selected, meaning that organisms with those variations are more likely to survive and thus pass on the new traits to their offspring, causing the number of organisms with those traits to increase over subsequent generations.
• Evolutionary change is constant and gradual via the accumulation of small, favorable variations.
• The collection of all of an organism’s physical traits--its phenotype--comes largely due to the character of interactions between proteins in cells. Proteins are long chains of molecules called amino acids.
• Subsequences of DNA from genes. Roughly, each gene codes for a particular protein. It does that by coding for each of the amino acids that make up the protein. The way amino acids are coded is called the genetic code. The code is the same for almost every organism on Earth.
• Each amino acid corresponds to a triplet of nucleotide bases.
• One simple measure of complexity is size. By this measure, humans are about 250 times as complex as yeast if we compare the number of base pairs, but only about four times as complex if we count genes.
• Another proposed measure of the complexity of an object is simply its Shannon entropy, defined [...] to be the average information content or “amount of surprise” a message source has for a receiver.
• An important missing piece for current day computers is the ability to make analogies.
• Perceiving abstract similarities is something computers are notoriously bad at. That’s why I can’t simply show the computer a picture, say, of a dog swimming in a pool, and ask it to find “other pictures like this” in my online photo collection.
• A model, in the context of science, is a simplified representation of some “real” phenomenon. Scientists supposedly study nature, but in reality much of what they do is construct and study models of nature.
• Models are ways for our minds to make sense of observed phenomena in terms of concepts that are familiar to us, concepts that we can get our heads around.
• Models are also a means for predicting the future [...].
• Network thinking means focusing on relationships between entities rather than entities themselves.