Information in Physics

Full Title or Meme

There doesn't seem to be a good definition of this at the present time.

• How to Better Define Information in Physics is an article that does a good job of exploring the various definitions.

Context

Since the time of Boltzmann information about physical objects has been important in some areas of physics, specifically in thermodynamics.

First published as Physics from Fisher Information in 1999, Roy Frieden has been claiming that observable physics (collapse of the wave function) can be described as the loss of Fisher Information. the fisher information can be interpreted as the inverse of the standard error (squared), but only when the log-likelihood is quadratic (i.e., the Gaussian log-likelihood). In the vast majority of cases, we are not dealing with a gaussian population … but, the log-likelihood of the MLE will often rapidly converge to a quadratic, especially around ±2 standard deviations. In this case, treating the inverse of the fisher information as an estimated precision of an estimate will be approximately correct.

Fisher Information and Shannon Information are two different concepts. Fisher Information is related to the asymptotic variability of a maximum likelihood estimator. Higher Fisher Information is associated with lower estimation error. Fisher Information is used to investigate the precision of the neural code when studying large populations of neurons. Shannon Information refers to the content of the message or distribution, not its variability. Shannon mutual information is used to investigate the precision of the neural code when investigating very small populations of neurons In summary, Fisher information is related to estimation error while Shannon information is related to message content or distribution.

Despite the above in 2010 Kevin Knuth declared " Information physics, which is based on understanding the ways in which we both quantify and process information about the world around us, is a fundamentally new approach to science."

How to Better Define Information in Physics

• See wiki page on Complexity Theory

Types of Information

• Gaussian - error bounds
• Boltzmann statistics of distinguishable (known) particles
• Planck & Bose - statistics of indistinguishable particles
• Pauli & Dirac - statistics of particles subject to exclusion principle
• Fisher - population statistics
• Shannon - information content

Entropy

the entropy of the universe is constantly increasing. The second law of thermodynamics states that the state of entropy of the entire universe, as an isolated system, will always increase over time. Energy always flows downhill, and this causes an increase of entropy. Entropy is the spreading out of energy, and energy tends to spread out as much as possible. It flows spontaneously from a hot (i.e. highly energetic) region to a cold (less energetic) region. As a result, energy becomes evenly distributed across the two regions, and the temperature of the two regions becomes equal. The same thing happens on a much larger scale. The Sun and every other star are radiating energy into the universe. However, they can’t do it forever. Eventually the stars will cool down, and heat will have spread out so much that there won’t be warmer objects and cooler objects. Everything will be the same very cold temperature. Once everything is at the same temperature, there’s no reason for anything to change what it’s doing. The universe will have run down completely, and the entropy of the universe will be as high as it is ever going to get.