Mathematical Modelling of Consciousness — Free Energy Principle
Since the boom of the era of Artificial Intelligence I have come across a lot of Literature on the Perceptron based Deep Learning Architectures which speaks for how these models are learning the human cognition phenomenons using the mathematical approximation methods. I was not able to accept the fact that these mathematical approximations whose end result is pattern matching are human alias of cognitive systems. The work of Francois Chollet on Measure of intelligence and ARC challenge clearly states the fact that we are way too digressed from the path of the creation of the intelligent systems. While if you have a look at the work of Walid Saba, his research in the field of NLU puts a big question mark on the state of the art current generation of Langauge Models, how these models are just using some intelligent shorctut learning methods to give an impression that they are actually learning language. The end result was creation of Winograd Schema a robust and thought through alternative of the Turning test.
Th above research literature brought me back to point zero, that the systems that we are building are just compute machines learning to interpolate the new data points on a N-X dimensional sub-vector space manifold present in a N dimensional vector space. While my ship of hopes was sailing towards a deep sinking eternal vortex, I came across this brilliant piece of literature which tries to formulate the notion of the consciousness, existence and being alive in a mathematical model called Free Energy Principle, which is the work of the best known Neuroscientist of any generation Karl Friston.
Existence, Survival and Optimisation
Every single being on the planet has an inherent or undeniable goal, survival and survival in a conditioned environment requires a strategy to be followed, which help achieve this goal and this inherently becomes an optimisation problem where value — expected reward is a quantified entity maximising the probability of achieving the goal while minimising the goal digression paths or prediction errors. This quantity is what optimized under the Free Energy Principle. Therefore, Existence, Survival and Optimization mechanism Hierarchy travels on the ladder of FEP.
Free Energy Principle
The free-energy principle says that any self-organising system that is at equilibrium with its environment must minimise its free energy. The principle is essentially a mathematical formulation of how adaptive systems (that is, biological agents, like animals or brains) resist a natural tendency to disorder.
Motivation : resisting a tendency of disorder, The brain has internal and external states, the internal states are the physiology intrinsic to the brain while external state are the state of the brain getting influenced by the environment. The sensor states in which an organism can exist at an instance of time defines its phenotype. If we talk in mathematical sense, the probability of sensory states have a low entropy or high determinism, so for example if there are 5 sensory states, 4 of them will have a low entropy and 1 of them will have a high entropy overall the organism has a low sensory states entropy.To understand what entropy is, I would suggest you read the work Claud Shannon on the information theory.
Example, A fish that frequently forsook water would have high entropy. Note that both surprise and entropy depend on the agent: what is surprising for one agent (for example, being out of water) may not be surprising for another. Biological agents must therefore minimise the long-term average of surprise to ensure that their sensory entropy remains low. In other words, biological systems somehow manage to violate the fluctuation theorem, which generalises the second law of thermodynamics.
Mathematical Formulation
In the above diagram, the section a shows the dependencies that define free energy. Internal states of the brain are defined by μ(t) and the quantities describing its exchange with environment(Sensations) s̃(t) = [s,s′,s′′…] and w.r.t to each sensory input a corresponding action is taken at t, a(t). The sensation is caused by Phi ⊃ {x̃, θ, γ } of sensory input comprise hidden states x̃(t), parameters θ and precisions γ controlling the amplitude of the random
fluctuations z̃(t) and w̃(t).
Internal brain states and action minimize free energy F(s̃,μ), which is a function of sensory input and a probabilistic representation q(θ|μ) of its causes. This representation is called the recognition density.
Section b : Therefore, the free energy depends on two probability densities, the recognition density(internal to the internal states) and the generative one which is outcome of the sensory samples and their causes, p(s̃,θ|m).
The agent implicitly infers or represent the causes of its sensory samples in a Bayes optimisation fashion. At the same time, the free energy becomes a tight bound on surprise, which is minimised through actions. That means actions have to be updated in the form of posterior probability, which reduces the surprise or the entropy coming to the sensory input, which increases the entropy of the internal states.
Acting on the environment by minimising free energy enforces a sampling of sensory data that is consistent with the
current representation. This can be seen with a second rearrangement of the free energy as a mixture of accuracy and complexity. The brain will reconfigure its sensory epithelia to sample inputs that are inferred by the internal state to minimise the prediction error.
In a nutshell, the internal state interacts with the external state and tries to minimise the external state’s influence on its internal states which helps reduce the variational free energy. The lower influence of the sensory input to the internal state of a being leads to an action which reduces prediction errors on the task for which the internal state is minimising its free energy. With this reduced influence of the external state on the sensory input, it stabilises the internal state’s entropy w.r.t to the environment and make the perception a consistent phenomenon, when the perception is consistent w.r.t to the environment the predictions are stable and errors are less.
Variational free energy has been exploited in machine learning and statistics to solve many inference and learning problems. In this setting, surprise is called the (negative) model evidence. This means that minimising surprise is the same as maximising the sensory evidence for an agent’s existence, if we regard the agent as a model of its world. In the present context, free energy provides the answer to a fundamental question: how do self-organising adaptive systems avoid surprising states? They can do this by minimising their free energy.
Thus, this phenomenon of making the perception of the environment and self, less ambiguous in terms of the entropy and gives rise to the recognition of the self and birth to the consciousness.
This is the first part of the review of the brain functioning hypotheses, which have been put forward in the academia community. In the next part of we will look into the Bayesian brain hypothesis.
Acknowledgement :
