Wednesday August 31 2022: Phases of learning

About neural learning, neural systems should have different phases of learning. I would argue that the first phase has to be one where the energy (or information, or prediction error) imposes itself onto the energy state of the neural system. At this point, the neural system's encoding of this novel information is still only in terms of thermodynamics or flows of the received energy. We might observe this in the form of the neural system's post-synaptic potential and axonal spike activity. It is critical to note that changes in activity are by definition a dynamic concept. Thus, the information is not encoded in one static activity state. Rather, the complete information packet as it were must be represented by a window of dynamic activity fluctuations. Sustaining such a non-homeostatic state of activity is untenable in the long run. Therefore, this dynamic activity must be resolved in the neural system's architecture. Here, the novel information graduates from thermodynamics to physical structures. We might observe short- or long-term potentiation or depression, directed shaping of short and long-range connections characteristic of pruning and neural development. All these consume energy, and likely in a way that reflects the dynamic activity that is the packet of information received.

A necessary question is then what sort of information is received, and what sort of information is bypassed by neural systems. This consideration stems from the fact that the neural system cannot possibly encode all encountered fluctuations of energy in the environment. I would argue that the information received and not bypassed might be construed as information that is meaningful to the system. By meaningful, I mean that the information is auto-encoded or auto-regressive. That is, meaningful information is information that sufficiently maximizes the match between the prior state and the information.

Here, we can then formulate the problem as one in which there exists a prior state, novel information, and the posterior state which is the cross product of the two.