IJCNN’2003, July 20-24, 2003, Portland, Oregon

Tutorial Proposal

1. Title: Neuropercolation: Dynamical Memory Neural Networks –Biological Systems and Computer Implementations

2. By-line: Neuropercolation

3. Tutors: Robert Kozma and Walter J Freeman

4. Affiliation: Institute for Intelligent Systems, University of Memphis, TN, and

Division of Neurobiology, University of California at Berkeley

5. Abstract

  • Conventional digital computers store information encoded in strings of binary digits. We propose an alternative approach of pattern-based computing, in which information is stored in the form of spatial patterns of amplitude modulation of an aperiodic oscillatory carrier wave. More precisely, information is encoded by spatial patterns of 'synaptic' weights of connections that couple nonlinear processing elements. Our proposed approach of oscillating memory devices is strongly biologically motivated. It is based on the observation that sensory information processing in the central nervous system is realized via collective oscillations of sparsely but globally interacting neuronal populations. Capacities of animals for recognizing sensory signals using these robust biological mechanisms far surpass any existing man-made devices. This approach provides a novel view on neural networks. It includes as special cases other models, including deterministic cellular automata, such as Conway's Game of Life, Chua’s cellular neural networks, as well as thermodynamic models like the Ising model and Hopfield’s neural network arrays.

The following issues will be dealt with:

  • Biologically inspired K-models: We introduce nonconvergent dynamical systems that construct attractor landscapes, in which each attractor represents a class of discriminable input, and the basin provides for generalization over variations in form of class members in the presence of noise. We show how endogenous, biologically modeled noise is essential for the robust performance in digital computer embodiment. A practical methodology of encoding input data in aperiodic oscillations is described. We demonstrate the feasibility of spatial pattern encoding on a number of difficult classification problems.
  • Neuropercolation approach: The main impediment to further progress is the lack of theory on how large masses of spatially distributed nonlinear processing elements operate dynamically when driven by noise. In the neuropercolation approach, we overcome this problem by describing the dynamical memories as random cellular automata. The family of random cellular automata thus introduced is much richer than the family of standard bootstrap percolations. We describe how first-order phase transitions can be generated in a noisy system and how phase transition become helpful in generating a robust memory.
  • Biological and digital computer embodiment of dynamical memory NNs: The dynamical memory NNs result in a breakthrough in understanding the unity of perception and intentional action in animal behavior. They also give guidance for implementation of these principles in autonomous robotic systems. From the theoretical viewpoint, our model helps to understand the role of phase transitions in biological and artificial systems. It also serves as a basis of a novel computational device. Issues related to hardware implementation of these principles in micro-chips and nano-devices are also addressed.

6. References:

Freeman, W.J. (2000) How Brains Make Up Their Minds, Columbia Univ. Press.

Freeman, WJ (1992) Tutorial in Neurobiology: From Single Neurons to Brain Chaos. International Journal of Bifurcation and Chaos 2: 451-482.

Kozma, R., Freeman, W.J. (2001). "Chaotic Resonance - Methods and Applications for Robust Classification of Noisy and Variable Patterns," Int. J. Bifurcation & Chaos, Vol. 11, No. 6, pp. 1607-1629.

Freeman, W.J., Kozma, R., Werbos, P.J. (2001) "Biocomplexity - Adaptive behavior in complex stochastic systems," BioSystems, 59, pp. 109-123.