Statistical Coding and Short-Term Synaptic Plasticity: A Scheme for Knowledge Representation in the Brain

  • Christoph von der Malsburg
  • Elie Bienenstock
Conference paper
Part of the NATO ASI Series book series (volume 20)


This work is a theoretical investigation of some consequences of the hypothesis that transmission efficacies of synapses in the Central Nervous System (CNS) undergo modification on a short time-scale. Short-term synaptic plasticity appears to be an almost necessary condition for the existence of activity states in the CNS which are stable for about 1 sec., the time-scale of psychological processes. It gives rise to joint “activity-and-connectivity” dynamics. This dynamics selects and stabilizes particular high-order statistical relationships in the timing of neuronal firing; at the same time, it selects and stabilizes particular connectivity patterns. In analogy to statistical mechanics, these stable states, the attractors of the dynamics, can be viewed as the minima of a hamiltonian, or cost function. It is found that these low-cost states, termed synaptic patterns, are topologically organized. Two important properties of synaptic patterns are demonstrated: (i) synaptic patterns can be “memorized” and later “retrieved”, and (ii) synaptic patterns have a tendency to assemble into compound patterns according to simple topological rules. A model of position-invariant and size-invariant pattern recognition based on these two properties is briefly described. It is suggested that the scheme of a synaptic pattern may be more adapted than the classical cell-assembly notion for explaining cognitive abilities such as generalization and categorization, which pertain to the notion of invariance.


Random Graph Synaptic Weight Label Pattern Fine Temporal Structure Synfire Chain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J. Von Neumann (1956) Probabilistic logic and the synthesis of reliable organisms from unreliable components. In: Automata Studies (C. E. Shannon and J. McCarthy, eds.), Princeton University Press, Princeton, NJ.Google Scholar
  2. 2.
    S. Winograd and J. D. Cowan (1963) Reliable Computation in the Presence of Noise. The MIT Press, Cambridge, MA.zbMATHGoogle Scholar
  3. 3.
    D. O. Hebb (1949) The Organization of Behavior. Wiley, New York.Google Scholar
  4. 4..
    W. A. Little (1974) The existence of persistent states in the brain. Math. Biosc. 19, 101–120.zbMATHCrossRefGoogle Scholar
  5. 5.
    J. J. Hopfield (1982) Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA. 79, 2554–2558.MathSciNetCrossRefGoogle Scholar
  6. 6.
    G. E. Hinton, T. J. Sejnowski, and D. H. Ackley (1984) Boltzmann machines: Constraint satisfaction networks that learn. Technical Report CMU-CS-84–119, Department of Computer Science, Carnegie-Mellon University, Pittsburgh PA.Google Scholar
  7. 7.
    M. Abeles (1982) Local Cortical Circuits. An Electrophysiological Study. (V. Brait-enberg, ed.), Springer-Verlag, Berlin.Google Scholar
  8. 8.
    C. von der Malsburg (in press) Am I thinking assemblies? In: Proceedings of the 1984 Trieste Meeting on Brain Theory. (G. Palm and A. Aertsen, eds.), Springer Verlag, Heidelberg.Google Scholar
  9. 9.
    R. Lorente de No (1938) Analysis of the activity of the chains of internuncial neurons. J. Neurophysiol. 1, 207–244.Google Scholar
  10. 10.
    C. von der Malsburg (1981) The correlation theory of brain function. Internal Report 81–2. Max-Planck Institute for Biophysical Chemistry, Department of Neurobiology, Göttingen, West-Germany.Google Scholar
  11. 11.
    J. Szentagothai and P. Erdi (preprint) Outline of a general brain theory. Hungarian Academy of Sciences, Budapest.Google Scholar
  12. 12.
    F. Crick (1982) Do dendritic spines twitch? Trends in Neurosci. 5, 44–46.CrossRefGoogle Scholar
  13. 13.
    W. Rall (1978) Dendritic spines and synaptic potency. In: Studies in Neurophysiology (R. Porter, ed.), pp. 203–209. Cambridge University Press.Google Scholar
  14. 14.
    C.Koch and T. Poggio (1983) A theoretical analysis of electrical properties of spines. Proc. R. Soc. B. 218, 455–477.CrossRefGoogle Scholar
  15. 15.
    J. P. Miller, W. Rall, and J. Rinzel (1985) Synaptic amplification by active membrane in dendritic spines. Brain Res. 325, 325–330.CrossRefGoogle Scholar
  16. 16.
    J. P. Changeux, A. Devillers-Thiéry, and P. Chemouilli (1984) Acetylcholine receptor: an allosteric protein. Science 225, 1335–1345.CrossRefGoogle Scholar
  17. 17.
    J. P. Changeux and T. Heidmann (in press) Allosteric receptors and molecular models of learning. In: New insights into synaptic function (G. Edelman, W.E. Gall, and W.M. Cowan, eds.), John Wiley Publishers, New York, NY.Google Scholar
  18. 18.
    N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller (1953) Equations of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1091.CrossRefGoogle Scholar
  19. 19.
    S. Kirkpatrick, C. D. Gelatt Jr, and M. P. Vecchi (1983) Optimization by simulated annealing. Science 220, 671–680.zbMATHMathSciNetCrossRefGoogle Scholar
  20. 20.
    G. Toulouse (1984) Frustration and disorder, new problems in statistical mechanics: Spin glasses in a historical perspective. In: Lecture Notes in Physics (J. van Hemmen and I. Morgenstern eds.), Springer Verlag, Heidelberg.Google Scholar
  21. 21.
    P. Erdös and A. Rényi (1960) On the evolution of random graphs. Publ. Math. Inst. Hung. Acad. Sci. 5, 17–61.zbMATHGoogle Scholar
  22. 22.
    E. Bienenstock (in press) Dynamics of central nervous system. In: Dynamics of Macrosystems. Proc. of a Symposium Held at the I.I.A.S.A., Laxenburg, Austria, Sept. 1984 (J. P. Aubin and K. Sigmund eds.).Google Scholar
  23. 23.
    D. J. Willshaw, and C. von der Malsburg (1976) How patterned neural connections can be set up by self-organization. Proc. R. Soc. Lond. B 194, 431–445.CrossRefGoogle Scholar
  24. 24.
    J. T. Schmidt, and D. L. Edwards (1983) Activity sharpens the map during regeneration of the retinotectal projection in goldfish. Brain Res. 269, 29–39.CrossRefGoogle Scholar
  25. 25.
    C. von der Malsburg (1985) Nervous structures with dynamical links. Ber. Bun-senges. Phys. Chem. 89, 703–710.Google Scholar
  26. 26.
    A. F. Haussier, and C. von der Malsburg (1983) Development of retinotopic projections: An analytical treatment. J. Theoret. Neurobiol. 2, 47–73.Google Scholar
  27. 27.
    D.N. Mastronarde (1983) Correlated firing of cat retinal ganglion cells, I and II.— Interactions between ganglion cells in cat retina. J. Neurophysiol. 49, 303–365.Google Scholar
  28. 28.
    D. W. Arnett (1978) Statistical dependence between neighbouring retinal ganglion cells in goldfish. Exp. Brain Res. 32, 49–53.CrossRefGoogle Scholar
  29. 29.
    Y. Frégnac (in press) Cellular mechanisms of epigenesis in cat visual cortex. In: Imprinting and Cortical Plasticity (J. P. Rauscheker, ed.), John Wiley Publishers, New-York, NY.Google Scholar
  30. 30.
    G. E Hinton, and K. Lang (in press) Shape recognition and illusory conjunctions. International Joint Conference on Artificial Intelligence.Google Scholar
  31. 31.
    J. Altmann, and H. J. P. Reitböck (1984) A fast correlation method for scale- and translation-invariant pattern recognition. IEEE Trans. PAMI-6, 46–57.Google Scholar
  32. 32.
    J. A. Feldman, and D. H. Ballard (1982) Connectionist models and their properties. Cognitive Science 6, 205–254.CrossRefGoogle Scholar
  33. 33.
    G. M. Edelman (1978) Group selection and phasic reentrant signalling: A theory of higher brain function. In: The Mindful Brain (G. M. Edelman and V. B. Mount-castle, eds.), The MIT Press, Cambridge, MA.Google Scholar
  34. 34.
    J. P. Changeux, T. Heidman, and P. Patte (1984) Learning by selection. In: The Biology of Learning. Proc. Dahlem Workshop, October 1983 (P. Marler and H. Terrace, eds.), Springer Verlag, Heidelberg.Google Scholar
  35. 35.
    C. von der Malsburg (1985) Algorithms, brain and organization. In: Dynamical Systems and Cellular Automata (J. Demongeot, E. Golès, and M. Tchuente, eds.), Academic Press, London.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1986

Authors and Affiliations

  • Christoph von der Malsburg
    • 1
  • Elie Bienenstock
    • 2
  1. 1.Abteilung für NeurobiologieMax-Planck-Institut für Biophysikalische ChemieGöttingenW. Germany
  2. 2.Laboratoire de Neurobiologie du DéveloppementUniversité de Paris-SudOrsay CedexFrance

Personalised recommendations