Learning while a sequence unfolds

Dynamic networks extended error-based learning from static patterns to time-varying streams.

A. J. Robinson and F. Fallside

AI topics

Explore related entries. Larger tags appear on more entries.

The contribution

Robinson and Fallside explored architectures for learning finite sequences as well as streams whose length is not known in advance. Their report developed dynamic state representations, examples including speech coding, and a utility signal for training behavior. It belongs to the early work on learning in systems whose current output depends on prior activity.

What this does not establish

The report analyzes particular architectures and examples. Its claims do not establish unlimited memory or reliable learning across arbitrarily long sequences.

Why this date?

The report is dated 4 November 1987 and identified as CUED/F-INFENG/TR.1. LSTM cites this technical report, not Robinson's later doctoral thesis.

This entry follows the linked publication. Read the source and date conventions.

The original work

The Utility Driven Dynamic Error Propagation Network

Authors: A. J. Robinson and F. Fallside
Publication: Cambridge University Engineering Department, CUED/F-INFENG/TR.1
Source type: Technical report

The links above support the description and dating of this entry. Full text may be open or publisher-restricted.

Cited By

Atlas entries whose reference lists cite this work or the publication version noted below.

Long Short-Term Memory (1997)
The bibliography cites Robinson and Fallside (1987), CUED/F-INFENG/TR.1. Check the reference list.

Citations

Works in the Atlas cited by this entry.

The perceptron: A probabilistic model for information storage and organization in the brain (1958)
Listed in the source bibliography. Check the reference list.
Perceptrons: An Introduction to Computational Geometry (1969)
Listed in the source bibliography. Check the reference list.
Neural networks and physical systems with emergent collective computational abilities (1982)
Listed in the source bibliography. Check the reference list.

Bibliography coverage: Reference section indexed. Open the source.

Reference text retains scan, spelling, line-break and column-order artifacts. Matching does not rely on publication year alone.

Read the indexed bibliography

Bibliography (PDF pages 23)

Ackley, D. H., Hinton, G. E., and Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Journal of Cognitive Science, 9, 147-169. Cottrell, G. W., Munro, P., and Zipser, D. (Febuary 1986). Image Compression by Back Propagation: An Example of Existential Programming. ICS Report 8702, Institute for Cognitive Science, University of California, San Diego. . Dennett, D. C. (1984). Elbow Room: The varieties of free will worth wanting. The Clarendon Press, Oxford. Elman, J. L. and Zipser, D. (1987). Learning the Hidden Structure of Speech. ICS Re- port 8701, University of California, San Diego. Hofstader, D. R. (1979). Godel, Escher, Bach: An eternal golden braid. The Harvester Press, Hassocks, Sussex. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computationalabilities. Proceedings of the National Academy of Science U.S.A., 79, 2554— 2558. Jacobs, O. L. R. (1974). Introduction to Control Theory. Clarendon Press, Oxford. Jordan, M. I. (May 1986). Serial Order: A Parallel Distributed Processing Approach. ICS Report 8604, Institute for Cognitive Science, University of California, San Diego. Kuffler, S. W., Nicholls, J. G., and Martin, A. R. (1984). From Neuron to Brain: A Cellular Approach to the Function of the Nervous System. Sinauer Associates Inc., Sunderland, MA, second edition. Lindsay, P. H. and Norman, D. A. (1977). Human Information Processing: An Introduction to Psychology. Academic Press, Inc., Orlando, Florida, second edition. Minsky, M. and Papert, S. (1969). Perceptrons: An Introduction to Computational Geometry. MIT Press, Cambridge, MA. Pearlmutter, B. A. and Hinton, G. E. (1986). G-maximization: An unsupervised learning procedure for discovering regularities. In Proceedings of the Conference on ‘Neural Networks for Computing’, American Institute of Physics. Poggio, T. and Koch, C. (1987). Synapses that compute motion. Scientific American, May, 42-48. Prager, R. W., Harrison, T. D., and Fallside, F. (1986). Boltzmann machines for speech recognition. Computer Speech and Language, 1, 3-27. Robinson, A. J. (1986). Speech Recognition with Associative Networks. M.Phil Computer Speech and Language Processing thesis , Cambridge University Engineering Depart- ment. Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organisation in the brain. Psychological Review, 65, 386-408. Rosenblatt, F. (1962). Principles of Neurodynamics. Spartan, New York. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations. (eds. D. E. Rumelhart and J. L. McClel- land), Bradford Books/MIT Press, Cambridge, MA. Rumelhart, D. E. and McClelland, J. L. (1986). Parallel Distributed Processing: Explorations im the Microstructure of Cognition. Vol. 1: Foundations. MIT Press, Cambridge, MA. Tank, D. W. and Hopfield, J. J. (1987). Neural computation by concentrating information in time. Proceedings of the National Academy of Science U.S.A.. 84, 1896-1900. Watrous, R. L., Shastri, L., and Waibel. A. H. (1987). Learned phonetic discrimina- tion using connectionist networks. In Proceedings of the European Conference on Speech Technology (eds. J. Laver and M. A. Jack), CEP Consultants Ltd, Edinburgh.

Download the bibliography index · Download Atlas citation links

These links cover verified matches within the Atlas. They are not total scholarly citation counts. A citation alone does not establish that the later system implements the same method.

AI topics

The contribution

What this does not establish

Why this date?

Comments