Recurrent state learns a grammar

A simple recurrent network learned temporal context sufficient to recognize a studied finite-state grammar.

Axel Cleeremans, David Servan-Schreiber and James L. McClelland

AI topics

Explore related entries. Larger tags appear on more entries.

The contribution

The authors trained a network to predict successive elements in sequences generated by a grammar. Feedback from its prior hidden activity let it distinguish contexts that could not be identified from the current symbol alone. The study connected continuous neural representations with the discrete states of an automaton.

What this does not establish

Learning the studied grammars does not establish unrestricted natural-language understanding or success on every formal language. The result concerns particular networks, training data, and tasks.

Why this date?

1989 dates the Neural Computation article. LSTM cites the paper as a study of finite-state automata and simple recurrent networks.

This entry follows the linked publication. Read the source and date conventions.

The original work

Finite State Automata and Simple Recurrent Networks

Authors: Axel Cleeremans, David Servan-Schreiber and James L. McClelland
Publication: Neural Computation, 1 (3), 372-381
Source type: Journal article

The links above support the description and dating of this entry. Full text may be open or publisher-restricted.

Cited By

Atlas entries whose reference lists cite this work or the publication version noted below.

Long Short-Term Memory (1997)
Cites Cleeremans, Servan-Schreiber and McClelland (1989). Check the reference list.

Citations

Works in the Atlas cited by this entry.

Learning representations by back-propagating errors (1986)
Cites Rumelhart, Hinton and Williams (1986), Nature 323, 533-536, using a variant title. The publisher DOI resolves to the original Atlas paper. Check the reference list.
Finding Structure in Time (1988)
Cites the 1988 technical report. The extracted report number reads 9901; the Atlas uses the independently verified report number 8801. Check the reference list.

Bibliography coverage: Reference section indexed. Open the source.

Reference text retains scan, spelling, line-break and column-order artifacts. Matching does not rely on publication year alone.

Read the indexed bibliography

Bibliography (PDF pages 10)

Elman, J.L. 1988. Finding Structure in Time. CRL Tech. Rep. 9901. Center for Research in Language, University of California, San Diego, CA. McClelland, J.L. 1988. The case for interactionism in language processing. In Attention and Performaiice XII, M. Coltheart, ed. Erlbaum, London. Reber, A.S. 1967. Implicit learning of artificial grammars. I. Verbal Leartiing Verbal Behau. 5, 855-863. Rumelhart, D.E., Hinton, G.E., and Williams, R.J. 1986. Learning internal rep- resentations by backpropagating errors. Nature (London) 323,533-536. Sejnowski,T.J.,and Rosenberg, C. 1986. NETtalk: A Parallel Network That Learns to Read Aloud. Tech. Rep. JHU-EECS-86-01,Johns Hopkins University. Servan-Schreiber, D., Cleeremans, A,, and McClelland, J.L. 1988. Learnitig Se- quential Structure in Sirnple Recirrretit Netzuorks. Tech. Rep. CMU-CS-183, Carnegie-Mellon University. Williams, R.J., and Zipser, D. 1988. A Learning Algorithm for Cotititiually Running Fully Recurrent Neural Netzuorks. ICS Tech. Rep. 8805. Institute for Cognitive Science, University of California, San Diego, CA.

Download the bibliography index · Download Atlas citation links

These links cover verified matches within the Atlas. They are not total scholarly citation counts. A citation alone does not establish that the later system implements the same method.

AI topics

The contribution

What this does not establish

Why this date?

Comments