Convolutional networks read documents

Convolutional networks and gradient-based training connected representation learning with document recognition.

Yann LeCun, Léon Bottou, Yoshua Bengio and Patrick Haffner

AI topics

Explore related entries. Larger tags appear on more entries.

The contribution

The paper brought together convolutional architectures, learning methods, and document-recognition systems. It showed how trainable components could be combined into systems for recognizing characters and interpreting documents, including bank-check applications.

What this does not establish

The work concerned specific recognition tasks and systems. It did not introduce all convolutional ideas or establish general visual understanding.

Why this date?

The paper appeared in November 1998.

This entry follows the linked publication. Read the source and date conventions.

The original work

Gradient-Based Learning Applied to Document Recognition

Authors: Yann LeCun, Léon Bottou, Yoshua Bengio and Patrick Haffner
Publication: Proceedings of the IEEE, 86 (11), 2278–2324
Source type: Journal article

The links above support the description and dating of this entry. Full text may be open or publisher-restricted.

Cited By

Atlas entries whose reference lists cite this work or the publication version noted below.

A Fast Learning Algorithm for Deep Belief Nets (2006)
Listed in the source bibliography. Check the reference list.
Generative Adversarial Networks (2014)
Listed in the source bibliography. Check the reference list.
Human-level control through deep reinforcement learning (2015)
Listed in the source bibliography. Check the reference list.

Citations

Works in the Atlas cited by this entry.

No references to other Atlas entries have been recorded for this work yet.

Bibliography coverage: Reference section indexed. Open the source.

Raw extraction retains OCR errors, ligatures, page headers and column-order artifacts. Entries have not all been normalized or individually verified.

Readable original article obtained from a public mirror; the author-hosted PDF had a broken text encoding. Number-to-entry alignment is unreliable in parts of this extraction.

Read the indexed bibliography

Reference section 1 (PDF pages 43, 44, 45, 46)

[1] R. O. Duda and P. E. Hart, Pattern Classiﬁcation and Scene Analysis. New York: Wiley, 1973. [2] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Backpropagation applied to handwritten zip code recognition,” Neural Computation, vol. 1, no. 4, pp. 541–551, Winter 1989. [3] S. Seung, H. Sompolinsky, and N. Tishby, “Statistical mechanics of learning from examples,” Phys. Rev. A, vol. 45, pp. 6056–6091, 1992. [4] V. N. Vapnik, E. Levin, and Y. LeCun, “Measuring the vcdimension of a learning machine,” Neural Computation, vol. 6, no. 5, pp. 851–876, 1994. [5] C. Cortes, L. Jackel, S. Solla, V. N. Vapnik, and J. Denker, “Learning curves: Asymptotic values and rate of convergence,” PROCEEDINGS OF THE IEEE, VOL. 86, NO. 11, NOVEMBER 1998 in Advances in Neural Information Processing Systems 6, J. D. Cowan, G. Tesauro, and J. Alspector, Eds. San Mateo, CA: Morgan Kaufmann, 1994, pp. 327–334. [6] V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer, 1995. [7] , Statistical Learning Theory. New York: Wiley, 1998. [8] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes: The Art of Scientiﬁc Computing. Cambridge, UK: Cambridge Univ., 1986. [9] S. I. Amari, “A theory of adaptive pattern classiﬁers,” IEEE Trans. Electron. Comput., vol. EC-16, pp. 299–307, 1967. [10] Y. Tsypkin, Adaptation and Learning in Automatic Systems New York: Academic, 1971. [11] , Foundations of the Theory of Learning Systems. New York: Academic, 1973. [12] M. Minsky and O. Selfridge, “Learning in random nets,” in Proc. 4th London Symp. Information Theory, pp. 335–347, 1961. [13] D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, “A learning algorithm for Boltzmann machines,” Cognitive Sci., vol. 9, pp. 147–169, 1985. [14] G. E. Hinton and T. J. Sejnowski, “Learning and relearning in Boltzmann machines,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT, 1986. [15] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learn- ing internal representations by error propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. I. Cambridge, MA: Bradford Books, 1986, pp. 318–362, [16] A. E. Bryson, Jr. and Y.-C. Ho, Applied Optimal Control. London, UK: Blaisdell, 1969. [17] Y. LeCun, “A learning scheme for asymmetric threshold networks,” in Proc. Cognitiva ’85, Paris, France, 1985, pp. 599–604. [18] , “Learning processes in an asymmetric threshold net- work,” in Disordered Systems and Biological Organization, E. Bienenstock, F. Fogelman-Soulie¨, and G. Weisbuch, Eds. Les Houches, France: Springer-Verlag, 1986, pp. 233–240. [19] D. B. Parker, “Learning-logic,” Sloan School Manage., MIT, Cambridge, MA, Tech. Rep., TR-47, Apr. 1985. [20] Y. LeCun, Mode´les Connexionnistes de l’Apprentissage (Con- nectionist Learning Models), Ph.D. dissertation, Universite´ P. et M. Curie (Paris 6), June 1987. [21] , “A theoretical framework for back-propagation,” in Proc. 1988 Connectionist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, Eds. Pittsburgh, PA: CMU, Morgan Kaufmann, 1988, pp. 21–28. [22] L. Bottou and P. Gallinari, “A framework for the cooperation of learning algorithms,” in Advances in Neural Information Processing Systems, vol. 3, D. Touretzky and R. Lippmann, Eds. Denver, CO: Morgan Kaufmann, 1991. [23] C. Y. Suen, C. Nadal, R. Legault, T. A. Mai, and L. Lam, “Computer recognition of unconstrained handwritten numerals,” Proc. IEEE, vol. 80, pp. 1162–1180, July 1992. [24] S. N. Srihari, “High-performance reading machines,” Proc. IEEE., vol. 80, pp. 1120–1132, July 1992. [25] Y. LeCun, L. D. Jackel, B. Boser, J. S. Denker, H. P. Graf, I. Guyon, D. Henderson, R. E. Howard, and W. Hubbard, “Handwritten digit recognition: Applications of neural net chips and automatic learning,” IEEE Trans. Commun., vol. 37, pp. 41–46, Nov. 1989. [26] J. Keeler, D. Rumelhart, and W. K. Leow, “Integrated seg- mentation and recognition of hand-printed numerals,” in Neural Information Processing Systems, R. P. Lippmann, J. M. Moody, and D. S. Touretzky, Eds. San Mateo, CA: Morgan Kaufmann, vol. 3, pp. 557–563, 1991. [27] O. Matan, C. J. C. Burges, Y. LeCun, and J. S. Denker, “Multi- digit recognition using a space displacement neural network,” vol. 4, in Neural Information Processing Systems, J. M. Moody, S. J. Hanson, and R. P. Lippman, Eds. San Mateo, CA: Morgan Kaufmann, 1992. [28] L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proc. IEEE, vol. 77, pp. 257–286, Feb. 1989. [29] H. A. Bourland and N. Morgan, Connectionist Speech Recog- nition: A Hybrid Approach. Boston: Kluwer, 1994. [30] D. H. Hubel and T. N. Wiesel, “Receptive ﬁelds, binocular in- teraction, and functional architecture in the cat’s visual cortex,” J. Physiology (London), vol. 160, pp. 106–154, 1962. [31] K. Fukushima, “Cognition: A self-organizing multilayered neu- ral network,” Biological Cybern., vol. 20, pp. 121–136, 1975. [32] K. Fukushima and S. Miyake, “Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position,” Pattern Recognit., vol. 15, no. 6, pp. 455–469, Nov. 1982. [33] M. C. Mozer, The Perception of Multiple Objects: A Con- nectionist Approach. Cambridge, MA: MIT-Bradford Books, 1991. [34] Y. LeCun, “Generalization and network design strategies,” in Connectionism in Perspective, R. Pfeifer, Z. Schreter, F. Fogelman, and L. Steels, Eds. Zurich, Switzerland: Elsevier, 1989. [35] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Handwritten digit recognition with a back-propagation network,” in Advances in Neural Information Processing Systems 2 (NIPS’89), David Touretzky, Ed. Denver, CO: Morgan Kaufmann, 1990. [36] G. L. Martin, “Centered-object integrated segmentation and recognition of overlapping hand-printed characters,” Neural Computation, vol. 5, no. 3, pp. 419–429, 1993. [37] J. Wang and J. Jean, “Multi-resolution neural networks for omnifont character recognition,” in Proc. Int. Conf. Neural Networks, vol. III, 1993, pp. 1588–1593. [38] Y. Bengio, Y. LeCun, C. Nohl, and C. Burges, “Lerec: A NN/HMM hybrid for on-line handwriting recognition,” Neural Computation, vol. 7, no. 5, 1995. [39] S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, “Face recognition: A convolutional neural network approach,” IEEE Trans. Neural Networks, vol. 8, pp. 98–113, Jan. 1997. [40] K. J. Lang and G. E. Hinton, “A time delay neural network architecture for speech recognition,” Carnegie-Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-88-152, 1988. [41] A. H. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. Lang, “Phoneme recognition using time-delay neural networks,” IEEE Trans. Acoustics, Speech, Signal Processing, vol. 37, pp. 328–339, Mar. 1989. [42] L. Bottou, F. Fogelman, P. Blanchet, and J. S. Lienard, “Speaker independent isolated digit recognition: Multilayer perceptron versus dynamic time warping,” Neural Networks, vol. 3, pp. 453–465, 1990. [43] P. Haffner and A. H. Waibel, “Time-delay neural networks embedding time alignment: A performance analysis,” in Proc. EUROSPEECH’91, 2nd Europ. Conf. Speech Communication and Technology, Genova, Italy. [44] I. Guyon, P. Albrecht, Y. LeCun, J. S. Denker, and W. Hubbard, “Design of a neural network character recognizer for a touch terminal,” Pattern Recognit., vol. 24, no. 2, pp. 105–119, 1991. [45] J. Bromley, J. W. Bentz, L. bottou, I. Guyon, Y. LeCun, C. Moore, E. Sa¨ckinger, and R. Shah, “Signature veriﬁcation using a siamese time delay neural network,” Int. J. Pattern Recognit. Artiﬁcial Intell., vol. 7, no. 4, pp. 669–687, Aug. 1993. [46] Y. LeCun, I. Kanter, and S. Solla, “Eigenvalues of covariance matrices: Application to neural-network learning,” Phys. Rev. Lett., vol. 66, no. 18, pp. 2396–2399, May 1991. [47] T. G. Dietterich and G. Bakiri, “Solving multiclass learning problems via error-correcting output codes,” J. Artiﬁcial Intell. Res., vol. 2, pp. 263–286, 1995. [48] L. R. Bahl, P. F. Brown, P. V. de Souza, and R. L. Mercer, “Maximum mutual information of hidden Markov model pa- rameters for speech recognition,” in Proc. Int. Conf. Acoustics, Speech, Signal Processing, 1986, pp. 49–52. [49] , “Speech recognition with continuous-parameter hidden Markov models,” Comput., Speech Language, vol. 2, pp. 219–234, 1987. [50] B. H. Juang and S. Katagiri, “Discriminative learning for minimum error classiﬁcation,” IEEE Trans. Acoustics, Speech, Signal Processing, vol. 40, pp. 3043–3054, Dec. 1992. [51] Y. LeCun, L. D. Jackel, L. Bottou, A. Brunot, C. Cortes, J. S. Denker, H. Drucker, I. Guyon, U. A. Muller, E. Sa¨ckinger, P. Simard, and V. N. Vapnik, “Comparison of learning algorithms for handwritten digit recognition,” in Int. Conf. Artiﬁcial Neural Networks, F. Fogelman and P. Gallinari, Eds. Paris: EC2 & Cie, 1995, pp. 53–60. [52] I. Guyon, I. Poujaud, L. Personnaz, G. Dreyfus, J. Denker, and Y. LeCun, “Comparing different neural net architectures for LECUN et al.: GRADIENT-BASED LEARNING APPLIED TO DOCUMENT RECOGNITION 2321 classifying handwritten digits,” in Proc. IEEE IJCNN, Washington, DC, vol. II, 1989, pp. 127–132,. [53] R. Ott, “Construction of quadratic polynomial classiﬁers,” in Proc. IEEE Int. Conf. Pattern Recognition, 1976, pp. 161–165. [54] J. Schu¨rmann, “A multifont word recognition system for postal address reading,” IEEE Trans. Comput., vol. C-27, pp. 721–732, Aug. 1978. [55] Y. Lee, “Handwritten digit recognition using k-nearest neighbor, radial-basis functions, and backpropagation neural networks,” Neural Computation, vol. 3, no. 3, pp. 440–449, 1991. [56] D. Saad and S. A. Solla, “Dynamics of on-line gradient descent learning for multilayer neural networks,” in Advances in Neural Information Processing Systems, vol. 8, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds. Cambridge, MA: MIT, 1996, pp. 302–308. [57] G. Cybenko, “Approximation by superpositions of sigmoidal functions,” Math. Control, Signals, Syst., vol. 2, no. 4, pp. 303–314, 1989. [58] L. Bottou and V. N. Vapnik, “Local learning algorithms,” Neural Computation, vol. 4, no. 6, pp. 888–900, 1992. [59] R. E. Schapire, “The strength of weak learnability,” Machine Learning, vol. 5, no. 2, pp. 197–227, 1990. [60] H. Drucker, R. Schapire, and P. Simard, “Improving performance inneural networks using a boosting algorithm,” in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993, pp. 42–49. [61] P. Simard, Y. LeCun, and J. Denker, “Efﬁcient pattern recognition using a new transformation distance,” in Advances in Neural Information Processing Systems, vol. 5, S. Hanson, J. Cowan, and L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993. [62] B. Boser, I. Guyon, and V. Vapnik, “A training algorithm for optimal margin classiﬁers,” in Proc. 5th Annu. Workshop Computational Learning Theory, vol. 5, 1992, pp. 144–152. [63] C. J. C. Burges and B. Schoelkopf, “Improving the accuracy and speed of support vector machines,” in Advances in Neural Information Processing Systems 9, M. Jordan, M. Mozer, and T. Petsche, Eds. Cambridge, MA: MIT, 1997. [64] E. Sa¨ckinger, B. Boser, J. Bromley, Y. LeCun, and L. D. Jackel, “Application of the ANNA neural network chip to high-speed character recognition,” IEEE Trans. Neural Networks, vol. 3, no. 3, pp. 498–505, Mar. 1992. [65] J. S. Bridle, “Probabilistic interpretation of feedforward classiﬁcation networks outputs, with relationship to statistical pattern recognition,” in Neurocomputing, Algorithms, Architectures and Applications, F. Fogelman, J. Herault, and Y. Burnod, Eds. Les Arcs, France: Springer, 1989. [66] Y. LeCun, L. Bottou, and Y. Bengio, “Reading checks with graph transformer networks,” in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing. Munich, Germany, vol. 1, 1997, pp. 151–154,. [67] Y. Bengio, Neural Networks for Speech and Sequence Recognition. London, UK: International Thompson, 1996. [68] C. Burges, O. Matan, Y. LeCun, J. Denker, L. Jackel, C. Stenard, C. Nohl, and J. Ben, “Shortest path segmentation: A method for training a neural network to recognize character strings,” in Proc. Int. Joint Conf. Neural Networks, Baltimore, MD, vol. 3, 1992, pp. 165–172. [69] T. M. Breuel, “A system for the off-line recognition of handwritten text,” in Proc. IEEE ICPR’94, Jerusalem, pp. 129–134. [70] A. Viterbi, “Error bounds for convolutional codes and an asymptotically optimum decoding algorithm,” IEEE Trans. Inform. Theory, vol. 15, pp. 260–269, Apr. 1967. [71] R. P. Lippmann and B. Gold, “Neural-net classiﬁers useful for speech recognition,” in Proc. IEEE 1st Int. Conf. Neural Networks, San Diego, CA, June 1987, pp. 417–422. [72] H. Sakoe, R. Isotani, K. Yoshida, K. Iso, and T. Watanabe, “Speaker-independent word recognition using dynamic programming neural networks,” in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Glasgow, 1989, pp. 29–32. [73] J. S. Bridle, “Alphanets: A recurrent ‘neural’ network architecture with a hidden Markov model interpretation,” Speech Commun., vol. 9, no. 1, pp. 83–92, 1990. [74] M. A. Franzini, K. F. Lee, and A. H. Waibel, “Connectionist viterbi training: A new hybrid method for continuous speech recognition,” in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Albuquerque, NM, 1990, pp. 425–428. 2322 [75] L. T. Niles and H. F. Silverman, “Combining hidden Markov models and neural network classiﬁers,” in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Albuquerque, NM, 1990, pp. 417–420. [76] X. Driancourt and L. Bottou, “MLP, LVQ and DP: Comparison & cooperation,” in Proc. Int. Joint Conf. Neural Networks, Seattle, WA, vol. 2, 1991, pp. 815–819. [77] Y. Bengio, R. De Mori, G. Flammia, and R. Kompe, “Global optimization of a neural network-hidden Markov model hybrid,” IEEE Trans. Neural Networks, vol. 3, pp. 252–259, March 1992. [78] P. Haffner and A. H. Waibel, “Multi-state time-delay neural networks for continuous speech recognition,” vol. 4, in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, pp. 579–588, 1992. [79] Y. Bengio, P. Simard, and P. Frasconi, “Learning long-term dependencies with gradient descent is difﬁcult,” IEEE Trans. Neural Networks, vol. 5, no. 2, pp. 157–166, Mar. 1994. [80] T. Kohonen, G. Barna, and R. Chrisley, “Statistical pattern recognition with neural network: Benchmarking studies,” in Proc. IEEE 2nd Int. Conf. Neural Networks, San Diego, CA, vol. 1, 1988, pp. 61–68. [81] P. Haffner, “Connectionist speech recognition with a global MMI algorithm,” in Proc. EUROSPEECH’93, 3rd Europ. Conf. Speech Communication and Technology, Berlin, pp. 1929–1932. [82] J. S. Denker and C. J. Burges, “Image segmentation and recognition,” in The Mathematics of Induction. Reading, MA: Addison Wesley, 1995. [83] L. Bottou, Une Approche the´orique de l’Apprentissage Connexionniste: Applications a` la Reconnaissance de la Parole, Ph.D. dissertation, Univ. Paris XI, France, 1991. [84] M. Rahim, Y. Bengio, and Y. LeCun, “Disriminative feature and model design for automatic speech recognition,” in Proc. Eurospeech, Rhodes, Greece, 1997, pp. 75–78. [85] U. Bodenhausen, S. Manke, and A. Waibel, “Connectionist architectural learning for high performance character and speech recognition,” in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Minneapolis, MN, vol. 1, 1993, pp. 625–628. [86] F. Pereira, M. Riley, and R. Sproat, “Weighted rational transductions and their application to human language processing,” in ARPA Natural Language Processing Workshop, 1994. [87] M. Lades, J. C. Vorbru¨ggen, J. Buhmann, and C. von der Malsburg, “Distortion invariant object recognition in the dynamic link architecture,” IEEE Trans. Comput., vol. 42, pp. 300–311, March 1993. [88] B. Boser, E. Sa¨ckinger, J. Bromley, Y. LeCun, and L. Jackel, “An analog neural network processor with programmable topology,” IEEE J. Solid-State Circuits, vol. 26, pp. 2017–2025, Dec. 1991. [89] M. Schenkel, H. Weissman, I. Guyon, C. Nohl, and D. Henderson, “Recognition-based segmentation of on-line hand-printed words,” in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. Denver, CO: Morgan Kaufmann, 1993, pp. 723–730. [90] C. Dugust, L. Devillers, and X. Aubert, “Combining TDNN and HMM in a hybrid system for improved continuous-speech recognition,” IEEE Trans. Speech Audio Processing, vol. 2, pp. 217–224, Jan. 1994. [91] O. Matan, H. S. Baird, J. Bromley, C. J. C. Burges, J. S. Denker, L. D. Jackel, Y. LeCun, E. P. D. Pednault, W. Satterﬁeld, C. E. Stenard, and T. J. Thompson, “Reading handwritten digits: A ZIP code recognition system,” IEEE Trans. Comput., vol. 25, no. 7, pp. 59–63, July 1992. [92] Y. Bengio and Y. LeCun, “Word normalization for on-line handwritten word recognition,” in Proc. IEEE Int. Conf. Pattern Recognition, Jerusalem, 1994. [93] R. Vaillant, C. Monrocq, and Y. LeCun, “Original approach for the localization of objects in images,” Proc. Inst. Elect. Eng., vol. 141, no. 4, pp. 245–250, Aug. 1994. [94] R. Wolf and J. Platt, “Postal address block location using a convolutional locator network,” in Advances in Neural Information Processing Systems 6, J. D. Cowan, G. Tesauro, and J. Alspector, Eds. San Mateo, CA: Morgan Kaufmann, 1994, pp. 745–752. [95] S. Nowlan and J. Platt, “A convolutional neural network hand tracker,” in Advances in Neural Information Processing Systems 7, G. Tesauro, D. Touretzky, and T. Leen, Eds. San Mateo, CA: Morgan Kaufmann, 1995, pp. 901–908. [96] H. A. Rowley, S. Baluja, and T. Kanade, “Neural network-based PROCEEDINGS OF THE IEEE, VOL. 86, NO. 11, NOVEMBER 1998 [97] [98] [99] [100] [101] [102] [103] [104] [105] [106] [107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] face detection,” in Proc. IEEE CVPR’96, pp. 203–208. E. Osuna, R. Freund, and F. Girosi, “Training support vector machines: An application to face detection,” in Proc. IEEE CVPR’96, pp. 130–136. H. Bourlard and C. J. Wellekens, “Links between Markov models and multilayer perceptrons,” in Advances in Neural Information Processing Systems, D. Touretzky, Ed. Denver: Morgan-Kaufmann, vol. 1, 1989, pp. 186–187. Y. Bengio, R. De Mori, G. Flammia, and R. Kompe, “Neural network—Gaussian mixture hybrid for speech recognition or density estimation,” in Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds. Denver, CO: Morgan Kaufmann, 1992, pp. 175–182. F. C. N. Pereira and M. Riley, “Speech recognition by composition of weighted ﬁnite automata,” in Finite-State Devices for Natural Lague Processing. Cambridge, MA: MIT, 1997. M. Mohri, “Finite-state transducers in language and speech processing,” Computational Linguistics, vol. 23, no. 2, pp. 269–311, 1997. I. Guyon, M. Schenkel, and J. Denker, “Overview and synthesis of on-line cursive handwriting recognition techniques,” in Handbook on Optical Character Recognition and Document Image Analysis, P. S. P. Wang and H. Bunke, Eds. New York: World Scientiﬁc, 1996. M. Mohri and M. Riley, “Weighted determinization and minimization for large vocabulary recognition,” in Proc. Eurospeech ’97, Rhodes, Greece, pp. 131–134. Y. Bengio and P. Frasconi, “An input/output HMM architecture,” in Advances in Neural Information Processing Systems, vol. 7, G. Tesauro, D. Touretzky, and T. Leen, Eds. Cambridge, MA: MIT, pp. 427–434, 1996. , “Input/output HMM’s for sequence processing,” IEEE Trans. Neural Networks, vol. 7, no. 5, pp. 1231–1249, 1996. M. Mohri, F. C. N. Pereira, and M. Riley, A Rational Design for a Weighted Finite-State Transducer Library (Lecture Notes in Computer Science). New York: Springer Verlag, 1997. M. Rahim, C. H. Lee, and B. H. Juang, “Discriminative utterance veriﬁcation for connected digits recognition,” IEEE Trans. Speech Audio Processing, vol. 5, pp. 266–277, 1997. M. Rahim, Y. Bengio, and Y. LeCun, “Discriminative feature and model design for automatic speech recognition,” in Proc. Eurospeech ’97, Rhodes, Greece. S. Bengio and Y. Bengio, “An EM algorithm for asynchronous input/output hidden Markov models,” in Proc. International Conference on Neural Information Processing, Hong-King, 1996, pp. 328–334. C. Tappert, C. Suen, and T. Wakahara, “The state of the art in on-line handwriting recognition,” IEEE Trans. Pattern Anal. Machine Intell., vol. 8, pp. 787–808, Dec. 1990. S. Manke and U. Bodenhausen, “A connectionist recognizer for on-line cursive handwriting recognition,” in Proc. Int. Conf. Acoustics, Speech, Signal Processing, Adelaide, vol. 2, 1994, pp. 633–636. M. Gilloux and M. Leroux, “Recognition of cursive script amounts on postal checks,” in Proc. Europ. Conf. Postal Technol., Nantes, France, June 1993, pp. 705–712. D. Guillevic and C. Y. Suen, “Cursive script recognition applied to the processing of bank checks,” in Proc. Int. Conf. Document Analysis Recognition, Montreal, Canada, Aug. 1995, pp. 11–14. L. Lam, C. Y. Suen, D. Guillevic, N. W. Strathy, M. Cheriet, K. Liu, and J. N. Said, “Automatic processing of information on checks,” in Int. Conf. Systems, Man, and Cybernetics, Vancouver, Canada, Oct. 1995, pp. 2353–2358. C. J. C. Burges, J. I. Ben, J. S. Denker, Y. LeCun, and C. R. Nohl, “Off line recognition of handwritten postal words using neural networks,” Int. J. Pattern Recognit. Artiﬁcial Intell., vol. 7, no. 4, p. 689, 1993. Y. LeCun, Y. Bengio, D. Henderson, A. Weisbuch, H. Weissman, and L. Jackel, “On-line handwriting recognition with neural networks: Spatial representation versus temporal representation,” in Proc. Int. Conf. Handwriting Drawing, 1993. U. Mu¨ller, A. Gunzinger, and W. Guggenbu¨hl, “Fast neural net simulation with a DSP processor array,” IEEE Trans. Neural Networks, vol. 6, pp. 203–213, Jan. 1995. R. Battiti, “First- and second-order methods for learning: Between steepest descent and Newton’s method,” Neural Computation, vol. 4, no. 2, pp. 141–166, 1992. A. H. Kramer and A. Sangiovanni-Vincentelli, “Efﬁcient par- [120] [121] allel learning algorithms for neural networks,” in Advances in Neural Information Processing Systems, vol. 1, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1988, pp. 40–48. M. Moller, Efﬁcient Training of Feed-Forward Neural Networks, Ph.D. dissertation, Aarhus Univ., Aarhus, Denmark, 1993. S. Becker and Y. LeCun, “Improving the convergence of back-propagation learning with second-order methods,” Univ. Toronto Connectionist Res. Group, Toronto, Ontario, Canada, Tech. Rep. CRG-TR-88-5, Sept. 1988.

Download the bibliography index · Download Atlas citation links

These links cover verified matches within the Atlas. They are not total scholarly citation counts. A citation alone does not establish that the later system implements the same method.

AI topics

The contribution

What this does not establish

Why this date?

Comments