Deep convolutional learning at ImageNet scale

GPU-trained convolutional networks substantially improved large-scale image classification.

Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton

AI topics

Explore related entries. Larger tags appear on more entries.

The contribution

The system commonly called AlexNet combined a deep convolutional architecture with effective training techniques and GPU computation. Its ImageNet results provided compelling evidence that learned visual representations could improve performance at scale.

What this does not establish

The result was a classification benchmark breakthrough, not proof that the system understood images in the full human sense.

Why this date?

The paper appeared at the 2012 neural information processing conference.

This entry follows the linked publication. Read the source and date conventions.

The original work

ImageNet Classification with Deep Convolutional Neural Networks

Authors: Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton
Publication: Advances in Neural Information Processing Systems
Source type: Conference paper

Original conference proceedings

The links above support the description and dating of this entry. Full text may be open or publisher-restricted.

Cited By

Atlas entries whose reference lists cite this work or the publication version noted below.

Dropout: A Simple Way to Prevent Neural Networks from Overfitting (2014)
Listed in the source bibliography. Check the reference list.
Generative Adversarial Networks (2014)
Listed in the source bibliography. Check the reference list.
Deep Residual Learning for Image Recognition (2015)
Listed in the source bibliography. Check the reference list.
Human-level control through deep reinforcement learning (2015)
Listed in the source bibliography. Check the reference list.
Mastering the game of Go with deep neural networks and tree search (2016)
Listed in the source bibliography. Check the reference list.
TensorFlow: A system for large-scale machine learning (2016)
Listed in the source bibliography. Check the reference list.
Language Models are Unsupervised Multitask Learners (2019)
Listed in the source bibliography. Check the reference list.
Scaling Laws for Neural Language Models (2020)
Listed in the source bibliography. Check the reference list.
Learning Transferable Visual Models From Natural Language Supervision (2021)
Listed in the source bibliography. Check the reference list.

Citations

Works in the Atlas cited by this entry.

ImageNet: A Large-Scale Hierarchical Image Database (2009)
Listed in the source bibliography. Check the reference list.

Bibliography coverage: Reference section indexed. Open the source.

Raw extraction retains OCR errors, ligatures, page headers and column-order artifacts. Entries have not all been normalized or individually verified.

Read the indexed bibliography

Reference section 1 (PDF pages 9, 10)

[1] R.M. Bell and Y. Koren. Lessons from the netﬂix prize challenge. ACM SIGKDD Explorations Newsletter, 9(2):75–79, 2007. [2] A. Berg, J. Deng, and L. Fei-Fei. Large scale visual recognition challenge 2010. www.imagenet.org/challenges. 2010. [3] L. Breiman. Random forests. Machine learning, 45(1):5–32, 2001. [4] D. Cires¸an, U. Meier, and J. Schmidhuber. Multi-column deep neural networks for image classiﬁcation. Arxiv preprint arXiv:1202.2745, 2012. [5] D.C. Cires¸an, U. Meier, J. Masci, L.M. Gambardella, and J. Schmidhuber. High-performance neural networks for visual object classiﬁcation. Arxiv preprint arXiv:1102.0183, 2011. [6] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009. [7] J. Deng, A. Berg, S. Satheesh, H. Su, A. Khosla, and L. Fei-Fei. ILSVRC-2012, 2012. URL http://www.image-net.org/challenges/LSVRC/2012/. [8] L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Image Understanding, 106(1):59–70, 2007. [9] G. Grifﬁn, A. Holub, and P. Perona. Caltech-256 object category dataset. Technical Report 7694, California Institute of Technology, 2007. URL http://authors.library.caltech.edu/7694. [10] G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R.R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012. [11] K. Jarrett, K. Kavukcuoglu, M. A. Ranzato, and Y. LeCun. What is the best multi-stage architecture for object recognition? In International Conference on Computer Vision, pages 2146–2153. IEEE, 2009. [12] A. Krizhevsky. Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto, 2009. [13] A. Krizhevsky. Convolutional deep belief networks on cifar-10. Unpublished manuscript, 2010. [14] A. Krizhevsky and G.E. Hinton. Using very deep autoencoders for content-based image retrieval. In ESANN, 2011. [15] Y. Le Cun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, L.D. Jackel, et al. Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems, 1990. [16] Y. LeCun, F.J. Huang, and L. Bottou. Learning methods for generic object recognition with invariance to pose and lighting. In Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, volume 2, pages II–97. IEEE, 2004. [17] Y. LeCun, K. Kavukcuoglu, and C. Farabet. Convolutional networks and applications in vision. In Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on, pages 253–256. IEEE, 2010. [18] H. Lee, R. Grosse, R. Ranganath, and A.Y. Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 609–616. ACM, 2009. [19] T. Mensink, J. Verbeek, F. Perronnin, and G. Csurka. Metric Learning for Large Scale Image Classiﬁcation: Generalizing to New Classes at Near-Zero Cost. In ECCV - European Conference on Computer Vision, Florence, Italy, October 2012. [20] V. Nair and G. E. Hinton. Rectiﬁed linear units improve restricted boltzmann machines. In Proc. 27th International Conference on Machine Learning, 2010. [21] N. Pinto, D.D. Cox, and J.J. DiCarlo. Why is real-world visual object recognition hard? PLoS computational biology, 4(1):e27, 2008. [22] N. Pinto, D. Doukhan, J.J. DiCarlo, and D.D. Cox. A high-throughput screening approach to discovering good forms of biologically inspired visual representation. PLoS computational biology, 5(11):e1000579, 2009. [23] B.C. Russell, A. Torralba, K.P. Murphy, and W.T. Freeman. Labelme: a database and web-based tool for image annotation. International journal of computer vision, 77(1):157–173, 2008. [24] J. Sánchez and F. Perronnin. High-dimensional signature compression for large-scale image classiﬁcation. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 1665–1672. IEEE, 2011. [25] P.Y. Simard, D. Steinkraus, and J.C. Platt. Best practices for convolutional neural networks applied to visual document analysis. In Proceedings of the Seventh International Conference on Document Analysis and Recognition, volume 2, pages 958–962, 2003. [26] S.C. Turaga, J.F. Murray, V. Jain, F. Roth, M. Helmstaedter, K. Briggman, W. Denk, and H.S. Seung. Convolutional networks can learn to generate afﬁnity graphs for image segmentation. Neural Computation, 22(2):511–538, 2010. 9

Download the bibliography index · Download Atlas citation links

These links cover verified matches within the Atlas. They are not total scholarly citation counts. A citation alone does not establish that the later system implements the same method.

AI topics

The contribution

What this does not establish

Why this date?

Comments