勾配消失問題 (Japanese Wikipedia)

Analysis of information sources in references of the Wikipedia article "勾配消失問題" in Japanese language version.

refsWebsite

Global rank Japanese rank

16doi.org

2^nd place

6^th place

9arxiv.org

69^th place

227^th place

6nih.gov

4^th place

24^th place

5worldcat.org

5^th place

19^th place

3harvard.edu

18^th place

107^th place

3acm.org

1,185^th place

2,667^th place

2sciencedirect.com

149^th place

558^th place

2idsia.ch

low place

2ieee.org

652^nd place

1,307^th place

1nowpublishers.com

low place

1titech.ac.jp

low place

1,204^th place

1toronto.edu

low place

1tue.nl

9,054^th place

low place

1mlr.press

low place

1uni-bonn.de

6,912^th place

low place

acm.org

dl.acm.org

Pascanu, Razvan; Mikolov, Tomas; Bengio, Yoshua (2013-06-16). “On the difficulty of training recurrent neural networks”. Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28 (Atlanta, GA, USA: JMLR.org): III–1310–III–1318. doi:10.5555/3042817.3043083.
Balduzzi, David; Frean, Marcus; Leary, Lennox; Lewis, J P; Ma, Kurt Wan-Duo; McWilliams, Brian (2017-08-06). “The shattered gradients problem: if resnets are the answer, then what is the question?”. Proceedings of the 34th International Conference on Machine Learning - Volume 70 (Sydney, NSW, Australia: JMLR.org): 344. doi:10.5555/3305381.3305417.
Ioffe, Sergey; Szegedy, Christian (2015-07-06). “Batch normalization: accelerating deep network training by reducing internal covariate shift”. Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 (Lille, France: JMLR.org): 449，455. doi:10.5555/3045118.3045167.

arxiv.org

Goh, Garrett B.; Hodas, Nathan O.; Vishnu, Abhinav (2017-06-15). “Deep learning for computational chemistry” (英語). Journal of Computational Chemistry 38 (16): 1291–1307. arXiv:1701.04503. Bibcode: 2017arXiv170104503G. doi:10.1002/jcc.24764. PMID 28272810.
Pascanu, Razvan; Mikolov, Tomas; Bengio, Yoshua (21 November 2012). "On the difficulty of training Recurrent Neural Networks". arXiv:1211.5063 [cs.LG]。
He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing; Sun, Jian (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE. pp. 770–778. arXiv:1512.03385. doi:10.1109/CVPR.2016.90. ISBN 978-1-4673-8851-1。
Zaeemzadeh, Alireza; Rahnavard, Nazanin; Shah, Mubarak (2021-11-01). “Norm-Preservation: Why Residual Networks Can Become Extremely Deep?”. IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (11): 3980–3990. arXiv:1805.07477. doi:10.1109/TPAMI.2020.2990339. ISSN 0162-8828.
Veit, Andreas; Wilber, Michael; Belongie, Serge (20 May 2016). "Residual Networks Behave Like Ensembles of Relatively Shallow Networks". arXiv:1605.06431 [cs.CV]。
Noel, Mathew Mithra; L, Arunkumar; Trivedi, Advait; Dutta, Praneet (4 September 2021). "Growing Cosine Unit: A Novel Oscillatory Activation Function That Can Speedup Training and Reduce Parameters in Convolutional Neural Networks". arXiv:2108.12943 [cs.LG]。
Ramachandran, Prajit; Barret, Zoph; Quoc, V. Le (16 October 2017). "Searching for Activation Functions". arXiv:1710.05941 [cs.NE]。
Noel, Matthew Mithra; Bharadwaj, Shubham; Muthiah-Nakarajan, Venkataraman; Dutta, Praneet; Amali, Geraldine Bessie (7 November 2021). "Biologically Inspired Oscillating Activation Functions Can Bridge the Performance Gap between Biological and Artificial Neurons". arXiv:2111.04020 [cs.NE]。
Schmidhuber, Jürgen (2015). “Deep learning in neural networks: An overview”. Neural Networks 61: 85–117. arXiv:1404.7828. doi:10.1016/j.neunet.2014.09.003. PMID 25462637.

doi.org

Okatani, Takayuki (2015). “On Deep Learning”. Journal of the Robotics Society of Japan 33 (2): 92–96. doi:10.7210/jrsj.33.92. ISSN 0289-1824. https://doi.org/10.7210/jrsj.33.92.
Schmidhuber, Jürgen (2015-01-01). “Deep learning in neural networks: An overview” (英語). Neural Networks 61: 90-91,93-94. doi:10.1016/j.neunet.2014.09.003. ISSN 0893-6080.
Schmidhuber, Jürgen (2015-01-01). “Deep learning in neural networks: An overview” (英語). Neural Networks 61: 93-94. doi:10.1016/j.neunet.2014.09.003. ISSN 0893-6080.
Goh, Garrett B.; Hodas, Nathan O.; Vishnu, Abhinav (2017-06-15). “Deep learning for computational chemistry” (英語). Journal of Computational Chemistry 38 (16): 1291–1307. arXiv:1701.04503. Bibcode: 2017arXiv170104503G. doi:10.1002/jcc.24764. PMID 28272810.
Pascanu, Razvan; Mikolov, Tomas; Bengio, Yoshua (2013-06-16). “On the difficulty of training recurrent neural networks”. Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28 (Atlanta, GA, USA: JMLR.org): III–1310–III–1318. doi:10.5555/3042817.3043083.
Deng, Li (2014). “A tutorial survey of architectures, algorithms, and applications for deep learning” (英語). APSIPA Transactions on Signal and Information Processing 3 (1): 18. doi:10.1017/atsip.2013.9. ISSN 2048-7703.
Hinton, G. E.; Osindero, S.; Teh, Y. (2006). “A fast learning algorithm for deep belief nets”. Neural Computation 18 (7): 1527–1554. doi:10.1162/neco.2006.18.7.1527. PMID 16764513.
Hinton, G. (2009). “Deep belief networks”. Scholarpedia 4 (5): 5947. Bibcode: 2009SchpJ...4.5947H. doi:10.4249/scholarpedia.5947.
Hochreiter, Sepp; Schmidhuber, Jürgen (1997). “Long Short-Term Memory”. Neural Computation 9 (8): 1735–1780. doi:10.1162/neco.1997.9.8.1735. PMID 9377276.
Graves, A.; Liwicki, M.; Fernandez, S.; Bertolami, R.; Bunke, H.; Schmidhuber, J. (2009). “A Novel Connectionist System for Improved Unconstrained Handwriting Recognition”. IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (5): 855–868. doi:10.1109/tpami.2008.137. PMID 19299860.
He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing; Sun, Jian (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE. pp. 770–778. arXiv:1512.03385. doi:10.1109/CVPR.2016.90. ISBN 978-1-4673-8851-1。
Zaeemzadeh, Alireza; Rahnavard, Nazanin; Shah, Mubarak (2021-11-01). “Norm-Preservation: Why Residual Networks Can Become Extremely Deep?”. IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (11): 3980–3990. arXiv:1805.07477. doi:10.1109/TPAMI.2020.2990339. ISSN 0162-8828.
LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey (2015). “Deep learning”. Nature 521 (7553): 436–444. Bibcode: 2015Natur.521..436L. doi:10.1038/nature14539. PMID 26017442.
Balduzzi, David; Frean, Marcus; Leary, Lennox; Lewis, J P; Ma, Kurt Wan-Duo; McWilliams, Brian (2017-08-06). “The shattered gradients problem: if resnets are the answer, then what is the question?”. Proceedings of the 34th International Conference on Machine Learning - Volume 70 (Sydney, NSW, Australia: JMLR.org): 344. doi:10.5555/3305381.3305417.
Ioffe, Sergey; Szegedy, Christian (2015-07-06). “Batch normalization: accelerating deep network training by reducing internal covariate shift”. Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 (Lille, France: JMLR.org): 449，455. doi:10.5555/3045118.3045167.
Schmidhuber, Jürgen (2015). “Deep learning in neural networks: An overview”. Neural Networks 61: 85–117. arXiv:1404.7828. doi:10.1016/j.neunet.2014.09.003. PMID 25462637.

harvard.edu

ui.adsabs.harvard.edu

Goh, Garrett B.; Hodas, Nathan O.; Vishnu, Abhinav (2017-06-15). “Deep learning for computational chemistry” (英語). Journal of Computational Chemistry 38 (16): 1291–1307. arXiv:1701.04503. Bibcode: 2017arXiv170104503G. doi:10.1002/jcc.24764. PMID 28272810.
Hinton, G. (2009). “Deep belief networks”. Scholarpedia 4 (5): 5947. Bibcode: 2009SchpJ...4.5947H. doi:10.4249/scholarpedia.5947.
LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey (2015). “Deep learning”. Nature 521 (7553): 436–444. Bibcode: 2015Natur.521..436L. doi:10.1038/nature14539. PMID 26017442.

idsia.ch

people.idsia.ch

Hochreiter, S. (1991). Untersuchungen zu dynamischen neuronalen Netzen (PDF) (Diplom thesis). Institut f. Informatik, Technische Univ. Munich.
“Sepp Hochreiter's Fundamental Deep Learning Problem (1991)”. people.idsia.ch. 2017年1月7日閲覧。

ieee.org

ieeexplore.ieee.org

He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing; Sun, Jian (2016). Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE. pp. 770–778. arXiv:1512.03385. doi:10.1109/CVPR.2016.90. ISBN 978-1-4673-8851-1。
Zaeemzadeh, Alireza; Rahnavard, Nazanin; Shah, Mubarak (2021-11-01). “Norm-Preservation: Why Residual Networks Can Become Extremely Deep?”. IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (11): 3980–3990. arXiv:1805.07477. doi:10.1109/TPAMI.2020.2990339. ISSN 0162-8828.

mlr.press

proceedings.mlr.press

Glorot, Xavier; Bordes, Antoine; Bengio, Yoshua (2011-06-14). “Deep Sparse Rectifier Neural Networks” (英語). PMLR: 315–323.

nih.gov

pubmed.ncbi.nlm.nih.gov

Goh, Garrett B.; Hodas, Nathan O.; Vishnu, Abhinav (2017-06-15). “Deep learning for computational chemistry” (英語). Journal of Computational Chemistry 38 (16): 1291–1307. arXiv:1701.04503. Bibcode: 2017arXiv170104503G. doi:10.1002/jcc.24764. PMID 28272810.
Hinton, G. E.; Osindero, S.; Teh, Y. (2006). “A fast learning algorithm for deep belief nets”. Neural Computation 18 (7): 1527–1554. doi:10.1162/neco.2006.18.7.1527. PMID 16764513.
Hochreiter, Sepp; Schmidhuber, Jürgen (1997). “Long Short-Term Memory”. Neural Computation 9 (8): 1735–1780. doi:10.1162/neco.1997.9.8.1735. PMID 9377276.
Graves, A.; Liwicki, M.; Fernandez, S.; Bertolami, R.; Bunke, H.; Schmidhuber, J. (2009). “A Novel Connectionist System for Improved Unconstrained Handwriting Recognition”. IEEE Transactions on Pattern Analysis and Machine Intelligence 31 (5): 855–868. doi:10.1109/tpami.2008.137. PMID 19299860.
LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey (2015). “Deep learning”. Nature 521 (7553): 436–444. Bibcode: 2015Natur.521..436L. doi:10.1038/nature14539. PMID 26017442.
Schmidhuber, Jürgen (2015). “Deep learning in neural networks: An overview”. Neural Networks 61: 85–117. arXiv:1404.7828. doi:10.1016/j.neunet.2014.09.003. PMID 25462637.