Stochastic gradient descent (English Wikipedia)

Jenny Rose Finkel, Alex Kleeman, Christopher D. Manning (2008). Efficient, Feature-based, Conditional Random Field Parsing. Proc. Annual Meeting of the ACL.

ams.org

mathscinet.ams.org

Kiwiel, Krzysztof C. (2001). "Convergence and efficiency of subgradient methods for quasiconvex minimization". Mathematical Programming, Series A. 90 (1). Berlin, Heidelberg: Springer: 1–25. doi:10.1007/PL00011414. ISSN 0025-5610. MR 1819784. S2CID 10043417.

archive.org

Bottou, Léon (1998). "Online Algorithms and Stochastic Approximations". Online Learning and Neural Networks. Cambridge University Press. ISBN 978-0-521-65263-6.

arxiv.org

Peng, Xinyu; Li, Li; Wang, Fei-Yue (2020). "Accelerating Minibatch Stochastic Gradient Descent Using Typicality Sampling". IEEE Transactions on Neural Networks and Learning Systems. 31 (11): 4649–4659. arXiv:1903.04192. doi:10.1109/TNNLS.2019.2957003. PMID 31899442. S2CID 73728964. Retrieved 2023-10-02.
Kingma, Diederik; Ba, Jimmy (2014). "Adam: A Method for Stochastic Optimization". arXiv:1412.6980 [cs.LG].
Toulis, Panos; Airoldi, Edoardo (2017). "Asymptotic and finite-sample properties of estimators based on stochastic gradients". Annals of Statistics. 45 (4): 1694–1727. arXiv:1408.2923. doi:10.1214/16-AOS1506. S2CID 10279395.
Zeiler, Matthew D. (2012). "ADADELTA: An adaptive learning rate method". arXiv:1212.5701 [cs.LG].
Borysenko, Oleksandr; Byshkin, Maksym (2021). "CoolMomentum: A Method for Stochastic Optimization by Langevin Dynamics with Simulated Annealing". Scientific Reports. 11 (1): 10705. arXiv:2005.14605. Bibcode:2021NatSR..1110705B. doi:10.1038/s41598-021-90144-3. PMC 8139967. PMID 34021212.
Kingma, Diederik; Ba, Jimmy (2014). "Adam: A Method for Stochastic Optimization". arXiv:1412.6980 [cs.LG].
Reddi, Sashank J.; Kale, Satyen; Kumar, Sanjiv (2018). On the Convergence of Adam and Beyond. 6th International Conference on Learning Representations (ICLR 2018). arXiv:1904.09237.
Zhang, Yushun; Chen, Congliang; Shi, Naichen; Sun, Ruoyu; Luo, Zhi-Quan (2022). "Adam Can Converge Without Any Modification On Update Rules". Advances in Neural Information Processing Systems 35. Advances in Neural Information Processing Systems 35 (NeurIPS 2022). arXiv:2208.09632.
Hu, Yuzheng; Lin, Licong; Tang, Shange (2019-12-20). "Second-order Information in First-order Optimization Methods". arXiv:1912.09926. {{cite journal}}: Cite journal requires |journal= (help)
Reddi, Sashank J.; Kale, Satyen; Kumar, Sanjiv (2018). "On the Convergence of Adam and Beyond". arXiv:1904.09237. {{cite journal}}: Cite journal requires |journal= (help)
Tran, Phuong Thi; Phong, Le Trieu (2019). "On the Convergence Proof of AMSGrad and a New Version". IEEE Access. 7: 61706–61716. arXiv:1904.03590. doi:10.1109/ACCESS.2019.2916341. ISSN 2169-3536.
Loshchilov, Ilya; Hutter, Frank (4 January 2019). "Decoupled Weight Decay Regularization". arXiv:1711.05101. {{cite journal}}: Cite journal requires |journal= (help)
Byrd, R. H.; Hansen, S. L.; Nocedal, J.; Singer, Y. (2016). "A Stochastic Quasi-Newton method for Large-Scale Optimization". SIAM Journal on Optimization. 26 (2): 1008–1031. arXiv:1401.7020. doi:10.1137/140954362. S2CID 12396034.
Gess, Benjamin; Kassing, Sebastian; Konarovskyi, Vitalii (14 February 2023). "Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descent". arXiv:2302.07125 [math.PR].

books.google.com

Bottou, Léon; Bousquet, Olivier (2012). "The Tradeoffs of Large Scale Learning". In Sra, Suvrit; Nowozin, Sebastian; Wright, Stephen J. (eds.). Optimization for Machine Learning. Cambridge: MIT Press. pp. 351–368. ISBN 978-0-262-01646-9.

boostedml.com

"Gradient Descent and Momentum: The Heavy Ball Method". 13 July 2020.

bottou.org

leon.bottou.org

Bottou, Léon; Bousquet, Olivier (2008). The Tradeoffs of Large Scale Learning. Advances in Neural Information Processing Systems. Vol. 20. pp. 161–168.

damaru2.github.io

Rubio, David Martínez (2017). Convergence Analysis of an Adaptive Method of Gradient Descent (PDF) (Master thesis). University of Oxford. Retrieved 5 January 2024.

deeplearningbook.org

Goodfellow, Ian; Bengio, Yoshua; Courville, Aaron (2016). Deep Learning. MIT Press. p. 291. ISBN 978-0262035613.

doi.org

Ferguson, Thomas S. (1982). "An inconsistent maximum likelihood estimate". Journal of the American Statistical Association. 77 (380): 831–834. doi:10.1080/01621459.1982.10477894. JSTOR 2287314.
Bilmes, Jeff; Asanovic, Krste; Chin, Chee-Whye; Demmel, James (April 1997). "Using PHiPAC to speed error back-propagation learning". 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP. Munich, Germany: IEEE. pp. 4153–4156 vol.5. doi:10.1109/ICASSP.1997.604861.
Kiwiel, Krzysztof C. (2001). "Convergence and efficiency of subgradient methods for quasiconvex minimization". Mathematical Programming, Series A. 90 (1). Berlin, Heidelberg: Springer: 1–25. doi:10.1007/PL00011414. ISSN 0025-5610. MR 1819784. S2CID 10043417.
Robbins, H.; Monro, S. (1951). "A Stochastic Approximation Method". The Annals of Mathematical Statistics. 22 (3): 400. doi:10.1214/aoms/1177729586.
Kiefer, J.; Wolfowitz, J. (1952). "Stochastic Estimation of the Maximum of a Regression Function". The Annals of Mathematical Statistics. 23 (3): 462–466. doi:10.1214/aoms/1177729392.
Rosenblatt, F. (1958). "The perceptron: A probabilistic model for information storage and organization in the brain". Psychological Review. 65 (6): 386–408. doi:10.1037/h0042519. S2CID 12781225.
Bilmes, Jeff; Asanovic, Krste; Chin, Chee-Whye; Demmel, James (April 1997). "Using PHiPAC to speed error back-propagation learning". 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP. Munich, Germany: IEEE. pp. 4153–4156 vol.5. doi:10.1109/ICASSP.1997.604861.
Peng, Xinyu; Li, Li; Wang, Fei-Yue (2020). "Accelerating Minibatch Stochastic Gradient Descent Using Typicality Sampling". IEEE Transactions on Neural Networks and Learning Systems. 31 (11): 4649–4659. arXiv:1903.04192. doi:10.1109/TNNLS.2019.2957003. PMID 31899442. S2CID 73728964. Retrieved 2023-10-02.
Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (October 1986). "Learning representations by back-propagating errors". Nature. 323 (6088): 533–536. doi:10.1038/323533a0. ISSN 1476-4687. S2CID 205001834.
Nguyen, Giang; Dlugolinsky, Stefan; Bobák, Martin; Tran, Viet; García, Álvaro; Heredia, Ignacio; Malík, Peter; Hluchý, Ladislav (19 January 2019). "Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey" (PDF). Artificial Intelligence Review. 52: 77–124. doi:10.1007/s10462-018-09679-z. S2CID 254236976.
Cited by Darken, Christian; Moody, John (1990). Fast adaptive k-means clustering: some empirical results. Int'l Joint Conf. on Neural Networks (IJCNN). IEEE. doi:10.1109/IJCNN.1990.137720.
Toulis, Panos; Airoldi, Edoardo (2017). "Asymptotic and finite-sample properties of estimators based on stochastic gradients". Annals of Statistics. 45 (4): 1694–1727. arXiv:1408.2923. doi:10.1214/16-AOS1506. S2CID 10279395.
Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning representations by back-propagating errors". Nature. 323 (6088): 533–536. Bibcode:1986Natur.323..533R. doi:10.1038/323533a0. S2CID 205001834.
Borysenko, Oleksandr; Byshkin, Maksym (2021). "CoolMomentum: A Method for Stochastic Optimization by Langevin Dynamics with Simulated Annealing". Scientific Reports. 11 (1): 10705. arXiv:2005.14605. Bibcode:2021NatSR..1110705B. doi:10.1038/s41598-021-90144-3. PMC 8139967. PMID 34021212.
Polyak, Boris T.; Juditsky, Anatoli B. (1992). "Acceleration of stochastic approximation by averaging" (PDF). SIAM J. Control Optim. 30 (4): 838–855. doi:10.1137/0330046. S2CID 3548228. Archived from the original (PDF) on 2016-01-12. Retrieved 2018-02-14.
Naveen, Philip (2022-08-09). "FASFA: A Novel Next-Generation Backpropagation Optimizer". doi:10.36227/techrxiv.20427852.v1. Retrieved 2022-11-19. {{cite journal}}: Cite journal requires |journal= (help)
Tran, Phuong Thi; Phong, Le Trieu (2019). "On the Convergence Proof of AMSGrad and a New Version". IEEE Access. 7: 61706–61716. arXiv:1904.03590. doi:10.1109/ACCESS.2019.2916341. ISSN 2169-3536.
Byrd, R. H.; Hansen, S. L.; Nocedal, J.; Singer, Y. (2016). "A Stochastic Quasi-Newton method for Large-Scale Optimization". SIAM Journal on Optimization. 26 (2): 1008–1031. arXiv:1401.7020. doi:10.1137/140954362. S2CID 12396034.
Spall, J. C. (2000). "Adaptive Stochastic Approximation by the Simultaneous Perturbation Method". IEEE Transactions on Automatic Control. 45 (10): 1839−1853. doi:10.1109/TAC.2000.880982.
Spall, J. C. (2009). "Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm". IEEE Transactions on Automatic Control. 54 (6): 1216–1229. doi:10.1109/TAC.2009.2019793. S2CID 3564529.
Ruppert, D. (1985). "A Newton-Raphson Version of the Multivariate Robbins-Monro Procedure". Annals of Statistics. 13 (1): 236–245. doi:10.1214/aos/1176346589.
Amari, S. (1998). "Natural gradient works efficiently in learning". Neural Computation. 10 (2): 251–276. doi:10.1162/089976698300017746. S2CID 207585383.

dx.doi.org

Naveen, Philip (2022-08-09). "FASFA: A Novel Next-Generation Backpropagation Optimizer". doi:10.36227/techrxiv.20427852.v1. Retrieved 2022-11-19. {{cite journal}}: Cite journal requires |journal= (help)

harvard.edu

ui.adsabs.harvard.edu

Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning representations by back-propagating errors". Nature. 323 (6088): 533–536. Bibcode:1986Natur.323..533R. doi:10.1038/323533a0. S2CID 205001834.
Borysenko, Oleksandr; Byshkin, Maksym (2021). "CoolMomentum: A Method for Stochastic Optimization by Langevin Dynamics with Simulated Annealing". Scientific Reports. 11 (1): 10705. arXiv:2005.14605. Bibcode:2021NatSR..1110705B. doi:10.1038/s41598-021-90144-3. PMC 8139967. PMID 34021212.

seas.harvard.edu

Avi Pfeffer. "CS181 Lecture 5 — Perceptrons" (PDF). Harvard University.^{[permanent dead link]}

ieee.org

ieeexplore.ieee.org

Bilmes, Jeff; Asanovic, Krste; Chin, Chee-Whye; Demmel, James (April 1997). "Using PHiPAC to speed error back-propagation learning". 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP. Munich, Germany: IEEE. pp. 4153–4156 vol.5. doi:10.1109/ICASSP.1997.604861.
Bilmes, Jeff; Asanovic, Krste; Chin, Chee-Whye; Demmel, James (April 1997). "Using PHiPAC to speed error back-propagation learning". 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing. ICASSP. Munich, Germany: IEEE. pp. 4153–4156 vol.5. doi:10.1109/ICASSP.1997.604861.
Peng, Xinyu; Li, Li; Wang, Fei-Yue (2020). "Accelerating Minibatch Stochastic Gradient Descent Using Typicality Sampling". IEEE Transactions on Neural Networks and Learning Systems. 31 (11): 4649–4659. arXiv:1903.04192. doi:10.1109/TNNLS.2019.2957003. PMID 31899442. S2CID 73728964. Retrieved 2023-10-02.
Tran, Phuong Thi; Phong, Le Trieu (2019). "On the Convergence Proof of AMSGrad and a New Version". IEEE Access. 7: 61706–61716. arXiv:1904.03590. doi:10.1109/ACCESS.2019.2916341. ISSN 2169-3536.

jmlr.org

Duchi, John; Hazan, Elad; Singer, Yoram (2011). "Adaptive subgradient methods for online learning and stochastic optimization" (PDF). JMLR. 12: 2121–2159.
Duchi, John; Hazan, Elad; Singer, Yoram (2011). "Adaptive subgradient methods for online learning and stochastic optimization" (PDF). JMLR. 12: 2121–2159.
Gupta, Maya R.; Bengio, Samy; Weston, Jason (2014). "Training highly multiclass classifiers" (PDF). JMLR. 15 (1): 1461–1492.
Li, Qianxiao; Tai, Cheng; E, Weinan (2019). "Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations". Journal of Machine Learning Research. 20 (40): 1–47. ISSN 1533-7928.

jstor.org

Ferguson, Thomas S. (1982). "An inconsistent maximum likelihood estimate". Journal of the American Statistical Association. 77 (380): 831–834. doi:10.1080/01621459.1982.10477894. JSTOR 2287314.

lecun.com

yann.lecun.com

LeCun, Yann A., et al. "Efficient backprop." Neural networks: Tricks of the trade. Springer Berlin Heidelberg, 2012. 9-48

mlr.press

proceedings.mlr.press

"SignSGD: Compressed Optimisation for Non-Convex Problems". 3 July 2018. pp. 560–569.

nature.com

Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (October 1986). "Learning representations by back-propagating errors". Nature. 323 (6088): 533–536. doi:10.1038/323533a0. ISSN 1476-4687. S2CID 205001834.

nih.gov

pubmed.ncbi.nlm.nih.gov

Peng, Xinyu; Li, Li; Wang, Fei-Yue (2020). "Accelerating Minibatch Stochastic Gradient Descent Using Typicality Sampling". IEEE Transactions on Neural Networks and Learning Systems. 31 (11): 4649–4659. arXiv:1903.04192. doi:10.1109/TNNLS.2019.2957003. PMID 31899442. S2CID 73728964. Retrieved 2023-10-02.
Borysenko, Oleksandr; Byshkin, Maksym (2021). "CoolMomentum: A Method for Stochastic Optimization by Langevin Dynamics with Simulated Annealing". Scientific Reports. 11 (1): 10705. arXiv:2005.14605. Bibcode:2021NatSR..1110705B. doi:10.1038/s41598-021-90144-3. PMC 8139967. PMID 34021212.

ncbi.nlm.nih.gov

Borysenko, Oleksandr; Byshkin, Maksym (2021). "CoolMomentum: A Method for Stochastic Optimization by Langevin Dynamics with Simulated Annealing". Scientific Reports. 11 (1): 10705. arXiv:2005.14605. Bibcode:2021NatSR..1110705B. doi:10.1038/s41598-021-90144-3. PMC 8139967. PMID 34021212.

openreview.net

Reddi, Sashank J.; Kale, Satyen; Kumar, Sanjiv (2018). On the Convergence of Adam and Beyond. 6th International Conference on Learning Representations (ICLR 2018). arXiv:1904.09237.
Balles, Lukas; Hennig, Philipp (15 February 2018). "Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients".

oreilly.com

"4. Beyond Gradient Descent - Fundamentals of Deep Learning [Book]".

paperswithcode.com

"Papers with Code - Nesterov Accelerated Gradient Explained".

probml.github.io

Murphy, Kevin (2021). Probabilistic Machine Learning: An Introduction. MIT Press. Retrieved 10 April 2021.

pytorch.org

"torch.optim — PyTorch 2.0 documentation". pytorch.org. Retrieved 2023-10-02.

ruder.io

"An overview of gradient descent optimization algorithms". 19 January 2016.

seg.org

library.seg.org

Jerome R. Krebs, John E. Anderson, David Hinkley, Ramesh Neelamani, Sunwoong Lee, Anatoly Baumstein, and Martin-Daniel Lacasse, (2009), "Fast full-wavefield seismic inversion using encoded sources," GEOPHYSICS 74: WCC177-WCC188.

semanticscholar.org

api.semanticscholar.org

Kiwiel, Krzysztof C. (2001). "Convergence and efficiency of subgradient methods for quasiconvex minimization". Mathematical Programming, Series A. 90 (1). Berlin, Heidelberg: Springer: 1–25. doi:10.1007/PL00011414. ISSN 0025-5610. MR 1819784. S2CID 10043417.
Rosenblatt, F. (1958). "The perceptron: A probabilistic model for information storage and organization in the brain". Psychological Review. 65 (6): 386–408. doi:10.1037/h0042519. S2CID 12781225.
Peng, Xinyu; Li, Li; Wang, Fei-Yue (2020). "Accelerating Minibatch Stochastic Gradient Descent Using Typicality Sampling". IEEE Transactions on Neural Networks and Learning Systems. 31 (11): 4649–4659. arXiv:1903.04192. doi:10.1109/TNNLS.2019.2957003. PMID 31899442. S2CID 73728964. Retrieved 2023-10-02.
Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (October 1986). "Learning representations by back-propagating errors". Nature. 323 (6088): 533–536. doi:10.1038/323533a0. ISSN 1476-4687. S2CID 205001834.
Nguyen, Giang; Dlugolinsky, Stefan; Bobák, Martin; Tran, Viet; García, Álvaro; Heredia, Ignacio; Malík, Peter; Hluchý, Ladislav (19 January 2019). "Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey" (PDF). Artificial Intelligence Review. 52: 77–124. doi:10.1007/s10462-018-09679-z. S2CID 254236976.
Toulis, Panos; Airoldi, Edoardo (2017). "Asymptotic and finite-sample properties of estimators based on stochastic gradients". Annals of Statistics. 45 (4): 1694–1727. arXiv:1408.2923. doi:10.1214/16-AOS1506. S2CID 10279395.
Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (8 October 1986). "Learning representations by back-propagating errors". Nature. 323 (6088): 533–536. Bibcode:1986Natur.323..533R. doi:10.1038/323533a0. S2CID 205001834.
Polyak, Boris T.; Juditsky, Anatoli B. (1992). "Acceleration of stochastic approximation by averaging" (PDF). SIAM J. Control Optim. 30 (4): 838–855. doi:10.1137/0330046. S2CID 3548228. Archived from the original (PDF) on 2016-01-12. Retrieved 2018-02-14.
Dozat, T. (2016). "Incorporating Nesterov Momentum into Adam". S2CID 70293087. {{cite journal}}: Cite journal requires |journal= (help)
Byrd, R. H.; Hansen, S. L.; Nocedal, J.; Singer, Y. (2016). "A Stochastic Quasi-Newton method for Large-Scale Optimization". SIAM Journal on Optimization. 26 (2): 1008–1031. arXiv:1401.7020. doi:10.1137/140954362. S2CID 12396034.
Spall, J. C. (2009). "Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm". IEEE Transactions on Automatic Control. 54 (6): 1216–1229. doi:10.1109/TAC.2009.2019793. S2CID 3564529.
Amari, S. (1998). "Natural gradient works efficiently in learning". Neural Computation. 10 (2): 251–276. doi:10.1162/089976698300017746. S2CID 207585383.

springer.com

link.springer.com

Nguyen, Giang; Dlugolinsky, Stefan; Bobák, Martin; Tran, Viet; García, Álvaro; Heredia, Ignacio; Malík, Peter; Hluchý, Ladislav (19 January 2019). "Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey" (PDF). Artificial Intelligence Review. 52: 77–124. doi:10.1007/s10462-018-09679-z. S2CID 254236976.

tensorflow.org

"Module: tf.keras.optimizers | TensorFlow v2.14.0". TensorFlow. Retrieved 2023-10-02.

toronto.edu

cs.toronto.edu

Hinton, Geoffrey. "Lecture 6e rmsprop: Divide the gradient by a running average of its recent magnitude" (PDF). p. 26. Retrieved 19 March 2020.
Hinton, Geoffrey. "Lecture 6e rmsprop: Divide the gradient by a running average of its recent magnitude" (PDF). p. 26. Retrieved 19 March 2020.

ufl.edu

meyn.ece.ufl.edu

Polyak, Boris T.; Juditsky, Anatoli B. (1992). "Acceleration of stochastic approximation by averaging" (PDF). SIAM J. Control Optim. 30 (4): 838–855. doi:10.1137/0330046. S2CID 3548228. Archived from the original (PDF) on 2016-01-12. Retrieved 2018-02-14.

utoronto.ca

cs.utoronto.ca

Sutskever, Ilya; Martens, James; Dahl, George; Hinton, Geoffrey E. (June 2013). Sanjoy Dasgupta and David Mcallester (ed.). On the importance of initialization and momentum in deep learning (PDF). In Proceedings of the 30th international conference on machine learning (ICML-13). Vol. 28. Atlanta, GA. pp. 1139–1147. Retrieved 14 January 2016.
Sutskever, Ilya (2013). Training recurrent neural networks (PDF) (Ph.D.). University of Toronto. p. 74.

web.archive.org

Polyak, Boris T.; Juditsky, Anatoli B. (1992). "Acceleration of stochastic approximation by averaging" (PDF). SIAM J. Control Optim. 30 (4): 838–855. doi:10.1137/0330046. S2CID 3548228. Archived from the original (PDF) on 2016-01-12. Retrieved 2018-02-14.

worldcat.org

Kiwiel, Krzysztof C. (2001). "Convergence and efficiency of subgradient methods for quasiconvex minimization". Mathematical Programming, Series A. 90 (1). Berlin, Heidelberg: Springer: 1–25. doi:10.1007/PL00011414. ISSN 0025-5610. MR 1819784. S2CID 10043417.
Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (October 1986). "Learning representations by back-propagating errors". Nature. 323 (6088): 533–536. doi:10.1038/323533a0. ISSN 1476-4687. S2CID 205001834.
Whye, Schwarz, Jonathan Jayakumar, Siddhant M. Pascanu, Razvan Latham, Peter E. Teh, Yee (2021-10-01). Powerpropagation: A sparsity inducing weight reparameterisation. OCLC 1333722169.{{cite book}}: CS1 maint: multiple names: authors list (link)
Tran, Phuong Thi; Phong, Le Trieu (2019). "On the Convergence Proof of AMSGrad and a New Version". IEEE Access. 7: 61706–61716. arXiv:1904.03590. doi:10.1109/ACCESS.2019.2916341. ISSN 2169-3536.
Li, Qianxiao; Tai, Cheng; E, Weinan (2019). "Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations". Journal of Machine Learning Research. 20 (40): 1–47. ISSN 1533-7928.