Gradient descent (English Wikipedia)

Bernstein, Jeremy; Vahdat, Arash; Yue, Yisong; Liu, Ming-Yu (2020-06-12). "On the distance between two neural networks and the stability of learning". arXiv:2002.03432 [cs.LG].
Bouwmeester, Henricus; Dougherty, Andrew; Knyazev, Andrew V. (2015). "Nonsymmetric Preconditioning for Conjugate Gradient and Steepest Descent Methods". Procedia Computer Science. 51: 276–285. arXiv:1212.6680. doi:10.1016/j.procs.2015.05.241.
Diakonikolas, Jelena; Jordan, Michael I. (January 2021). "Generalized Momentum-Based Methods: A Hamiltonian Perspective". SIAM Journal on Optimization. 31 (1): 915–944. arXiv:1906.00436. doi:10.1137/20M1322716. ISSN 1052-6234.
Kingma, Diederik P.; Ba, Jimmy (2017-01-29), Adam: A Method for Stochastic Optimization, arXiv:1412.6980
Xie, Zeke; Yuan, Li; Zhu, Zhanxing; Sugiyama, Masashi (2021-07-01). "Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization". Proceedings of the 38th International Conference on Machine Learning. PMLR: 11448–11458. arXiv:2103.17182.
Kim, D.; Fessler, J. A. (2016). "Optimized First-order Methods for Smooth Convex Minimization". Mathematical Programming. 151 (1–2): 81–107. arXiv:1406.5468. doi:10.1007/s10107-015-0949-3. PMC 5067109. PMID 27765996. S2CID 207055414.
Drori, Yoel (2017). "The Exact Information-based Complexity of Smooth Convex Minimization". Journal of Complexity. 39: 1–16. arXiv:1606.01424. doi:10.1016/j.jco.2016.11.001. S2CID 205861966.
Combettes, P. L.; Pesquet, J.-C. (2011). "Proximal splitting methods in signal processing". In Bauschke, H. H.; Burachik, R. S.; Combettes, P. L.; Elser, V.; Luke, D. R.; Wolkowicz, H. (eds.). Fixed-Point Algorithms for Inverse Problems in Science and Engineering. New York: Springer. pp. 185–212. arXiv:0912.3522. ISBN 978-1-4419-9568-1.
Bubeck, Sébastien (2015). "Convex Optimization: Algorithms and Complexity". arXiv:1405.4980 [math.OC].

coursera.org (Global: low place; English: low place)

Geoffrey Hinton; Nitish Srivastava; Kevin Swersky. "The momentum method". Coursera. Retrieved 2 October 2018. Part of a lecture series for the Coursera online course Neural Networks for Machine Learning Archived 2016-12-31 at the Wayback Machine.

doi.org (Global: 2^nd place; English: 2^nd place)

doi.org

Boyd, Stephen; Vandenberghe, Lieven (2004-03-08). Convex Optimization. Cambridge University Press. doi:10.1017/cbo9780511804441. ISBN 978-0-521-83378-3.
Lemaréchal, C. (1 January 2012). "Cauchy and the gradient method" (PDF). In Grötschel, M. (ed.). Optimization Stories. Documenta Mathematica Series. Vol. 6 (1st ed.). EMS Press. pp. 251–254. doi:10.4171/dms/6/27. ISBN 978-3-936609-58-5. Archived from the original (PDF) on 2018-12-29. Retrieved 2020-01-26.
Courant, R. (1943). "Variational methods for the solution of problems of equilibrium and vibrations". Bulletin of the American Mathematical Society. 49 (1): 1–23. doi:10.1090/S0002-9904-1943-07818-4.
Curry, Haskell B. (1944). "The Method of Steepest Descent for Non-linear Minimization Problems". Quart. Appl. Math. 2 (3): 258–261. doi:10.1090/qam/10667.
Barzilai, Jonathan; Borwein, Jonathan M. (1988). "Two-Point Step Size Gradient Methods". IMA Journal of Numerical Analysis. 8 (1): 141–148. doi:10.1093/imanum/8.1.141.
Wolfe, Philip (April 1969). "Convergence Conditions for Ascent Methods". SIAM Review. 11 (2): 226–235. doi:10.1137/1011036.
Bouwmeester, Henricus; Dougherty, Andrew; Knyazev, Andrew V. (2015). "Nonsymmetric Preconditioning for Conjugate Gradient and Steepest Descent Methods". Procedia Computer Science. 51: 276–285. arXiv:1212.6680. doi:10.1016/j.procs.2015.05.241.
Abdulkadirov, Ruslan; Lyakhov, Pavel; Nagornov, Nikolay (January 2023). "Survey of Optimization Algorithms in Modern Neural Networks". Mathematics. 11 (11): 2466. doi:10.3390/math11112466. ISSN 2227-7390.
Diakonikolas, Jelena; Jordan, Michael I. (January 2021). "Generalized Momentum-Based Methods: A Hamiltonian Perspective". SIAM Journal on Optimization. 31 (1): 915–944. arXiv:1906.00436. doi:10.1137/20M1322716. ISSN 1052-6234.
Meyer, Gerard G. L. (November 1974). "Accelerated Frank–Wolfe Algorithms". SIAM Journal on Control. 12 (4): 655–663. doi:10.1137/0312050. ISSN 0036-1402.
Ross, I.M. (July 2019). "An optimal control theory for nonlinear optimization". Journal of Computational and Applied Mathematics. 354: 39–51. doi:10.1016/j.cam.2018.12.044. S2CID 127649426.
Walkington, Noel J. (2023). "Nesterov's Method for Convex Optimization". SIAM Review. 65 (2): 539–562. doi:10.1137/21M1390037. ISSN 0036-1445.
Kim, D.; Fessler, J. A. (2016). "Optimized First-order Methods for Smooth Convex Minimization". Mathematical Programming. 151 (1–2): 81–107. arXiv:1406.5468. doi:10.1007/s10107-015-0949-3. PMC 5067109. PMID 27765996. S2CID 207055414.
Drori, Yoel (2017). "The Exact Information-based Complexity of Smooth Convex Minimization". Journal of Complexity. 39: 1–16. arXiv:1606.01424. doi:10.1016/j.jco.2016.11.001. S2CID 205861966.
Qian, Ning (January 1999). "On the momentum term in gradient descent learning algorithms". Neural Networks. 12 (1): 145–151. CiteSeerX 10.1.1.57.5612. doi:10.1016/S0893-6080(98)00116-6. PMID 12662723. S2CID 2783597.

dx.doi.org

Boyd, Stephen; Vandenberghe, Lieven (2004-03-08). Convex Optimization. Cambridge University Press. doi:10.1017/cbo9780511804441. ISBN 978-0-521-83378-3.

mlr.press (Global: low place; English: low place)

proceedings.mlr.press

Xie, Zeke; Yuan, Li; Zhu, Zhanxing; Sugiyama, Masashi (2021-07-01). "Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization". Proceedings of the 38th International Conference on Machine Learning. PMLR: 11448–11458. arXiv:2103.17182.

nih.gov (Global: 4^th place; English: 4^th place)

pubmed.ncbi.nlm.nih.gov

Kim, D.; Fessler, J. A. (2016). "Optimized First-order Methods for Smooth Convex Minimization". Mathematical Programming. 151 (1–2): 81–107. arXiv:1406.5468. doi:10.1007/s10107-015-0949-3. PMC 5067109. PMID 27765996. S2CID 207055414.
Qian, Ning (January 1999). "On the momentum term in gradient descent learning algorithms". Neural Networks. 12 (1): 145–151. CiteSeerX 10.1.1.57.5612. doi:10.1016/S0893-6080(98)00116-6. PMID 12662723. S2CID 2783597.

ncbi.nlm.nih.gov

Kim, D.; Fessler, J. A. (2016). "Optimized First-order Methods for Smooth Convex Minimization". Mathematical Programming. 151 (1–2): 81–107. arXiv:1406.5468. doi:10.1007/s10107-015-0949-3. PMC 5067109. PMID 27765996. S2CID 207055414.

psu.edu (Global: 207^th place; English: 136^th place)

citeseerx.ist.psu.edu

Qian, Ning (January 1999). "On the momentum term in gradient descent learning algorithms". Neural Networks. 12 (1): 145–151. CiteSeerX 10.1.1.57.5612. doi:10.1016/S0893-6080(98)00116-6. PMID 12662723. S2CID 2783597.

researchgate.net (Global: 120^th place; English: 125^th place)

Polyak, Boris (1987). Introduction to Optimization.

semanticscholar.org (Global: 11^th place; English: 8^th place)

api.semanticscholar.org

Ross, I.M. (July 2019). "An optimal control theory for nonlinear optimization". Journal of Computational and Applied Mathematics. 354: 39–51. doi:10.1016/j.cam.2018.12.044. S2CID 127649426.
Kim, D.; Fessler, J. A. (2016). "Optimized First-order Methods for Smooth Convex Minimization". Mathematical Programming. 151 (1–2): 81–107. arXiv:1406.5468. doi:10.1007/s10107-015-0949-3. PMC 5067109. PMID 27765996. S2CID 207055414.
Drori, Yoel (2017). "The Exact Information-based Complexity of Smooth Convex Minimization". Journal of Complexity. 39: 1–16. arXiv:1606.01424. doi:10.1016/j.jco.2016.11.001. S2CID 205861966.
Qian, Ning (January 1999). "On the momentum term in gradient descent learning algorithms". Neural Networks. 12 (1): 145–151. CiteSeerX 10.1.1.57.5612. doi:10.1016/S0893-6080(98)00116-6. PMID 12662723. S2CID 2783597.

siam.org (Global: low place; English: 6,473^rd place)

epubs.siam.org

Diakonikolas, Jelena; Jordan, Michael I. (January 2021). "Generalized Momentum-Based Methods: A Hamiltonian Perspective". SIAM Journal on Optimization. 31 (1): 915–944. arXiv:1906.00436. doi:10.1137/20M1322716. ISSN 1052-6234.
Meyer, Gerard G. L. (November 1974). "Accelerated Frank–Wolfe Algorithms". SIAM Journal on Control. 12 (4): 655–663. doi:10.1137/0312050. ISSN 0036-1402.
Walkington, Noel J. (2023). "Nesterov's Method for Convex Optimization". SIAM Review. 65 (2): 539–562. doi:10.1137/21M1390037. ISSN 0036-1445.

tlienart.github.io (Global: low place; English: low place)

"Mirror descent algorithm".

ucla.edu (Global: 782^nd place; English: 585^th place)

seas.ucla.edu

Vandenberghe, Lieven (2019). "Fast Gradient Methods" (PDF). Lecture notes for EE236C at UCLA.

uni-bielefeld.de (Global: 9,437^th place; English: low place)

math.uni-bielefeld.de

Lemaréchal, C. (1 January 2012). "Cauchy and the gradient method" (PDF). In Grötschel, M. (ed.). Optimization Stories. Documenta Mathematica Series. Vol. 6 (1st ed.). EMS Press. pp. 251–254. doi:10.4171/dms/6/27. ISBN 978-3-936609-58-5. Archived from the original (PDF) on 2018-12-29. Retrieved 2020-01-26.

web.archive.org (Global: 1^st place; English: 1^st place)

Lemaréchal, C. (1 January 2012). "Cauchy and the gradient method" (PDF). In Grötschel, M. (ed.). Optimization Stories. Documenta Mathematica Series. Vol. 6 (1st ed.). EMS Press. pp. 251–254. doi:10.4171/dms/6/27. ISBN 978-3-936609-58-5. Archived from the original (PDF) on 2018-12-29. Retrieved 2020-01-26.
Geoffrey Hinton; Nitish Srivastava; Kevin Swersky. "The momentum method". Coursera. Retrieved 2 October 2018. Part of a lecture series for the Coursera online course Neural Networks for Machine Learning Archived 2016-12-31 at the Wayback Machine.

willamette.edu (Global: low place; English: low place)

"Momentum and Learning Rate Adaptation". Willamette University. Retrieved 17 October 2014.

worldcat.org (Global: 5^th place; English: 5^th place)

search.worldcat.org

Abdulkadirov, Ruslan; Lyakhov, Pavel; Nagornov, Nikolay (January 2023). "Survey of Optimization Algorithms in Modern Neural Networks". Mathematics. 11 (11): 2466. doi:10.3390/math11112466. ISSN 2227-7390.
Diakonikolas, Jelena; Jordan, Michael I. (January 2021). "Generalized Momentum-Based Methods: A Hamiltonian Perspective". SIAM Journal on Optimization. 31 (1): 915–944. arXiv:1906.00436. doi:10.1137/20M1322716. ISSN 1052-6234.
Meyer, Gerard G. L. (November 1974). "Accelerated Frank–Wolfe Algorithms". SIAM Journal on Control. 12 (4): 655–663. doi:10.1137/0312050. ISSN 0036-1402.
Walkington, Noel J. (2023). "Nesterov's Method for Convex Optimization". SIAM Review. 65 (2): 539–562. doi:10.1137/21M1390037. ISSN 0036-1445.

Gradient descent (English Wikipedia)

archive.org (Global: 6th place; English: 6th place)

arxiv.org (Global: 69th place; English: 59th place)

coursera.org (Global: low place; English: low place)

doi.org (Global: 2nd place; English: 2nd place)

doi.org

dx.doi.org

mlr.press (Global: low place; English: low place)

proceedings.mlr.press

nih.gov (Global: 4th place; English: 4th place)

pubmed.ncbi.nlm.nih.gov

ncbi.nlm.nih.gov

psu.edu (Global: 207th place; English: 136th place)

citeseerx.ist.psu.edu

researchgate.net (Global: 120th place; English: 125th place)

semanticscholar.org (Global: 11th place; English: 8th place)

api.semanticscholar.org

siam.org (Global: low place; English: 6,473rd place)

epubs.siam.org

tlienart.github.io (Global: low place; English: low place)

ucla.edu (Global: 782nd place; English: 585th place)

seas.ucla.edu

uni-bielefeld.de (Global: 9,437th place; English: low place)

math.uni-bielefeld.de

web.archive.org (Global: 1st place; English: 1st place)

willamette.edu (Global: low place; English: low place)

worldcat.org (Global: 5th place; English: 5th place)

search.worldcat.org

archive.org (Global: 6^th place; English: 6^th place)

arxiv.org (Global: 69^th place; English: 59^th place)

doi.org (Global: 2^nd place; English: 2^nd place)

nih.gov (Global: 4^th place; English: 4^th place)

psu.edu (Global: 207^th place; English: 136^th place)

researchgate.net (Global: 120^th place; English: 125^th place)

semanticscholar.org (Global: 11^th place; English: 8^th place)

siam.org (Global: low place; English: 6,473^rd place)

ucla.edu (Global: 782^nd place; English: 585^th place)

uni-bielefeld.de (Global: 9,437^th place; English: low place)

web.archive.org (Global: 1^st place; English: 1^st place)

worldcat.org (Global: 5^th place; English: 5^th place)