Attention (machine learning) (English Wikipedia)

Analysis of information sources in references of the Wikipedia article "Attention (machine learning)" in English language version.

refsWebsite

Global rank English rank

30arxiv.org

69^th place

59^th place

10doi.org

2^nd place

3semanticscholar.org

11^th place

8^th place

3pytorch.org

low place

3youtube.com

9^th place

13^th place

3nih.gov

4^th place

2harvard.edu

18^th place

17^th place

2ieee.org

652^nd place

515^th place

2worldcat.org

5^th place

1handle.net

102^nd place

76^th place

1stanford.edu

179^th place

183^rd place

1sciencedirect.com

149^th place

178^th place

1springer.com

274^th place

309^th place

1catalyzex.com

low place

1unite.ai

low place

1github.com

383^rd place

320^th place

arxiv.org (Global: 69^th place; English: 59^th place)

Bahdanau, Dzmitry; Cho, Kyunghyun; Bengio, Yoshua (2014). "Neural Machine Translation by Jointly Learning to Align and Translate". arXiv:1409.0473 [cs.CL].
Xu, Kelvin; Ba, Jimmy; Kiros, Ryan (2015). Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. arXiv:1502.03044.
Cheng, Jianpeng (2016). "Long Short-Term Memory-Networks for Machine Reading". arXiv:1601.06733 [cs.CL].
Paulus, Romain (2017). "A Deep Reinforced Model for Abstractive Summarization". arXiv:1705.04304 [cs.CL].
Parikh, Anees (2016). Decomposable Attention Model for Natural Language Inference. EMNLP. arXiv:1606.01933.
Lin, Zichao (2017). A Structured Self-Attentive Sentence Embedding. ICLR. arXiv:1703.03130.
Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N.; Kaiser, Lukasz; Polosukhin, Illia (2017). "Attention is All You Need". arXiv:1706.03762 [cs.CL].
Santoro, Adam (2017). Relation Networks for Relational Reasoning. ICLR. arXiv:1706.01427.
Lee, Juho (2019). Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks. ICML. arXiv:1810.00825.
Kitaev, Nikita (2020). Reformer: The Efficient Transformer. ICLR. arXiv:2001.04451.
Wang, Salah (2020). Linformer: Self-Attention with Linear Complexity. ICLR. arXiv:2006.04768.
Choromanski, Krzysztof (2020). Rethinking Attention with Performers. ICLR. arXiv:2009.14794.
Ramsauer, Johannes (2021). Hopfield Networks is All You Need. NeurIPS. arXiv:2008.02217.
Dosovitskiy, Aleksander (2021). An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale. ICLR. arXiv:2010.11929.
Huang, Xiangyu (2019). CCNet: Criss-Cross Attention for Semantic Segmentation. ICCV. arXiv:1811.11721.
Fu, Jing (2019). Dual Attention Network for Scene Segmentation. CVPR. arXiv:1809.02983.
Soydaner, Derya (August 2022). "Attention mechanism in neural networks: where it comes and where it goes". Neural Computing and Applications. 34 (16): 13371–13385. arXiv:2204.13154. doi:10.1007/s00521-022-07366-3. ISSN 0941-0643.
Britz, Denny; Goldie, Anna; Luong, Minh-Thanh; Le, Quoc (2017-03-21). "Massive Exploration of Neural Machine Translation Architectures". arXiv:1703.03906 [cs.CV].
Luong, Minh-Thang (2015-09-20). "Effective Approaches to Attention-Based Neural Machine Translation". arXiv:1508.04025v5 [cs.CL].
Zhu, Xizhou; Cheng, Dazhi; Zhang, Zheng; Lin, Stephen; Dai, Jifeng (2019). "An Empirical Study of Spatial Attention Mechanisms in Deep Networks". 2019 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 6687–6696. arXiv:1904.05873. doi:10.1109/ICCV.2019.00679. ISBN 978-1-7281-4803-8. S2CID 118673006.
Hu, Jie; Shen, Li; Sun, Gang (2018). "Squeeze-and-Excitation Networks". 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7132–7141. arXiv:1709.01507. doi:10.1109/CVPR.2018.00745. ISBN 978-1-5386-6420-9. S2CID 206597034.
Woo, Sanghyun; Park, Jongchan; Lee, Joon-Young; Kweon, In So (2018-07-18). "CBAM: Convolutional Block Attention Module". arXiv:1807.06521 [cs.CV].
Georgescu, Mariana-Iuliana; Ionescu, Radu Tudor; Miron, Andreea-Iuliana; Savencu, Olivian; Ristea, Nicolae-Catalin; Verga, Nicolae; Khan, Fahad Shahbaz (2022-10-12). "Multimodal Multi-Head Convolutional Attention with Various Kernel Sizes for Medical Image Super-Resolution". arXiv:2204.04218 [eess.IV].
Dosovitskiy, Alexey; Beyer, Lucas; Kolesnikov, Alexander; Weissenborn, Dirk; Zhai, Xiaohua; Unterthiner, Thomas; Dehghani, Mostafa; Minderer, Matthias; Heigold, Georg (2021-06-03), An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, arXiv:2010.11929
Abnar, Samira; Zuidema, Willem (2020-05-31), Quantifying Attention Flow in Transformers, arXiv:2005.00928
Brocki, Lennart; Binda, Jakub; Chung, Neo Christopher (2024-10-25), Class-Discriminative Attention Maps for Vision Transformers, arXiv:2312.02364
Mullenbach, James; Wiegreffe, Sarah; Duke, Jon; Sun, Jimeng; Eisenstein, Jacob (2018-04-16), Explainable Prediction of Medical Codes from Clinical Text, arXiv:1802.05695
Bahdanau, Dzmitry; Cho, Kyunghyun; Bengio, Yoshua (2016-05-19), Neural Machine Translation by Jointly Learning to Align and Translate, arXiv:1409.0473
Serrano, Sofia; Smith, Noah A. (2019-06-09), Is Attention Interpretable?, arXiv:1906.03731
Lee, Juho; Lee, Yoonho; Kim, Jungtaek; Kosiorek, Adam R; Choi, Seungjin; Teh, Yee Whye (2018). "Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks". arXiv:1810.00825 [cs.LG].

catalyzex.com (Global: low place; English: low place)

Luo, Fan; Zhang, Juan; Xu, Shenghui (3 July 2024). "Learning Positional Attention for Sequential Recommendation". catalyzex.com.

doi.org (Global: 2^nd place; English: 2^nd place)

Cherry, E. Colin (1953). "Some Experiments on the Recognition of Speech, with One and with Two Ears". The Journal of the Acoustical Society of America. 25 (5): 975–979. Bibcode:1953ASAJ...25..975C. doi:10.1121/1.1907229. hdl:11858/00-001M-0000-002A-F750-3.
Schmidhuber, Jürgen (1992). "Learning to control fast-weight memories: an alternative to recurrent nets". Neural Computation. 4 (1): 131–139. doi:10.1162/neco.1992.4.1.131. S2CID 16683347.
Feldman, Jerome A. (1982). "Dynamic connections in neural networks". Biological Cybernetics. 46 (1): 27–39. doi:10.1007/BF00335349. PMID 6307398.
Hinton, Geoffrey E. (1989). "Connectionist learning procedures". Artificial Intelligence. 40 (1–3): 185–234. doi:10.1016/0004-3702(89)90049-0.
Vinyals, Oriol; Toshev, Alexander; Bengio, Samy; Erhan, Dumitru (2015). "Show and Tell: A Neural Image Caption Generator". 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 3156–3164. doi:10.1109/CVPR.2015.7298935. ISBN 978-1-4673-6964-0.
Jumper, John (2021). "Highly accurate protein structure prediction with AlphaFold". Nature. 596 (7873): 583–589. Bibcode:2021Natur.596..583J. doi:10.1038/s41586-021-03819-2. PMC 8371605. PMID 34265844.
Niu, Zhaoyang; Zhong, Guoqiang; Yu, Hui (2021-09-10). "A review on the attention mechanism of deep learning". Neurocomputing. 452: 48–62. doi:10.1016/j.neucom.2021.03.091. ISSN 0925-2312.
Soydaner, Derya (August 2022). "Attention mechanism in neural networks: where it comes and where it goes". Neural Computing and Applications. 34 (16): 13371–13385. arXiv:2204.13154. doi:10.1007/s00521-022-07366-3. ISSN 0941-0643.
Zhu, Xizhou; Cheng, Dazhi; Zhang, Zheng; Lin, Stephen; Dai, Jifeng (2019). "An Empirical Study of Spatial Attention Mechanisms in Deep Networks". 2019 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 6687–6696. arXiv:1904.05873. doi:10.1109/ICCV.2019.00679. ISBN 978-1-7281-4803-8. S2CID 118673006.
Hu, Jie; Shen, Li; Sun, Gang (2018). "Squeeze-and-Excitation Networks". 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7132–7141. arXiv:1709.01507. doi:10.1109/CVPR.2018.00745. ISBN 978-1-5386-6420-9. S2CID 206597034.

github.com (Global: 383^rd place; English: 320^th place)

Gildenblat, Jacob (2025-07-21), jacobgil/pytorch-grad-cam, retrieved 2025-07-21

handle.net (Global: 102^nd place; English: 76^th place)

hdl.handle.net

Cherry, E. Colin (1953). "Some Experiments on the Recognition of Speech, with One and with Two Ears". The Journal of the Acoustical Society of America. 25 (5): 975–979. Bibcode:1953ASAJ...25..975C. doi:10.1121/1.1907229. hdl:11858/00-001M-0000-002A-F750-3.

harvard.edu (Global: 18^th place; English: 17^th place)

ui.adsabs.harvard.edu

Cherry, E. Colin (1953). "Some Experiments on the Recognition of Speech, with One and with Two Ears". The Journal of the Acoustical Society of America. 25 (5): 975–979. Bibcode:1953ASAJ...25..975C. doi:10.1121/1.1907229. hdl:11858/00-001M-0000-002A-F750-3.
Jumper, John (2021). "Highly accurate protein structure prediction with AlphaFold". Nature. 596 (7873): 583–589. Bibcode:2021Natur.596..583J. doi:10.1038/s41586-021-03819-2. PMC 8371605. PMID 34265844.

ieee.org (Global: 652^nd place; English: 515^th place)

ieeexplore.ieee.org

Tomasi, Carlo (1998). Bilateral filtering for gray and color images. ICCV.
Buades, Antoni (2005). A non-local algorithm for image denoising. CVPR.

nih.gov (Global: 4^th place; English: 4^th place)

pubmed.ncbi.nlm.nih.gov

Feldman, Jerome A. (1982). "Dynamic connections in neural networks". Biological Cybernetics. 46 (1): 27–39. doi:10.1007/BF00335349. PMID 6307398.
Jumper, John (2021). "Highly accurate protein structure prediction with AlphaFold". Nature. 596 (7873): 583–589. Bibcode:2021Natur.596..583J. doi:10.1038/s41586-021-03819-2. PMC 8371605. PMID 34265844.

ncbi.nlm.nih.gov

Jumper, John (2021). "Highly accurate protein structure prediction with AlphaFold". Nature. 596 (7873): 583–589. Bibcode:2021Natur.596..583J. doi:10.1038/s41586-021-03819-2. PMC 8371605. PMID 34265844.

pytorch.org (Global: low place; English: low place)

"Pytorch.org seq2seq tutorial". Retrieved December 2, 2021.
Robertson, Sean. "NLP From Scratch: Translation With a Sequence To Sequence Network and Attention". pytorch.org. Retrieved 2021-12-22.
"FlexAttention: The Flexibility of PyTorch with the Performance of FlashAttention – PyTorch".

sciencedirect.com (Global: 149^th place; English: 178^th place)

Niu, Zhaoyang; Zhong, Guoqiang; Yu, Hui (2021-09-10). "A review on the attention mechanism of deep learning". Neurocomputing. 452: 48–62. doi:10.1016/j.neucom.2021.03.091. ISSN 0925-2312.

semanticscholar.org (Global: 11^th place; English: 8^th place)

api.semanticscholar.org

Schmidhuber, Jürgen (1992). "Learning to control fast-weight memories: an alternative to recurrent nets". Neural Computation. 4 (1): 131–139. doi:10.1162/neco.1992.4.1.131. S2CID 16683347.
Zhu, Xizhou; Cheng, Dazhi; Zhang, Zheng; Lin, Stephen; Dai, Jifeng (2019). "An Empirical Study of Spatial Attention Mechanisms in Deep Networks". 2019 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 6687–6696. arXiv:1904.05873. doi:10.1109/ICCV.2019.00679. ISBN 978-1-7281-4803-8. S2CID 118673006.
Hu, Jie; Shen, Li; Sun, Gang (2018). "Squeeze-and-Excitation Networks". 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7132–7141. arXiv:1709.01507. doi:10.1109/CVPR.2018.00745. ISBN 978-1-5386-6420-9. S2CID 206597034.

springer.com (Global: 274^th place; English: 309^th place)

link.springer.com

Soydaner, Derya (August 2022). "Attention mechanism in neural networks: where it comes and where it goes". Neural Computing and Applications. 34 (16): 13371–13385. arXiv:2204.13154. doi:10.1007/s00521-022-07366-3. ISSN 0941-0643.

stanford.edu (Global: 179^th place; English: 183^rd place)

Rumelhart, David E.; Hinton, G. E.; Mcclelland, James L. (1987-07-29). "A General Framework for Parallel Distributed Processing" (PDF). In Rumelhart, David E.; Hinton, G. E.; PDP Research Group (eds.). Parallel Distributed Processing, Volume 1: Explorations in the Microstructure of Cognition: Foundations. Cambridge, Massachusetts: MIT Press. ISBN 978-0-262-68053-0.

unite.ai (Global: low place; English: low place)

Mittal, Aayush (2024-07-17). "Flash Attention: Revolutionizing Transformer Efficiency". Unite.AI. Retrieved 2024-11-16.

worldcat.org (Global: 5^th place; English: 5^th place)

search.worldcat.org

Niu, Zhaoyang; Zhong, Guoqiang; Yu, Hui (2021-09-10). "A review on the attention mechanism of deep learning". Neurocomputing. 452: 48–62. doi:10.1016/j.neucom.2021.03.091. ISSN 0925-2312.
Soydaner, Derya (August 2022). "Attention mechanism in neural networks: where it comes and where it goes". Neural Computing and Applications. 34 (16): 13371–13385. arXiv:2204.13154. doi:10.1007/s00521-022-07366-3. ISSN 0941-0643.

youtube.com (Global: 9^th place; English: 13^th place)

Neil Rhodes (2021). CS 152 NN—27: Attention: Keys, Queries, & Values. Event occurs at 06:30. Retrieved 2021-12-22.
Alfredo Canziani & Yann Lecun (2021). NYU Deep Learning course, Spring 2020. Event occurs at 05:30. Retrieved 2021-12-22.
Alfredo Canziani & Yann Lecun (2021). NYU Deep Learning course, Spring 2020. Event occurs at 20:15. Retrieved 2021-12-22.

Attention (machine learning) (English Wikipedia)

arxiv.org (Global: 69th place; English: 59th place)

catalyzex.com (Global: low place; English: low place)

doi.org (Global: 2nd place; English: 2nd place)

github.com (Global: 383rd place; English: 320th place)

handle.net (Global: 102nd place; English: 76th place)

hdl.handle.net

harvard.edu (Global: 18th place; English: 17th place)

ui.adsabs.harvard.edu

ieee.org (Global: 652nd place; English: 515th place)

ieeexplore.ieee.org

nih.gov (Global: 4th place; English: 4th place)

pubmed.ncbi.nlm.nih.gov

ncbi.nlm.nih.gov

pytorch.org (Global: low place; English: low place)

sciencedirect.com (Global: 149th place; English: 178th place)

semanticscholar.org (Global: 11th place; English: 8th place)

api.semanticscholar.org

springer.com (Global: 274th place; English: 309th place)

link.springer.com

stanford.edu (Global: 179th place; English: 183rd place)

unite.ai (Global: low place; English: low place)

worldcat.org (Global: 5th place; English: 5th place)

search.worldcat.org

youtube.com (Global: 9th place; English: 13th place)

arxiv.org (Global: 69^th place; English: 59^th place)

doi.org (Global: 2^nd place; English: 2^nd place)

github.com (Global: 383^rd place; English: 320^th place)

handle.net (Global: 102^nd place; English: 76^th place)

harvard.edu (Global: 18^th place; English: 17^th place)

ieee.org (Global: 652^nd place; English: 515^th place)

nih.gov (Global: 4^th place; English: 4^th place)

sciencedirect.com (Global: 149^th place; English: 178^th place)

semanticscholar.org (Global: 11^th place; English: 8^th place)

springer.com (Global: 274^th place; English: 309^th place)

stanford.edu (Global: 179^th place; English: 183^rd place)

worldcat.org (Global: 5^th place; English: 5^th place)

youtube.com (Global: 9^th place; English: 13^th place)