PyTorch (Japanese Wikipedia)

PETALS: Collaborative Inference and Fine-tuning of Large Models p.2 Alexander Borzunov et al. 2023年
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU p.6 Ying Sheng et al. 2023年
"torch.fx, a program capture and transformation library for PyTorch" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"torch.fx represents programs in a DAG-based IR" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"graph-based ... IR ... Program transformations ... is as simple embedded programming languages that are meta-programmed from a host language, predominantly Python" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"torch.fx provides an fx.graph_drawer package, which gives the user the ability to visualize torch.fx graphs with Graphviz" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"Quantization makes use of torch.fx’s graph and GraphModule representation to simultaneously modify the program code and weight values." Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"6.2.2 Fusion Optimizations ... torch.fx provides the necessary non-local program context and state modification facilities needed for this transformation with its ahead-of-time, graph-based nature" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
" The project was quickly developed using torch.fx’s Python APIs as well as TensorRT’s Python APIs, creating a translation layer between the two." Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"the purposes of serialization or export. For instance, TorchScript" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.

chainer.org (Global: low place; Japanese: low place)

Chainer Team (2010年12月5日). “Chainer/CuPy v7のリリースと今後の開発体制について”. Chainer Blog. 2020年8月10日閲覧。

cloudflare.com (Global: 8,991^st place; Japanese: low place)

blog.cloudflare.com

Cloudflare R2 and MosaicML enable training LLMs on any compute, anywhere in the world, with zero switching costs Cloudflare 2023年5月16日

deepspeed.ai (Global: low place; Japanese: low place)

DeepSpeed-MII: instant speedup on 24,000+ open-source DL models with up to 40x cheaper inference Microsoft 2022年10月10日
ZeRO++ Microsoft
ZeRO-Offload Microsoft
Pipeline Parallelism - DeepSpeed Microsoft
Mixture of Experts - DeepSpeed Microsoft
ZeRO-Inference: Democratizing massive model inference Microsoft 2022年9月9日

deepspeed.readthedocs.io (Global: low place; Japanese: low place)

ZeRO Microsoft

dl4nlp.info (Global: low place; Japanese: low place)

“Natural Language Processing (NLP) with PyTorch — NLP with PyTorch documentation” (英語). dl4nlp.info. 2019年3月30日閲覧。

doi.org (Global: 2^nd place; Japanese: 6^th place)

Ketkar, Nikhil (2017). “Introduction to PyTorch” (英語). Deep Learning with Python. Apress, Berkeley, CA. pp. 195–208. doi:10.1007/978-1-4842-2766-4_12. ISBN 9781484227657

fairscale.readthedocs.io (Global: low place; Japanese: low place)

Efficient Memory management - FairScale documentation Facebook Research
Pipeline Parallelism - FairScale documentation Facebook Research

github.com (Global: 383^rd place; Japanese: 521^st place)

"Release 2.9.0"; 閲覧日: 2025年10月20日; 出版日: 2025年10月15日.
GitHub - catalyst-team/catalyst: Accelerated DL & RL, Catalyst-Team, (2019-12-05) 2019年12月5日閲覧。
PiPPy PyTorch Foundation
"TorchScript is executed using an interpreter attached to a JIT-optimizer and compiler." PyTorch. JIT Technical Overview. 2022-03-19閲覧.
"The executor specializes the Graph for the particular set of inputs." PyTorch. JIT Technical Overview. 2022-03-19閲覧.
"It propagates constants, pre-computing as much as possible" PyTorch. JIT Technical Overview. 2022-03-19閲覧.
"Eliminating dead code" PyTorch. JIT Technical Overview. 2022-03-19閲覧.
Fairseq Meta AI
TorchRL PyTorch Foundation

googleblog.com (Global: 1,272^nd place; Japanese: 1,494^th place)

developers.googleblog.com

AI Edge Torch: High Performance Inference of PyTorch Models on Mobile Devices Google 2024年5月14日

habana.ai (Global: low place; Japanese: low place)

docs.habana.ai

Intel Gaudi PyTorch Python API (habana_frameworks.torch) — Gaudi Documentation 1.18.0 documentation Intel

huggingface.co (Global: low place; Japanese: low place)

Model Parallelism — transformers 4.11.3 documentation Hugging Face
Handling big models for inference HuggingFace
How 🤗 Accelerate runs very large models thanks to PyTorch HuggingFace 2022年9月27日
Pickle Scanning HuggingFace
BetterTransformer - Overview Hugging Face
AltDiffusion Hugging Face
Summary of the tokenizers Hugging Face
Integrate your library with the Hub Hugging Face

impress.co.jp (Global: 648^th place; Japanese: 46^th place)

pc.watch.impress.co.jp

NVIDIA Ampereにおけるプルーニング対応の特徴 Impress 2020年7月20日

infoworld.com (Global: 3,696^th place; Japanese: 7,892^nd place)

Yegulalp, Serdar (2017年1月19日). “Facebook brings GPU-powered machine learning to Python”. InfoWorld 2019年3月30日閲覧。

itmedia.co.jp (Global: 624^th place; Japanese: 41^st place)

動画も音楽もゲームも　広がるAIエンターテインメントの可能性 ITmedia 2022年11月11日

kaitai.io (Global: low place; Japanese: low place)

formats.kaitai.io

Python pickle serialization format: format specification Kaitai Project

medium.com (Global: 551^st place; Japanese: 1,605^th place)

Synced (2018年4月2日). “Caffe2 Merges With PyTorch”. 2019年3月30日閲覧。

microsoft.com (Global: 153^rd place; Japanese: 260^th place)

ZeRO-Infinity and DeepSpeed: Unlocking unprecedented model scale for deep learning training Microsoft

nod.ai (Global: low place; Japanese: low place)

My spring internship – torch-mlir eager mode, OPT and blowing away the main git repo nod.ai 2022年6月20日

nvidia.com (Global: 2,503^rd place; Japanese: 2,605^th place)

oracle.com (Global: 1,514^th place; Japanese: 1,491^st place)

blogs.oracle.com

Train and deploy generative AI faster with MosaicML and Oracle オラクル 2023年5月16日

oreilly.com (Global: 4,347^th place; Japanese: low place)

Lorica, Ben (2017年8月3日). “Why AI and machine learning researchers are beginning to embrace PyTorch”. O'Reilly Media. 2019年3月30日閲覧。
Patel, Mo (2017年12月7日). “When two trends fuse: PyTorch and recommender systems” (英語). O'Reilly Media 2019年3月30日閲覧。

phoronix.com (Global: 4,683^rd place; Japanese: 9,830^th place)

PyTorch 2.5 Released With Improved Intel GPU Support Phoronix 2024年10月17日

pypi.org (Global: low place; Japanese: low place)

fairseq · PyPI Facebook

pytorch.org (Global: low place; Japanese: low place)

PYTORCH-TRANSFORMERS: PyTorch implementations of popular NLP Transformers, PyTorch Hub, (2019-12-01) 2019年12月1日閲覧。
“PyTorch” (英語). www.pytorch.org. 2019年12月5日閲覧。
“PyTorch – About”. pytorch.org. 2018年6月15日時点のオリジナルよりアーカイブ。2018年6月11日閲覧。
PyTorch 2.1: automatic dynamic shape compilation, distributed checkpointing
torch.tensor PyTorch Foundation
Autograd PyTorch Foundation
Complex Numbers PyTorch Foundation
Quantization PyTorch Foundation
Sparse Semi-Structured Tensors PyTorch Foundation
Module PyTorch Foundation
“The C++ Frontend”. PyTorch Master Documentation. 2019年7月29日閲覧。
Tensor Basics PyTorch Foundation
Getting Started on Intel GPU PyTorch Foundation
Facilitating New Backend Integration by PrivateUse1 PyTorch Foundation
Challenges and Efforts in PyTorch Multi-Device Integration: Compatibility, Portability, and Integration Efficiencies PyTorch Foundation 2024年9月18日
PyTorch 2.0 PyTorch Foundation
Installation PyTorch Foundation
Getting Started with Distributed Data Parallel — PyTorch Tutorials 2.3.0+cu121 documentation PyTorch Foundation
Introducing PyTorch Fully Sharded Data Parallel (FSDP) API PyTorch Foundation 2022年3月14日
Pipeline Parallelism PyTorch Foundation
Tensor Parallelism - torch.distributed.tensor.parallel PyTorch Foundation
Docs > CUDA semantics - CUDA streams PyTorch Foundation
torch.onnx PyTorch Foundation
"TorchScript is a way to create serializable and optimizable models from PyTorch code." PyTorch. TorchScript.
"TorchScript is a statically typed subset of Python" PyTorch. TorchScript Language Reference.
"TorchScript program that can be run independently from Python, such as in a standalone C++ program." PyTorch. TorchScript.
"In PyTorch 1.10, we’ve added an LLVM-based JIT compiler for CPUs that can fuse together sequences of torch library calls" PyTorch. PyTorch 1.10 Release.
Saving and Loading Models — PyTorch Tutorials 1.12.1+cu102 documentation PyTorch Foundation
torch.onnx — PyTorch 1.13 documentation PyTorch Foundation
"Scripting a function or nn.Module will inspect the source code, compile it as TorchScript code using the TorchScript compiler, and return a ScriptModule or ScriptFunction." PyTorch. TORCH.JIT.SCRIPT. pytorch.org. 2023-08-29閲覧.
"Using torch.jit.trace and torch.jit.trace_module, you can turn an existing module or Python function into a TorchScript ScriptFunction or ScriptModule. You must provide example inputs, and we run the function, recording the operations performed on all the tensors." Python. TORCH.JIT.TRACE. pytorch.org. 2023-08-29閲覧.
"Tracing will not record any control-flow like if-statements or loops." Python. TORCH.JIT.TRACE. pytorch.org. 2023-08-29閲覧.
"FX is a toolkit for developers to use to transform nn.Module instances." PyTorch. TORCH.FX. 2022-03-23閲覧.
"FX consists of three main components: a symbolic tracer, an intermediate representation, and Python code generation." PyTorch. TORCH.FX. 2022-03-23閲覧.
functorch.compile (experimental) — functorch 1.13 documentation PyTorch Foundation
functorch.compile.ts_compile — functorch 1.13 documentation PyTorch Foundation
TorchDynamo(torch.compile) integration in PyTorch XLA PyTorch Foundation
The Path to Achieve Ultra-Low Inference Latency With LLaMA 65B on PyTorch/XLA PyTorch Foundation
Quantization PyTorch Foundation
PyTorch Mobile PyTorch Foundation
ExecuTorch PyTorch Foundation
torch.utils.mobile_optimizer PyTorch Foundation
torch.nn - Transformer Layers PyTorch Foundation
New Releases: PyTorch 1.2, torchtext 0.4, torchaudio 0.3, and torchvision 0.4 PyTorch Foundation 2019年8月8日
PyTorch 2.0: Our next generation release that is faster, more Pythonic and Dynamic as ever PyTorch Foundation 2023年3月15日
A BetterTransformer for Fast Transformer Inference PyTorch Foundation 2022年7月12日
Models and pre-trained weights — Torchvision main documentation PyTorch Foundation
New Library Updates in PyTorch 2.1 PyTorch Foundation 2023年10月4日
torchaudio.pipelines PyTorch Foundation
SST-2 Binary text classification with XLM-RoBERTa model PyTorch Foundation
T5-Base Model for Summarization, Sentiment Classification, and Translation PyTorch Foundation
torchtext.transforms PyTorch Foundation
torchtext.vocab PyTorch Foundation
torchtext PyTorch Foundation
Introducing torchchat: Accelerating Local LLM Inference on Laptop, Desktop and Mobile PyTorch Foundation 2024年7月30日
Accelerating Generative AI with PyTorch II: GPT, Fast PyTorch Foundation 2023年11月30日
torch.hub — PyTorch 1.13 documentation PyTorch Foundation
PyTorch 1.12: TorchArrow, Functional API for Modules and nvFuser, are now available PyTorch Foundation 2022年6月28日
"torch.fx.symbolic_trace() ... can’t handle control flow" PyTorch. Why do you need another way of optimizing PyTorch code? - Getting Started - TorchDynamo Overview. pytorch.org. 2023-08-28閲覧.

techcrunch.com (Global: 187^th place; Japanese: 440^th place)

Mannes, John. “Facebook and Microsoft collaborate to simplify conversions from PyTorch to Caffe2” (英語). TechCrunch 2019年3月30日閲覧. "FAIR is accustomed to working with PyTorch — a deep learning framework optimized for achieving state of the art results in research, regardless of resource constraints. Unfortunately in the real world, most of us are limited by the computational capabilities of our smartphones and computers."

towardsdatascience.com (Global: 8,920^th place; Japanese: low place)

DeepSpeed Deep Dive — Model Implementations for Inference (MII) Towards Data Science 2022年11月17日

uber.com (Global: low place; Japanese: low place)

eng.uber.com

“Uber AI Labs Open Sources Pyro, a Deep Probabilistic Programming Language” (英語). Uber Engineering Blog. (2017年11月3日). https://eng.uber.com/pyro/ 2017年12月18日閲覧。

ultralytics.com (Global: low place; Japanese: low place)

docs.ultralytics.com

PyTorch Hub Ultralytics

venturebeat.com (Global: 616^th place; Japanese: 2,168^th place)

Arakelyan, Sophia (2017年11月29日). “Tech giants are using open source frameworks to dominate the AI community” (英語). VentureBeat. 2019年3月30日閲覧。

wandb.ai (Global: low place; Japanese: low place)

Hugging Face Introduces Private Hub Weights & Biases 2022年8月3日

web.archive.org (Global: 1^st place; Japanese: 1^st place)

“PyTorch – About”. pytorch.org. 2018年6月15日時点のオリジナルよりアーカイブ。2018年6月11日閲覧。

PyTorch (Japanese Wikipedia)

amazon.com (Global: 105th place; Japanese: 738th place)

aws.amazon.com

analyticsvidhya.com (Global: low place; Japanese: low place)

arxiv.org (Global: 69th place; Japanese: 227th place)

chainer.org (Global: low place; Japanese: low place)

cloudflare.com (Global: 8,991st place; Japanese: low place)

blog.cloudflare.com

deepspeed.ai (Global: low place; Japanese: low place)

deepspeed.readthedocs.io (Global: low place; Japanese: low place)

dl4nlp.info (Global: low place; Japanese: low place)

doi.org (Global: 2nd place; Japanese: 6th place)

fairscale.readthedocs.io (Global: low place; Japanese: low place)

github.com (Global: 383rd place; Japanese: 521st place)

googleblog.com (Global: 1,272nd place; Japanese: 1,494th place)

developers.googleblog.com

habana.ai (Global: low place; Japanese: low place)

docs.habana.ai

huggingface.co (Global: low place; Japanese: low place)

impress.co.jp (Global: 648th place; Japanese: 46th place)

pc.watch.impress.co.jp

infoworld.com (Global: 3,696th place; Japanese: 7,892nd place)

itmedia.co.jp (Global: 624th place; Japanese: 41st place)

kaitai.io (Global: low place; Japanese: low place)

formats.kaitai.io

medium.com (Global: 551st place; Japanese: 1,605th place)

microsoft.com (Global: 153rd place; Japanese: 260th place)

nod.ai (Global: low place; Japanese: low place)

nvidia.com (Global: 2,503rd place; Japanese: 2,605th place)

docs.nvidia.com

developer.nvidia.com

catalog.ngc.nvidia.com

oracle.com (Global: 1,514th place; Japanese: 1,491st place)

blogs.oracle.com

oreilly.com (Global: 4,347th place; Japanese: low place)

phoronix.com (Global: 4,683rd place; Japanese: 9,830th place)

pypi.org (Global: low place; Japanese: low place)

pytorch.org (Global: low place; Japanese: low place)

techcrunch.com (Global: 187th place; Japanese: 440th place)

towardsdatascience.com (Global: 8,920th place; Japanese: low place)

uber.com (Global: low place; Japanese: low place)

eng.uber.com

ultralytics.com (Global: low place; Japanese: low place)

docs.ultralytics.com

venturebeat.com (Global: 616th place; Japanese: 2,168th place)

wandb.ai (Global: low place; Japanese: low place)

web.archive.org (Global: 1st place; Japanese: 1st place)

amazon.com (Global: 105^th place; Japanese: 738^th place)

arxiv.org (Global: 69^th place; Japanese: 227^th place)

cloudflare.com (Global: 8,991^st place; Japanese: low place)

doi.org (Global: 2^nd place; Japanese: 6^th place)

github.com (Global: 383^rd place; Japanese: 521^st place)

googleblog.com (Global: 1,272^nd place; Japanese: 1,494^th place)

impress.co.jp (Global: 648^th place; Japanese: 46^th place)

infoworld.com (Global: 3,696^th place; Japanese: 7,892^nd place)

itmedia.co.jp (Global: 624^th place; Japanese: 41^st place)

medium.com (Global: 551^st place; Japanese: 1,605^th place)

microsoft.com (Global: 153^rd place; Japanese: 260^th place)

nvidia.com (Global: 2,503^rd place; Japanese: 2,605^th place)

oracle.com (Global: 1,514^th place; Japanese: 1,491^st place)

oreilly.com (Global: 4,347^th place; Japanese: low place)

phoronix.com (Global: 4,683^rd place; Japanese: 9,830^th place)

techcrunch.com (Global: 187^th place; Japanese: 440^th place)

towardsdatascience.com (Global: 8,920^th place; Japanese: low place)

venturebeat.com (Global: 616^th place; Japanese: 2,168^th place)

web.archive.org (Global: 1^st place; Japanese: 1^st place)