PyTorch (Japanese Wikipedia)

PETALS: Collaborative Inference and Fine-tuning of Large Models p.2 Alexander Borzunov et al. 2023年
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU p.6 Ying Sheng et al. 2023年
"torch.fx, a program capture and transformation library for PyTorch" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"torch.fx represents programs in a DAG-based IR" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"graph-based ... IR ... Program transformations ... is as simple embedded programming languages that are meta-programmed from a host language, predominantly Python" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"torch.fx provides an fx.graph_drawer package, which gives the user the ability to visualize torch.fx graphs with Graphviz" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"Quantization makes use of torch.fx’s graph and GraphModule representation to simultaneously modify the program code and weight values." Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"6.2.2 Fusion Optimizations ... torch.fx provides the necessary non-local program context and state modification facilities needed for this transformation with its ahead-of-time, graph-based nature" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
" The project was quickly developed using torch.fx’s Python APIs as well as TensorRT’s Python APIs, creating a translation layer between the two." Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.
"the purposes of serialization or export. For instance, TorchScript" Reed, et al. (2021). Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python. arxiv.

chainer.org (Global: low place; Japanese: low place)

Chainer Team (2010年12月5日). “Chainer/CuPy v7のリリースと今後の開発体制について”. Chainer Blog. 2020年8月10日閲覧。

cloudflare.com (Global: low place; Japanese: low place)

blog.cloudflare.com

Cloudflare R2 and MosaicML enable training LLMs on any compute, anywhere in the world, with zero switching costs Cloudflare 2023年5月16日

deepspeed.ai (Global: low place; Japanese: low place)

DeepSpeed-MII: instant speedup on 24,000+ open-source DL models with up to 40x cheaper inference Microsoft 2022年10月10日
ZeRO++ Microsoft
ZeRO-Offload Microsoft
Pipeline Parallelism - DeepSpeed Microsoft
Mixture of Experts - DeepSpeed Microsoft
ZeRO-Inference: Democratizing massive model inference Microsoft 2022年9月9日

deepspeed.readthedocs.io (Global: low place; Japanese: low place)

ZeRO Microsoft

dl4nlp.info (Global: low place; Japanese: low place)

“Natural Language Processing (NLP) with PyTorch — NLP with PyTorch documentation” (英語). dl4nlp.info. 2019年3月30日閲覧。

doi.org (Global: 2^nd place; Japanese: 3^rd place)

Ketkar, Nikhil (2017). “Introduction to PyTorch” (英語). Deep Learning with Python. Apress, Berkeley, CA. pp. 195–208. doi:10.1007/978-1-4842-2766-4_12. ISBN 9781484227657

fairscale.readthedocs.io (Global: low place; Japanese: low place)

Efficient Memory management - FairScale documentation Facebook Research
Pipeline Parallelism - FairScale documentation Facebook Research

github.com (Global: 380^th place; Japanese: 404^th place)

“Release 2.14.0” (2026年9月2日). 2026年9月3日閲覧。
GitHub - catalyst-team/catalyst: Accelerated DL & RL, Catalyst-Team, (2019-12-05) 2019年12月5日閲覧。
PiPPy PyTorch Foundation
"TorchScript is executed using an interpreter attached to a JIT-optimizer and compiler." PyTorch. JIT Technical Overview. 2022-03-19閲覧.
"The executor specializes the Graph for the particular set of inputs." PyTorch. JIT Technical Overview. 2022-03-19閲覧.
"It propagates constants, pre-computing as much as possible" PyTorch. JIT Technical Overview. 2022-03-19閲覧.
"Eliminating dead code" PyTorch. JIT Technical Overview. 2022-03-19閲覧.
Fairseq Meta AI
TorchRL PyTorch Foundation

googleblog.com (Global: 3,285^th place; Japanese: 2,035^th place)

developers.googleblog.com

AI Edge Torch: High Performance Inference of PyTorch Models on Mobile Devices Google 2024年5月14日

habana.ai (Global: low place; Japanese: low place)

docs.habana.ai

Intel Gaudi PyTorch Python API (habana_frameworks.torch) — Gaudi Documentation 1.18.0 documentation Intel

huggingface.co (Global: low place; Japanese: low place)

Model Parallelism — transformers 4.11.3 documentation Hugging Face
Handling big models for inference HuggingFace
How 🤗 Accelerate runs very large models thanks to PyTorch HuggingFace 2022年9月27日
Pickle Scanning HuggingFace
BetterTransformer - Overview Hugging Face
AltDiffusion Hugging Face
Summary of the tokenizers Hugging Face
Integrate your library with the Hub Hugging Face

impress.co.jp (Global: 543^rd place; Japanese: 39^th place)

pc.watch.impress.co.jp

NVIDIA Ampereにおけるプルーニング対応の特徴 Impress 2020年7月20日

infoworld.com (Global: 5,156^th place; Japanese: 6,297^th place)

Yegulalp, Serdar (19 January 2017). “Facebook brings GPU-powered machine learning to Python”. InfoWorld. 2019年3月30日閲覧.

itmedia.co.jp (Global: 670^th place; Japanese: 49^th place)

動画も音楽もゲームも　広がるAIエンターテインメントの可能性 ITmedia 2022年11月11日

kaitai.io (Global: low place; Japanese: low place)

formats.kaitai.io

Python pickle serialization format: format specification Kaitai Project

medium.com (Global: 651^st place; Japanese: 1,429^th place)

Synced (2018年4月2日). “Caffe2 Merges With PyTorch”. 2019年3月30日閲覧。

microsoft.com (Global: 263^rd place; Japanese: 228^th place)

ZeRO-Infinity and DeepSpeed: Unlocking unprecedented model scale for deep learning training Microsoft

nod.ai (Global: low place; Japanese: low place)

My spring internship – torch-mlir eager mode, OPT and blowing away the main git repo nod.ai 2022年6月20日

nvidia.com (Global: 2,958^th place; Japanese: 2,190^th place)

oracle.com (Global: 2,532^nd place; Japanese: 1,769^th place)

blogs.oracle.com

Train and deploy generative AI faster with MosaicML and Oracle オラクル 2023年5月16日

oreilly.com (Global: 6,944^th place; Japanese: 9,560^th place)

Lorica, Ben (2017年8月3日). “Why AI and machine learning researchers are beginning to embrace PyTorch”. O'Reilly Media. 2019年3月30日閲覧。
Patel, Mo (2017年12月7日). “When two trends fuse: PyTorch and recommender systems”. O'Reilly Media (英語). 2019年3月30日閲覧.

phoronix.com (Global: 4,058^th place; Japanese: 4,580^th place)

PyTorch 2.5 Released With Improved Intel GPU Support Phoronix 2024年10月17日

pypi.org (Global: 3,588^th place; Japanese: low place)

fairseq · PyPI Facebook

pytorch.org (Global: low place; Japanese: low place)

PYTORCH-TRANSFORMERS: PyTorch implementations of popular NLP Transformers, PyTorch Hub, (2019-12-01) 2019年12月1日閲覧。
“PyTorch” (英語). www.pytorch.org. 2019年12月5日閲覧。
“PyTorch – About”. pytorch.org. 2018年6月15日時点のオリジナルよりアーカイブ。2018年6月11日閲覧。
PyTorch 2.1: automatic dynamic shape compilation, distributed checkpointing
torch.tensor PyTorch Foundation
Autograd PyTorch Foundation
Complex Numbers PyTorch Foundation
Quantization PyTorch Foundation
Sparse Semi-Structured Tensors PyTorch Foundation
Module PyTorch Foundation
“The C++ Frontend”. PyTorch Master Documentation. 2019年7月29日閲覧。
Tensor Basics PyTorch Foundation
Getting Started on Intel GPU PyTorch Foundation
Facilitating New Backend Integration by PrivateUse1 PyTorch Foundation
Challenges and Efforts in PyTorch Multi-Device Integration: Compatibility, Portability, and Integration Efficiencies PyTorch Foundation 2024年9月18日
PyTorch 2.0 PyTorch Foundation
Installation PyTorch Foundation
Getting Started with Distributed Data Parallel — PyTorch Tutorials 2.3.0+cu121 documentation PyTorch Foundation
Introducing PyTorch Fully Sharded Data Parallel (FSDP) API PyTorch Foundation 2022年3月14日
Pipeline Parallelism PyTorch Foundation
Tensor Parallelism - torch.distributed.tensor.parallel PyTorch Foundation
Docs > CUDA semantics - CUDA streams PyTorch Foundation
torch.onnx PyTorch Foundation
"TorchScript is a way to create serializable and optimizable models from PyTorch code." PyTorch. TorchScript.
"TorchScript is a statically typed subset of Python" PyTorch. TorchScript Language Reference.
"TorchScript program that can be run independently from Python, such as in a standalone C++ program." PyTorch. TorchScript.
"In PyTorch 1.10, we’ve added an LLVM-based JIT compiler for CPUs that can fuse together sequences of torch library calls" PyTorch. PyTorch 1.10 Release.
Saving and Loading Models — PyTorch Tutorials 1.12.1+cu102 documentation PyTorch Foundation
torch.onnx — PyTorch 1.13 documentation PyTorch Foundation
"Scripting a function or nn.Module will inspect the source code, compile it as TorchScript code using the TorchScript compiler, and return a ScriptModule or ScriptFunction." PyTorch. TORCH.JIT.SCRIPT. pytorch.org. 2023-08-29閲覧.
"Using torch.jit.trace and torch.jit.trace_module, you can turn an existing module or Python function into a TorchScript ScriptFunction or ScriptModule. You must provide example inputs, and we run the function, recording the operations performed on all the tensors." Python. TORCH.JIT.TRACE. pytorch.org. 2023-08-29閲覧.
"Tracing will not record any control-flow like if-statements or loops." Python. TORCH.JIT.TRACE. pytorch.org. 2023-08-29閲覧.
"FX is a toolkit for developers to use to transform nn.Module instances." PyTorch. TORCH.FX. 2022-03-23閲覧.
"FX consists of three main components: a symbolic tracer, an intermediate representation, and Python code generation." PyTorch. TORCH.FX. 2022-03-23閲覧.
functorch.compile (experimental) — functorch 1.13 documentation PyTorch Foundation
functorch.compile.ts_compile — functorch 1.13 documentation PyTorch Foundation
TorchDynamo(torch.compile) integration in PyTorch XLA PyTorch Foundation
The Path to Achieve Ultra-Low Inference Latency With LLaMA 65B on PyTorch/XLA PyTorch Foundation
Quantization PyTorch Foundation
PyTorch Mobile PyTorch Foundation
ExecuTorch PyTorch Foundation
torch.utils.mobile_optimizer PyTorch Foundation
torch.nn - Transformer Layers PyTorch Foundation
New Releases: PyTorch 1.2, torchtext 0.4, torchaudio 0.3, and torchvision 0.4 PyTorch Foundation 2019年8月8日
PyTorch 2.0: Our next generation release that is faster, more Pythonic and Dynamic as ever PyTorch Foundation 2023年3月15日
A BetterTransformer for Fast Transformer Inference PyTorch Foundation 2022年7月12日
Models and pre-trained weights — Torchvision main documentation PyTorch Foundation
New Library Updates in PyTorch 2.1 PyTorch Foundation 2023年10月4日
torchaudio.pipelines PyTorch Foundation
SST-2 Binary text classification with XLM-RoBERTa model PyTorch Foundation
T5-Base Model for Summarization, Sentiment Classification, and Translation PyTorch Foundation
torchtext.transforms PyTorch Foundation
torchtext.vocab PyTorch Foundation
torchtext PyTorch Foundation
Introducing torchchat: Accelerating Local LLM Inference on Laptop, Desktop and Mobile PyTorch Foundation 2024年7月30日
Accelerating Generative AI with PyTorch II: GPT, Fast PyTorch Foundation 2023年11月30日
torch.hub — PyTorch 1.13 documentation PyTorch Foundation
PyTorch 1.12: TorchArrow, Functional API for Modules and nvFuser, are now available PyTorch Foundation 2022年6月28日
"torch.fx.symbolic_trace() ... can’t handle control flow" PyTorch. Why do you need another way of optimizing PyTorch code? - Getting Started - TorchDynamo Overview. pytorch.org. 2023-08-28閲覧.

techcrunch.com (Global: 317^th place; Japanese: 483^rd place)

Mannes, John. “Facebook and Microsoft collaborate to simplify conversions from PyTorch to Caffe2”. TechCrunch (英語). 2019年3月30日閲覧. FAIR is accustomed to working with PyTorch — a deep learning framework optimized for achieving state of the art results in research, regardless of resource constraints. Unfortunately in the real world, most of us are limited by the computational capabilities of our smartphones and computers.

towardsdatascience.com (Global: low place; Japanese: low place)

DeepSpeed Deep Dive — Model Implementations for Inference (MII) Towards Data Science 2022年11月17日

uber.com (Global: low place; Japanese: low place)

eng.uber.com

“Uber AI Labs Open Sources Pyro, a Deep Probabilistic Programming Language”. Uber Engineering Blog (アメリカ英語). 2017年11月3日. 2017年12月18日閲覧.

ultralytics.com (Global: low place; Japanese: low place)

docs.ultralytics.com

PyTorch Hub Ultralytics

venturebeat.com (Global: 831^st place; Japanese: 1,684^th place)

Arakelyan, Sophia (2017年11月29日). “Tech giants are using open source frameworks to dominate the AI community” (英語). VentureBeat. 2019年3月30日閲覧。

wandb.ai (Global: low place; Japanese: low place)

Hugging Face Introduces Private Hub Weights & Biases 2022年8月3日

web.archive.org (Global: 1^st place; Japanese: 1^st place)

“PyTorch – About”. pytorch.org. 2018年6月15日時点のオリジナルよりアーカイブ。2018年6月11日閲覧。

PyTorch (Japanese Wikipedia)

amazon.com (Global: 159th place; Japanese: 742nd place)

aws.amazon.com

analyticsvidhya.com (Global: low place; Japanese: low place)

arxiv.org (Global: 49th place; Japanese: 115th place)

chainer.org (Global: low place; Japanese: low place)

cloudflare.com (Global: low place; Japanese: low place)

blog.cloudflare.com

deepspeed.ai (Global: low place; Japanese: low place)

deepspeed.readthedocs.io (Global: low place; Japanese: low place)

dl4nlp.info (Global: low place; Japanese: low place)

doi.org (Global: 2nd place; Japanese: 3rd place)

fairscale.readthedocs.io (Global: low place; Japanese: low place)

github.com (Global: 380th place; Japanese: 404th place)

googleblog.com (Global: 3,285th place; Japanese: 2,035th place)

developers.googleblog.com

habana.ai (Global: low place; Japanese: low place)

docs.habana.ai

huggingface.co (Global: low place; Japanese: low place)

impress.co.jp (Global: 543rd place; Japanese: 39th place)

pc.watch.impress.co.jp

infoworld.com (Global: 5,156th place; Japanese: 6,297th place)

itmedia.co.jp (Global: 670th place; Japanese: 49th place)

kaitai.io (Global: low place; Japanese: low place)

formats.kaitai.io

medium.com (Global: 651st place; Japanese: 1,429th place)

microsoft.com (Global: 263rd place; Japanese: 228th place)

nod.ai (Global: low place; Japanese: low place)

nvidia.com (Global: 2,958th place; Japanese: 2,190th place)

docs.nvidia.com

developer.nvidia.com

catalog.ngc.nvidia.com

oracle.com (Global: 2,532nd place; Japanese: 1,769th place)

blogs.oracle.com

oreilly.com (Global: 6,944th place; Japanese: 9,560th place)

phoronix.com (Global: 4,058th place; Japanese: 4,580th place)

pypi.org (Global: 3,588th place; Japanese: low place)

pytorch.org (Global: low place; Japanese: low place)

techcrunch.com (Global: 317th place; Japanese: 483rd place)

towardsdatascience.com (Global: low place; Japanese: low place)

uber.com (Global: low place; Japanese: low place)

eng.uber.com

ultralytics.com (Global: low place; Japanese: low place)

docs.ultralytics.com

venturebeat.com (Global: 831st place; Japanese: 1,684th place)

wandb.ai (Global: low place; Japanese: low place)

web.archive.org (Global: 1st place; Japanese: 1st place)

amazon.com (Global: 159^th place; Japanese: 742^nd place)

arxiv.org (Global: 49^th place; Japanese: 115^th place)

doi.org (Global: 2^nd place; Japanese: 3^rd place)

github.com (Global: 380^th place; Japanese: 404^th place)

googleblog.com (Global: 3,285^th place; Japanese: 2,035^th place)

impress.co.jp (Global: 543^rd place; Japanese: 39^th place)

infoworld.com (Global: 5,156^th place; Japanese: 6,297^th place)

itmedia.co.jp (Global: 670^th place; Japanese: 49^th place)

medium.com (Global: 651^st place; Japanese: 1,429^th place)

microsoft.com (Global: 263^rd place; Japanese: 228^th place)

nvidia.com (Global: 2,958^th place; Japanese: 2,190^th place)

oracle.com (Global: 2,532^nd place; Japanese: 1,769^th place)

oreilly.com (Global: 6,944^th place; Japanese: 9,560^th place)

phoronix.com (Global: 4,058^th place; Japanese: 4,580^th place)

pypi.org (Global: 3,588^th place; Japanese: low place)

techcrunch.com (Global: 317^th place; Japanese: 483^rd place)

venturebeat.com (Global: 831^st place; Japanese: 1,684^th place)

web.archive.org (Global: 1^st place; Japanese: 1^st place)