This post explains the internals of PyTorch, including GPU support and automatic differentiation. It also demonstrates how to recreate PyTorch from scratch, including building a tensor library, implementing modules for neural networks, and creating custom optimizers.
Table of contents
Recreating PyTorch from Scratch (with GPU Support and Automatic Differentiation)Introduction#1 — Tensor#2 — GPU Support#3 — Automatic Differentation (Autograd)#4 — nn and optim modulesConclusionReferences33 Impressions