--- title: Tensor is the might | daily.dev description: A deep walkthrough of building a complete tensor library from scratch in C, covering everything from basic tensor data structures and strides, to GPU... canonical: https://daily.dev/posts/tensor-is-the-might-v87pxfcye twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Tensor is the might | daily.dev og:description: A deep walkthrough of building a complete tensor library from scratch in C, covering everything from basic tensor data structures and strides, to GPU... og:url: https://daily.dev/posts/tensor-is-the-might-v87pxfcye og:image: https://api.daily.dev/og/posts/V87pXFcyE.png og:image:alt: Tensor is the might og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Tensor is the might **[Hacker News](https://daily.dev/sources/hn)** · 24 min read · 0 upvotes · 0 comments ## Summary A deep walkthrough of building a complete tensor library from scratch in C, covering everything from basic tensor data structures and strides, to GPU acceleration via Apple Metal, matrix multiplication with BLAS/cblas_sgemm, manual backpropagation, cross-entropy loss, SGD optimizer, and finally training a 2-layer MLP on MNIST achieving ~97% test accuracy. The author benchmarks it against PyTorch (2x faster per epoch on M1), discusses design tradeoffs like header-only single-file layout, memory management with reference counting, and outlines future directions including autograd via a tape mechanism and OpenCL backend support. The full library (utensil) is open-sourced on GitHub. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much faster is a custom C tensor library compared to PyTorch for training a simple MLP on MNIST? A hand-built C tensor library trained an MLP on MNIST in about 0.8 seconds per epoch on an M1 MacBook, versus almost 2 seconds per epoch for PyTorch at the same accuracy. On larger models, the C implementation was observed to run roughly 2-4x faster than PyTorch, especially when operations are properly fused. _Comparing framework overhead against hand-rolled implementations is easier with daily.dev when researching ML performance tradeoffs._ ### How can cblas_sgemm speed up matrix multiplication on Apple Silicon compared to a naive triple-loop implementation? Using cblas_sgemm from Apple's Accelerate framework leverages the AMX matrix co-processor and runs 50 to 100 times faster than a naive three-nested-loop matrix multiplication for any matrix above 32x32. It serves as a drop-in CPU replacement before considering GPU-based matmul through Metal's MPSMatrix for larger matrices. _Developers optimizing custom ML kernels can keep tabs on Apple Accelerate techniques through daily.dev._ ### How do you compute the gradient for a linear layer's weight and input during manual backpropagation? Given a forward pass y = x@w, the gradient with respect to the input is dldx = dldy @ w.T, and the gradient with respect to the weights is dldw = x.T @ dldy. If a bias term is included, its gradient dldb equals dldy summed across the batch dimension, since bias addition passes gradients through unchanged. _Anyone implementing backprop by hand can find similar deep dives on daily.dev while building autograd systems._ ## Similar posts on daily.dev - [0byte](https://daily.dev/posts/0byte-1ywh7bopx) · Hacker News · 0 upvotes · 0 comments - [PyTorch Tutorial for Deep Learning](https://daily.dev/posts/pytorch-tutorial-for-deep-learning-hxdgznfc2) · JetBrains · 0 upvotes · 0 comments - [Writing An Optimizing Tensor Compiler From Scratch](https://daily.dev/posts/writing-an-optimizing-tensor-compiler-from-scratch-qu2lwyx6h) · Hackaday · 0 upvotes · 0 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#c](https://daily.dev/tags/c), [#neural-networks](https://daily.dev/tags/neural-networks) [View this post on daily.dev](https://daily.dev/posts/tensor-is-the-might-v87pxfcye) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Tensor is the might","url":"https://daily.dev/posts/tensor-is-the-might-v87pxfcye","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/tensor-is-the-might-v87pxfcye"},"datePublished":"2026-07-14T15:19:04.860Z","dateModified":"2026-09-14T08:28:26.265Z","description":"A deep walkthrough of building a complete tensor library from scratch in C, covering everything from basic tensor data structures and strides, to GPU...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e20fa7d2746e912df3ae19135c9dcbc3?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e20fa7d2746e912df3ae19135c9dcbc3?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/tensor-is-the-might-v87pxfcye","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,c,neural-networks","timeRequired":"PT24M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Tensor is the might"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/tensor-is-the-might-v87pxfcye#faq","mainEntity":[{"@type":"Question","name":"How much faster is a custom C tensor library compared to PyTorch for training a simple MLP on MNIST?","acceptedAnswer":{"@type":"Answer","text":"A hand-built C tensor library trained an MLP on MNIST in about 0.8 seconds per epoch on an M1 MacBook, versus almost 2 seconds per epoch for PyTorch at the same accuracy. On larger models, the C implementation was observed to run roughly 2-4x faster than PyTorch, especially when operations are properly fused. Comparing framework overhead against hand-rolled implementations is easier with daily.dev when researching ML performance tradeoffs."}},{"@type":"Question","name":"How can cblas_sgemm speed up matrix multiplication on Apple Silicon compared to a naive triple-loop implementation?","acceptedAnswer":{"@type":"Answer","text":"Using cblas_sgemm from Apple's Accelerate framework leverages the AMX matrix co-processor and runs 50 to 100 times faster than a naive three-nested-loop matrix multiplication for any matrix above 32x32. It serves as a drop-in CPU replacement before considering GPU-based matmul through Metal's MPSMatrix for larger matrices. Developers optimizing custom ML kernels can keep tabs on Apple Accelerate techniques through daily.dev."}},{"@type":"Question","name":"How do you compute the gradient for a linear layer's weight and input during manual backpropagation?","acceptedAnswer":{"@type":"Answer","text":"Given a forward pass y = x@w, the gradient with respect to the input is dldx = dldy @ w.T, and the gradient with respect to the weights is dldw = x.T @ dldy. If a bias term is included, its gradient dldb equals dldy summed across the batch dimension, since bias addition passes gradients through unchanged. Anyone implementing backprop by hand can find similar deep dives on daily.dev while building autograd systems."}}]} ```